Pickling unbound methods on Python 3

[cool-RR]

Hello,

I've had a problem when porting my project from Python 2.6 to Python 3.x.

At some point in my project I need to be able to pickle unbound methods.

In Python 2.x it's impossible to pickle unbound methods by default; I
managed to do it there in some weird way I don't understand: I wrote a
reducer with the `copy_reg` module for the MethodType class, which covers
both bound and unbound methods. But the reducer only solved the case of the
bound method, because it depended on `my_method.im_self`. Mysteriously it
has also caused Python 2.x to be able to pickle unbound methods. This does
not happen on Python 3.x.

When trying to pickle a method, I'm getting this error:

>>> class A:
... def m(self):
... pass
>>> import pickle
>>> pickle.dumps(A.m)
Traceback (most recent call last):
File "<pyshell#3>", line 1, in <module>
pickle.dumps(A.m)
File "C:\Python31\lib\pickle.py", line 1358, in dumps
Pickler(f, protocol, fix_imports=fix_imports).dump(obj)
_pickle.PicklingError: Can't pickle <class 'function'>: attribute lookup
builtins.function failed

How can this been solved?

One person told me that given an unbound method in Python 3.x, it's
*impossible* to tell to which class it belongs. Is it true?

Ram.

[R. David Murray]

On Fri, 28 May 2010 21:55:28 +0200, cool-RR wrote:
> One person told me that given an unbound method in Python 3.x, it's
> *impossible* to tell to which class it belongs. Is it true?

I believe that that is true. In Python3 there is no such thing as an
unbound method as a distinct object type. There are functions, and
there are bound methods. See the second sentence in this section:

http://docs.python.org/release/3.0.1/whatsnew/3.0.html#operators-and-special-methods

You'll probably have to explain more about the problem you are
trying to solve in order for us to help you find a solution.

--
R. David Murray www.bitdance.com

[cool-RR]

On Sat, May 29, 2010 at 5:46 PM, R. David Murray <rdmurray-2P5OLIWfD11Wk0Htik3J/***@public.gmane.org>wrote:

> On Fri, 28 May 2010 21:55:28 +0200, cool-RR wrote:
> > One person told me that given an unbound method in Python 3.x, it's
> > *impossible* to tell to which class it belongs. Is it true?
>
> I believe that that is true. In Python3 there is no such thing as an
> unbound method as a distinct object type. There are functions, and
> there are bound methods. See the second sentence in this section:
>
>
> http://docs.python.org/release/3.0.1/whatsnew/3.0.html#operators-and-special-methods


I see. Where would be a good place to discuss this decision? I would want
3.2 to allow pickling of unbound methods.



> You'll probably have to explain more about the problem you are
> trying to solve in order for us to help you find a solution.
>
> --
> R. David Murray www.bitdance.com
>

I found a hacky workaround: When I have a method `my_method` that I want to
pickle, I write `my_method = my_class.my_method` in the method's module.

So no need to burden this list with the specifics of my problem.

Now I only worry about the future: I want 3.2 to enable pickling of unbound
methods so I could get rid of this workaround in the future.

Ram.

[M.-A. Lemburg]

cool-RR wrote:
> On Sat, May 29, 2010 at 5:46 PM, R. David Murray <rdmurray-2P5OLIWfD11Wk0Htik3J/***@public.gmane.org>wrote:
>
>> On Fri, 28 May 2010 21:55:28 +0200, cool-RR wrote:
>>> One person told me that given an unbound method in Python 3.x, it's
>>> *impossible* to tell to which class it belongs. Is it true?
>>
>> I believe that that is true. In Python3 there is no such thing as an
>> unbound method as a distinct object type. There are functions, and
>> there are bound methods. See the second sentence in this section:
>>
>>
>> http://docs.python.org/release/3.0.1/whatsnew/3.0.html#operators-and-special-methods
>
>
> I see. Where would be a good place to discuss this decision? I would want
> 3.2 to allow pickling of unbound methods.

Unbound methods don't exist in Python3. You only have functions and
(bound) methods.

You can see that if you try to call an unbound method with a
non-instance first arg:

>>> class X:
... def test(self): return 42
...
>>> X.test(X())
42
>>> X.test(3)
42

Doing the same in Python2 gives an error:

>>> X.test(X())
42
>>> X.test(3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unbound method test() must be called with X instance as first argument (got int instance
instead)

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source (#1, May 29 2010)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
2010-07-19: EuroPython 2010, Birmingham, UK 50 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/

[cool-RR]

On Sat, May 29, 2010 at 6:56 PM, M.-A. Lemburg <mal-SVD0I98eSHvQT0dZR+***@public.gmane.org> wrote:

> cool-RR wrote:
> > On Sat, May 29, 2010 at 5:46 PM, R. David Murray <rdmurray-2P5OLIWfD11Wk0Htik3J/***@public.gmane.org
> >wrote:
> >
> >> On Fri, 28 May 2010 21:55:28 +0200, cool-RR wrote:
> >>> One person told me that given an unbound method in Python 3.x, it's
> >>> *impossible* to tell to which class it belongs. Is it true?
> >>
> >> I believe that that is true. In Python3 there is no such thing as an
> >> unbound method as a distinct object type. There are functions, and
> >> there are bound methods. See the second sentence in this section:
> >>
> >>
> >>
> http://docs.python.org/release/3.0.1/whatsnew/3.0.html#operators-and-special-methods
> >
> >
> > I see. Where would be a good place to discuss this decision? I would want
> > 3.2 to allow pickling of unbound methods.
>
>
Unbound methods don't exist in Python3. You only have functions and
> (bound) methods.
>

I know that unbound methods are of the function type in Python 3. I'm
calling them unbound methods because they are methods of classes which are
not bound to an instance. I'm aware they have no special stance, but I still
refer to them as unbound methods.

You can see that if you try to call an unbound method with a
> non-instance first arg:
>
> >>> class X:
> ... def test(self): return 42
> ...
> >>> X.test(X())
> 42
> >>> X.test(3)
> 42
>
> Doing the same in Python2 gives an error:
>
> >>> X.test(X())
> 42
> >>> X.test(3)
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> TypeError: unbound method test() must be called with X instance as first
> argument (got int instance
> instead)
>

Thanks for the interesting example, Marc-Andre.

Ram.

[Martin v. Löwis]

> I know that unbound methods are of the function type in Python 3. I'm
> calling them unbound methods because they are methods of classes which
> are not bound to an instance. I'm aware they have no special stance, but
> I still refer to them as unbound methods.

Neglecting reality doesn't help your cause, though. You'll need to
understand *why* pickling fails, and then see whether there might be
a solution.

A work-around is to put all methods you want to pickle into module-scope
of your module

class Foo:
def bar(self):
pass

bar = Foo.bar

Now you can pickle Foo.bar.

Regards,
Martin

[cool-RR]

Hello Martin, thanks for joining the discussion.

On Sat, May 29, 2010 at 8:43 PM, "Martin v. Löwis" <***@v.loewis.de>wrote:

> I know that unbound methods are of the function type in Python 3. I'm
>> calling them unbound methods because they are methods of classes which
>> are not bound to an instance. I'm aware they have no special stance, but
>> I still refer to them as unbound methods.
>>
>
> Neglecting reality doesn't help your cause, though. You'll need to
> understand *why* pickling fails, and then see whether there might be
> a solution.
>

I think I understand why pickling fails. An unbound method is
not differentiated in any way from a function. The `save_global` function
tries to look for it in the main module namespace, instead of in the class.



> A work-around is to put all methods you want to pickle into module-scope
> of your module
>
> class Foo:
> def bar(self):
> pass
>
> bar = Foo.bar
>
> Now you can pickle Foo.bar.
>

Yes, I've already done this, but I'd prefer a less hackish solution.



> Regards,
> Martin
>

I can think of a few solutions:

1. Make `save_global` look in the classes inside the module as well.

2. Allow me to specify a reducer for FunctionType. This is currently not
working, see here:
http://stackoverflow.com/questions/2932742/python-using-copyreg-to-define-reducers-for-types-that-already-have-reducers


Ram.

[Martin v. Löwis]

> I can think of a few solutions:
>
> 1. Make `save_global` look in the classes inside the module as well.

How exactly would you do this? I don't think this is implementable,
in a reasonable way.

> 2. Allow me to specify a reducer for FunctionType.

How would the reducer work?

Regards,
Martin

[cool-RR]

On Sun, May 30, 2010 at 9:19 AM, "Martin v. Löwis" <***@v.loewis.de>wrote:

> I can think of a few solutions:
>>
>> 1. Make `save_global` look in the classes inside the module as well.
>>
>
> How exactly would you do this? I don't think this is implementable,
> in a reasonable way.
>

I don't understand what the problem is with just looking in the classes
defined in the module:

"""
def find_containing_object_of_function(function):
function_name = function.__name__
module = sys.modules[function.__module__]
objects_to_check = [module]
while objects_to_check:
object_to_check = objects_to_check.pop()
if getattr(object_to_check, function_name, None) is function:
return object_to_check
try:
sub_objects = vars(object_to_check)
except TypeError: # The object doesn't have a vars/__dict__
continue
for sub_object in sub_objects.values():
if isinstance(sub_object, type):
objects_to_check.append(sub_object)

raise LookupError('''Couldn't find the %s function for
pickling/deepcopying \
it. I tried looking around in the %s module, but it either wasn't there or \
was in a non-obvious place.''' % (function, module))


def reduce_function(function):
containing_object = find_containing_object_of_function(function)
return (getattr, (containing_object, function.__name__))
"""

This is not bulletproof, of course. But it'll work for most cases.


> 2. Allow me to specify a reducer for FunctionType.
>>
>
> How would the reducer work?
>
>
As above.

Ram.

[Lennart Regebro]

On Sun, May 30, 2010 at 12:00, cool-RR <cool-rr-***@public.gmane.org> wrote:
> I don't understand what the problem is with just looking in the classes
> defined in the module:

How do you handle name conflicts?

--
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
[phone]

[cool-RR]

On Sun, May 30, 2010 at 12:50 PM, Lennart Regebro <regebro-***@public.gmane.org> wrote:

> On Sun, May 30, 2010 at 12:00, cool-RR <cool-rr-***@public.gmane.org> wrote:
> > I don't understand what the problem is with just looking in the classes
> > defined in the module:
>
> How do you handle name conflicts?
>

Can you specify which kind of naming conflict you mean?


Ram.

[Lennart Regebro]

On Sun, May 30, 2010 at 12:52, cool-RR <cool-rr-***@public.gmane.org> wrote:
> On Sun, May 30, 2010 at 12:50 PM, Lennart Regebro <regebro-***@public.gmane.org> wrote:
>>
>> On Sun, May 30, 2010 at 12:00, cool-RR <cool-rr-***@public.gmane.org> wrote:
>> > I don't understand what the problem is with just looking in the classes
>> > defined in the module:
>>
>> How do you handle name conflicts?
>
> Can you specify which kind of naming conflict you mean?

Two methods with the same name but in different classes would get a
name class. They would be unpickled as the same method, leading to a
lot of really hairy debugging. Especially if they take the same
parameters, which seems likely in your case. You would simply unpickle
and call the wrong method, with no errors, just unexpected behavior.


Or in other words: Your solution is way more hackish and brittle than
the one Martin suggested, which is simple, neat and Pythonic.

--
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
[phone]

[cool-RR]

On Sun, May 30, 2010 at 12:58 PM, Lennart Regebro <regebro-***@public.gmane.org> wrote:

> On Sun, May 30, 2010 at 12:52, cool-RR <cool-rr-***@public.gmane.org> wrote:
> > On Sun, May 30, 2010 at 12:50 PM, Lennart Regebro <regebro-***@public.gmane.org>
> wrote:
> >>
> >> On Sun, May 30, 2010 at 12:00, cool-RR <cool-rr-***@public.gmane.org> wrote:
> >> > I don't understand what the problem is with just looking in the
> classes
> >> > defined in the module:
> >>
> >> How do you handle name conflicts?
> >
> > Can you specify which kind of naming conflict you mean?
>
> Two methods with the same name but in different classes would get a
> name class. They would be unpickled as the same method, leading to a
> lot of really hairy debugging. Especially if they take the same
> parameters, which seems likely in your case. You would simply unpickle
> and call the wrong method, with no errors, just unexpected behavior.
>

I don't understand how that would happen. The code that I gave confirms that
the function found inside the class `is` the function that is being pickled,
so if it finds a different function with the same name, it will ignore it.
Or maybe I'm misunderstanding something?



> Or in other words: Your solution is way more hackish and brittle than
> the one Martin suggested, which is simple, neat and Pythonic.


I disagree, Lennart.


Ram.

[Lennart Regebro]

On Sun, May 30, 2010 at 13:02, cool-RR <cool-rr-***@public.gmane.org> wrote:
> I don't understand how that would happen. The code that I gave confirms that
> the function found inside the class `is` the function that is being pickled,
> so if it finds a different function with the same name, it will ignore it.

How can it do that? How does it know that it should ignore it?


But you are right, Martins solution wasn't as neat as I thought first,
you basically need to do it the other way around, ie define the
methods as functions, and then set them on the classes. Probably
wrapper methods is neater and less hacky.

--
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
[phone]

[cool-RR]

On Sun, May 30, 2010 at 1:33 PM, Lennart Regebro <regebro-***@public.gmane.org> wrote:

> On Sun, May 30, 2010 at 13:02, cool-RR <cool-rr-***@public.gmane.org> wrote:
> > I don't understand how that would happen. The code that I gave confirms
> that
> > the function found inside the class `is` the function that is being
> pickled,
> > so if it finds a different function with the same name, it will ignore
> it.
>
> How can it do that? How does it know that it should ignore it?


I'm not completely sure we're talking about the same thing here. Did you
read the code and try to follow the problematic case?

Ram.

[Lennart Regebro]

On Sun, May 30, 2010 at 13:35, cool-RR <cool-rr-***@public.gmane.org> wrote:
> On Sun, May 30, 2010 at 1:33 PM, Lennart Regebro <regebro-***@public.gmane.org> wrote:
>>
>> On Sun, May 30, 2010 at 13:02, cool-RR <cool-rr-***@public.gmane.org> wrote:
>> > I don't understand how that would happen. The code that I gave confirms
>> > that
>> > the function found inside the class `is` the function that is being
>> > pickled,
>> > so if it finds a different function with the same name, it will ignore
>> > it.
>>
>> How can it do that? How does it know that it should ignore it?
>
> I'm not completely sure we're talking about the same thing here. Did you
> read the code and try to follow the problematic case?

You are right, I misread.

[Martin v. Löwis]

Am 30.05.2010 12:00, schrieb cool-RR:
> On Sun, May 30, 2010 at 9:19 AM, "Martin v. Löwis" <***@v.loewis.de
> <mailto:***@v.loewis.de>> wrote:
>
> I can think of a few solutions:
>
> 1. Make `save_global` look in the classes inside the module as well.
>
>
> How exactly would you do this? I don't think this is implementable,
> in a reasonable way.
>
>
> I don't understand what the problem is with just looking in the classes
> defined in the module:

I wouldn't call this a reasonable implementation. It is fairly expensive.

Regards,
Martin

[cool-RR]

On Sun, May 30, 2010 at 11:34 PM, "Martin v. Löwis" <***@v.loewis.de>wrote:

> Am 30.05.2010 12:00, schrieb cool-RR:
>
>> On Sun, May 30, 2010 at 9:19 AM, "Martin v. Löwis" <***@v.loewis.de
>> <mailto:***@v.loewis.de>> wrote:
>>
>> I can think of a few solutions:
>>
>> 1. Make `save_global` look in the classes inside the module as
>> well.
>>
>>
>> How exactly would you do this? I don't think this is implementable,
>> in a reasonable way.
>>
>>
>> I don't understand what the problem is with just looking in the classes
>> defined in the module:
>>
>
> I wouldn't call this a reasonable implementation. It is fairly expensive.
>
> Regards,
> Martin
>

What if we'll try the expensive stuff after the old way (shallow module
search) fails and is about to raise an exception?

Ram.

[Martin v. Löwis]

> What if we'll try the expensive stuff after the old way (shallow module
> search) fails and is about to raise an exception?

Feel free to submit a patch. However, if this needs to be fixed at all
(and I'm not convinced it does), then I'd rather see a proper fix, i.e.
one where you find the class from the function object directly, or which
has an even more general way of identifying global objects.

Regards,
Martin

[cool-RR]

On Sun, May 30, 2010 at 11:48 PM, "Martin v. Löwis" <***@v.loewis.de>wrote:

> What if we'll try the expensive stuff after the old way (shallow module
>> search) fails and is about to raise an exception?
>>
>
> Feel free to submit a patch. However, if this needs to be fixed at all (and
> I'm not convinced it does), then I'd rather see a proper fix, i.e. one where
> you find the class from the function object directly, or which
> has an even more general way of identifying global objects.
>
> Regards,
> Martin
>

I agree, and that was what I wanted from the beginning. But several people
have told me that it's *impossible* to find the class that an unbound method
was defined on. So I came up with this workaround instead.

I'll consider making a patch. Thanks for your help Martin.

By the way, when this discussion continues / when I make the patch, should I
post on python-dev instead of here?

Ram.

[M.-A. Lemburg]

cool-RR wrote:
> On Sun, May 30, 2010 at 11:48 PM, "Martin v. Löwis" <***@v.loewis.de>wrote:
>
>> What if we'll try the expensive stuff after the old way (shallow module
>>> search) fails and is about to raise an exception?
>>>
>>
>> Feel free to submit a patch. However, if this needs to be fixed at all (and
>> I'm not convinced it does), then I'd rather see a proper fix, i.e. one where
>> you find the class from the function object directly, or which
>> has an even more general way of identifying global objects.
>>
>> Regards,
>> Martin
>>
>
> I agree, and that was what I wanted from the beginning. But several people
> have told me that it's *impossible* to find the class that an unbound method
> was defined on. So I came up with this workaround instead.
>
> I'll consider making a patch. Thanks for your help Martin.
>
> By the way, when this discussion continues / when I make the patch, should I
> post on python-dev instead of here?

python-dev would be more appropriate.

I suppose the easiest way to add the information about the defining
class is to have the class constructor add a function attribute
to the (unbound) method functions. This should be the name of
the defining class to avoid circular references.

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source (#1, May 31 2010)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
2010-07-19: EuroPython 2010, Birmingham, UK 48 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/

[cool-RR]

On Mon, May 31, 2010 at 9:31 AM, M.-A. Lemburg <mal-SVD0I98eSHvQT0dZR+***@public.gmane.org> wrote:
>
> I suppose the easiest way to add the information about the defining
> class is to have the class constructor add a function attribute
> to the (unbound) method functions. This should be the name of
> the defining class to avoid circular references.
>
> --
> Marc-Andre Lemburg
>

As a workaround that'll work, yeah. But we were discussing whether we can
make a real solution. (Unless you were proposing that Python itself will do
this and not just my program; In that case I'm totally with you, but it
seems that the Python community isn't.)

Ram.

[M.-A. Lemburg]

cool-RR wrote:
> On Mon, May 31, 2010 at 9:31 AM, M.-A. Lemburg <mal-SVD0I98eSHvQT0dZR+***@public.gmane.org> wrote:
>>
>> I suppose the easiest way to add the information about the defining
>> class is to have the class constructor add a function attribute
>> to the (unbound) method functions. This should be the name of
>> the defining class to avoid circular references.
>>
>> --
>> Marc-Andre Lemburg
>>
>
> As a workaround that'll work, yeah. But we were discussing whether we can
> make a real solution. (Unless you were proposing that Python itself will do
> this and not just my program; In that case I'm totally with you, but it
> seems that the Python community isn't.)

I was thinking of having this in Python itself. The compiler has
all the information available, so why not add some of this useful
meta-information to the created objects ?!

This is not the same as creating a new bound method type - we'd
only add some extra information for function objects that happen
to be defined as methods.

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source (#1, May 31 2010)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
2010-07-19: EuroPython 2010, Birmingham, UK 48 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/

[cool-RR]

On Mon, May 31, 2010 at 12:18 PM, M.-A. Lemburg <mal-SVD0I98eSHvQT0dZR+***@public.gmane.org> wrote:

> cool-RR wrote:
> > On Mon, May 31, 2010 at 9:31 AM, M.-A. Lemburg <mal-SVD0I98eSHvQT0dZR+***@public.gmane.org> wrote:
> >>
> >> I suppose the easiest way to add the information about the defining
> >> class is to have the class constructor add a function attribute
> >> to the (unbound) method functions. This should be the name of
> >> the defining class to avoid circular references.
> >>
> >> --
> >> Marc-Andre Lemburg
> >>
> >
> > As a workaround that'll work, yeah. But we were discussing whether we can
> > make a real solution. (Unless you were proposing that Python itself will
> do
> > this and not just my program; In that case I'm totally with you, but it
> > seems that the Python community isn't.)
>
> I was thinking of having this in Python itself. The compiler has
> all the information available, so why not add some of this useful
> meta-information to the created objects ?!
>
> This is not the same as creating a new bound method type - we'd
> only add some extra information for function objects that happen
> to be defined as methods.
>
> --
> Marc-Andre Lemburg
>

I totally agree with you. If you start talking on this in python-dev, `cc`
me.

Ram.

[Martin v. Löwis]

Received: from localhost (HELO mail.python.org) (127.0.0.1)
by albatross.python.org with SMTP; 31 May 2010 22:07:43 +0200
Received: from smtprelay04.ispgateway.de (smtprelay04.ispgateway.de
[80.67.31.27]) by mail.python.org (Postfix) with ESMTP
for <python-porting-+ZN9ApsXKcEdnm+***@public.gmane.org>; Mon, 31 May 2010 22:07:43 +0200 (CEST)
Received: from [92.195.162.11] (helo=[192.168.178.23])
by smtprelay04.ispgateway.de with esmtpsa (TLSv1:AES256-SHA:256)
(Exim 4.68) (envelope-from <***@v.loewis.de>)
id 1OJBGw-0003b6-Py; Mon, 31 May 2010 22:07:42 +0200
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
rv:1.9.1.9) Gecko/20100411 Icedove/3.0.4
In-Reply-To: <AANLkTimyTYmT1Foj0i8OSvCaG4br4usfGG8WseQUW4Mr-JsoAwUIsXosN+***@public.gmane.org>
X-Df-Sender: ***@v.loewis.de
X-BeenThere: python-porting-+ZN9ApsXKcEdnm+***@public.gmane.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion about porting Python code \(mainly 2.x -> 3.x\)"
<python-porting.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-porting>,
<mailto:python-porting-request-+ZN9ApsXKcEdnm+***@public.gmane.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-porting>
List-Post: <mailto:python-porting-+ZN9ApsXKcEdnm+***@public.gmane.org>
List-Help: <mailto:python-porting-request-+ZN9ApsXKcEdnm+***@public.gmane.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-porting>,
<mailto:python-porting-request-+ZN9ApsXKcEdnm+***@public.gmane.org?subject=subscribe>
Sender: python-porting-bounces+gcpp-python-porting=m.gmane.org-+ZN9ApsXKcEdnm+***@public.gmane.org
Errors-To: python-porting-bounces+gcpp-python-porting=m.gmane.org-+ZN9ApsXKcEdnm+***@public.gmane.org
Archived-At: <http://permalink.gmane.org/gmane.comp.python.porting/79>

> I suppose the easiest way to add the information about the defining
> class is to have the class constructor add a function attribute
> to the (unbound) method functions. This should be the name of
> the defining class to avoid circular references.
>
> As a workaround that'll work, yeah. But we were discussing whether we
> can make a real solution. (Unless you were proposing that Python itself
> will do this and not just my program; In that case I'm totally with you,
> but it seems that the Python community isn't.)

What makes you think so? I (personally) have objected to your hackish
way of scanning the all classes in the module. Adding a function
attribute is fine with me.

I only wonder if it would be useful to generalize this beyond functions.

Regards,
Martin

[cool-RR]

On Mon, May 31, 2010 at 10:07 PM, "Martin v. Löwis" <***@v.loewis.de>wrote:

> I suppose the easiest way to add the information about the defining
>> class is to have the class constructor add a function attribute
>> to the (unbound) method functions. This should be the name of
>> the defining class to avoid circular references.
>>
>> As a workaround that'll work, yeah. But we were discussing whether we
>> can make a real solution. (Unless you were proposing that Python itself
>> will do this and not just my program; In that case I'm totally with you,
>> but it seems that the Python community isn't.)
>>
>
> What makes you think so? I (personally) have objected to your hackish way
> of scanning the all classes in the module. Adding a function attribute is
> fine with me.
>

Great! I prefer adding an attribute to the unbound methods as well. Would
you like to raise this on python-dev or would you like me to? (I haven't
been active there and I don't know which issues are in consensus and which
aren't.) In any case `cc` me if you talk about it.


> I only wonder if it would be useful to generalize this beyond functions.
>

Sounds good.


Ram.

[Martin v. Löwis]

> Great! I prefer adding an attribute to the unbound methods as well.
> Would you like to raise this on python-dev or would you like me to?

I'm very skeptical that "raising" it will have any effect. Submit a
patch instead.

> (I haven't been active there and I don't know which issues are
> in consensus and which aren't.)

Consensus is irrelevant in absence of a patch.

Regards,
Martin

[cool-RR]

On Mon, May 31, 2010 at 10:19 PM, "Martin v. Löwis" <***@v.loewis.de>wrote:

> Great! I prefer adding an attribute to the unbound methods as well.
>> Would you like to raise this on python-dev or would you like me to?
>>
>
> I'm very skeptical that "raising" it will have any effect. Submit a patch
> instead.


I don't understand Martin. Unbound methods had an `.im_class` attribute in
Python 2.x, which is what we're talking about, and for Python 3.x that
attribute was purposefully removed. I am assuming it was a deliberate
decision by the python-dev community and that they had good reasons for it.
Am I not supposed to ask them about this before I put it back?


Ram.

[Brett Cannon]

On Mon, May 31, 2010 at 13:25, cool-RR <cool-***@cool-rr.com> wrote:
> On Mon, May 31, 2010 at 10:19 PM, "Martin v. Löwis" <***@v.loewis.de>
> wrote:
>>>
>>> Great! I prefer adding an attribute to the unbound methods as well.
>>> Would you like to raise this on python-dev or would you like me to?
>>
>> I'm very skeptical that "raising" it will have any effect. Submit a patch
>> instead.
>
> I don't understand Martin. Unbound methods had an `.im_class` attribute in
> Python 2.x, which is what we're talking about, and for Python 3.x that
> attribute was purposefully removed. I am assuming it was a deliberate
> decision by the python-dev community and that they had good reasons for it.
> Am I not supposed to ask them about this before I put it back?

What Martin is saying is that you will need a solid proposal that
includes a patch in order to convince python-dev to change something.
Making changes to the function type are not taken lightly and a clear
definition of what you want to happen needs to exist, and code is the
clearest definition you can have. Then it can be discussed on
python-dev to try to convince us to accept the patch.

[Martin v. Löwis]

Am 31.05.2010 22:25, schrieb cool-RR:
> On Mon, May 31, 2010 at 10:19 PM, "Martin v. Löwis" <***@v.loewis.de
> <mailto:***@v.loewis.de>> wrote:
>
> Great! I prefer adding an attribute to the unbound methods as well.
> Would you like to raise this on python-dev or would you like me to?
>
>
> I'm very skeptical that "raising" it will have any effect. Submit a
> patch instead.
>
>
> I don't understand Martin. Unbound methods had an `.im_class` attribute
> in Python 2.x, which is what we're talking about, and for Python 3.x
> that attribute was purposefully removed. I am assuming it was a
> deliberate decision by the python-dev community and that they had good
> reasons for it. Am I not supposed to ask them about this before I put it
> back?

Sure, you can ask. However, instead of asking, please study the code:
the subversion log is available to the general public. Find out (for
yourself) what specific revision made the change, and you can save other
contributors the time of doing the research for you.

You are mistaken that the the attribute was removed. It was not removed.
Instead, the object returned from the attribute lookup was changed, and
the "new" type just happened to have no im_class attribute.
It didn't have that attribute in 2.x, either.

Regards,
Martin

[cool-RR]

On Mon, May 31, 2010 at 11:26 PM, "Martin v. Löwis" <***@v.loewis.de>wrote:

> Am 31.05.2010 22:25, schrieb cool-RR:
>
>> On Mon, May 31, 2010 at 10:19 PM, "Martin v. Löwis" <***@v.loewis.de
>> <mailto:***@v.loewis.de>> wrote:
>>
>> Great! I prefer adding an attribute to the unbound methods as well.
>> Would you like to raise this on python-dev or would you like me to?
>>
>>
>> I'm very skeptical that "raising" it will have any effect. Submit a
>> patch instead.
>>
>>
>> I don't understand Martin. Unbound methods had an `.im_class` attribute
>> in Python 2.x, which is what we're talking about, and for Python 3.x
>> that attribute was purposefully removed. I am assuming it was a
>> deliberate decision by the python-dev community and that they had good
>> reasons for it. Am I not supposed to ask them about this before I put it
>> back?
>>
>
> Sure, you can ask. However, instead of asking, please study the code: the
> subversion log is available to the general public. Find out (for yourself)
> what specific revision made the change, and you can save other contributors
> the time of doing the research for you.
>
> You are mistaken that the the attribute was removed. It was not removed.
> Instead, the object returned from the attribute lookup was changed, and the
> "new" type just happened to have no im_class attribute.
> It didn't have that attribute in 2.x, either.
>
> Regards,
> Martin
>

Thanks for the clarifications, Martin and Brett.

At this time I'm both under-qualified and don't have time to work on a
patch. (I'm under-qualified because I don't code C and I'm not familiar with
Python's innards.)

Thanks for your help.


Ram.

[Jesus Cea]

On 01/06/10 13:13, cool-RR wrote:
> Thanks for the clarifications, Martin and Brett.
>
> At this time I'm both under-qualified and don't have time to work on a
> patch. (I'm under-qualified because I don't code C and I'm not familiar
> with Python's innards.)

Although having a patch is important, I think that first we need a
discussion about this subject in python-dev. Then opening an issue in
the bugtracker. The patch doesn't need to be written by you, and
python-dev must decide first it is a good idea (for somebody other to
develop! :) to avoid wasting the developer time.

- --
Jesus Cea Avion _/_/ _/_/_/ _/_/_/
jcea-***@public.gmane.org - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/
jabber / xmpp:jcea-/eSpBmjxGS4dnm+***@public.gmane.org _/_/ _/_/ _/_/_/_/_/
. _/_/ _/_/ _/_/ _/_/ _/_/
"Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/
"My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz

[Jesus Cea]

On 31/05/10 22:07, "Martin v. Löwis" wrote:
> What makes you think so? I (personally) have objected to your
> hackish way of scanning the all classes in the module. Adding a
> function attribute is fine with me.
>
> I only wonder if it would be useful to generalize this beyond
> functions.

Could be done thru a metaclass, trivially.

Adding an attribute automatically in python 3.2/3.3 would be some I
would support. It really needs a brainstorm in python-dev, though.

- --
Jesus Cea Avion _/_/ _/_/_/ _/_/_/
jcea-***@public.gmane.org - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/
jabber / xmpp:jcea-/eSpBmjxGS4dnm+***@public.gmane.org _/_/ _/_/ _/_/_/_/_/
. _/_/ _/_/ _/_/ _/_/ _/_/
"Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/
"My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz

[Lennart Regebro]

On Sat, May 29, 2010 at 20:43, "Martin v. Löwis" <***@v.loewis.de> wrote:
> bar = Foo.bar
>
> Now you can pickle Foo.bar.

Ah yes, of course. So simple and neat.

--
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
[phone]

[Lennart Regebro]

On Sat, May 29, 2010 at 18:40, cool-RR <cool-rr-***@public.gmane.org> wrote:
> I see. Where would be a good place to discuss this decision? I would want
> 3.2 to allow pickling of unbound methods.

That would be python-dev-+ZN9ApsXKcEdnm+***@public.gmane.org

However, it has already been discussed:

http://mail.python.org/pipermail/python-dev/2005-January/050625.html
http://mail.python.org/pipermail/python-dev/2007-November/075279.html

It turns out, even the pickling argument was discussed.

http://mail.python.org/pipermail/python-dev/2005-January/051143.html

Essentially, the argument is that it's easier to use a function
instead of writing a pickler for a method in the first case, so
support for pickling unbound methods isn't a high priority as there
are easier and less magic ways to solve the use case.


I know that's not what you want to hear, but that's likely to be the
answer you'll get.

--
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
[phone]

[cool-RR]

On Sat, May 29, 2010 at 7:41 PM, Lennart Regebro <regebro-***@public.gmane.org> wrote:

> On Sat, May 29, 2010 at 18:40, cool-RR <cool-rr-***@public.gmane.org> wrote:
> > I see. Where would be a good place to discuss this decision? I would want
> > 3.2 to allow pickling of unbound methods.
>
> That would be python-dev-+ZN9ApsXKcEdnm+***@public.gmane.org
>
> However, it has already been discussed:
>
> http://mail.python.org/pipermail/python-dev/2005-January/050625.html
> http://mail.python.org/pipermail/python-dev/2007-November/075279.html
>
> It turns out, even the pickling argument was discussed.
>
> http://mail.python.org/pipermail/python-dev/2005-January/051143.html
>
> Essentially, the argument is that it's easier to use a function
> instead of writing a pickler for a method in the first case, so
> support for pickling unbound methods isn't a high priority as there
> are easier and less magic ways to solve the use case.
>
>
> I know that's not what you want to hear, but that's likely to be the
> answer you'll get.
>
> --
> Lennart Regebro
>

Thanks for the info Lennart.

Ram.