How does automatic currying with self when assigning a method into a var work in Python 3?

Question

I am writing a context manager to wrap the builtins.print function. And this works fine. However I encountered a Python behaviour that I can't wrap my head around:

Whenever a classes' method is assigned into a variable for later calling, the first "self" argument seems to be automatically stored as well and used for all later calls.

Here's an example illustrating the point:

import functools

class Wrapper:
    def wrap(self):
        return self._wrapped   #functools.partial(self._wrapped, self)

    def _wrapped(self, *args, **kwargs):
        print('WRAPPED!', *args, **kwargs)
        print('..knows about self:', self)

wrapped = Wrapper().wrap()
wrapped('expect self here', 'but', 'get', 'all', 'output')

The output:

WRAPPED! expect self here but get all output
..knows about self: <__main__.Wrapper object at 0x2aaaab2d9f50>

Of course for normal functions (outside of classes) this magic does not happen. I can even assign that method in the example above directly without going through instantiation:

wrapped = Wrapper._wrapped
wrapped('expect self here', 'but', 'get', 'all', 'output')

And now I get what I first expected:

WRAPPED! but get all output
..knows about self: expect self here

In my original code, I used the functools.partial to curry-in the self, but then discovered that this is not even required.

I like the current behaviour, but I'm not yet understanding the reasoning with respect to consistency and "being obvious".

I'm working with Python 3.1.2 here.

Is this question with the answer to use types.MethodType related? Searching here and in the 'net largely results in basic info on currying/partial function calls and packing/unpacking of arg lists. Maybe I used inadequate search terms (e.g. "python currying methods".)

Can anyone shed some light into this behaviour?

Is this the same in Py2 and Py3?

After some more research I can see the relationship to bound versus unbound methods. See [this Q](http://stackoverflow.com/q/114214/923794). But I'm still wondering about the storage of one particular `self` to that method during the assignment. — cfi, Dec 01 '11 at 10:00
If you want to suppress this behaviour, you can use staticmethod: http://docs.python.org/library/functions.html#staticmethod — Thomas K, Dec 01 '11 at 13:25
Coming from Perl, I had this same question. (For a Perl module which does something close, see [curry](http://metacpan.org/module/curry)) — Joel Berger, Aug 19 '13 at 16:28

Ethan Furman · Accepted Answer · 2016-07-05T17:22:39.073

1

Whenever you take the method from an instance (as in return self._wrapped) then self will be remembered.

Whenever you take the method from a class (as in Wrapper._wrapped) then self is not (cannot be) remembered.

As an example, try this:

upper = 'hello'.upper
print(upper())

upper = str.upper
print(upper())

You'll see HELLO, followed by TypeError: descriptor 'upper' of 'str' object needs an argument

edited Jul 05 '16 at 17:22

answered Dec 01 '11 at 13:23

Ethan Furman

63,992
20
159
237

I believe that's a good and valid simplification of the example I provided in my question. Thanks for that. Can you also point out reasons why this behavior was introduced? I could argue both ways: That this is a nice feature (which I come to believe it is), or that this is introducing an inconsistency in behavior that is not obvious. – cfi Dec 01 '11 at 13:33
@cfi: This behavior was "introduced" at least ten years ago, of not twenty. :-) AFAIK it's always been like that. This behavior is consistent. – Lennart Regebro Dec 01 '11 at 13:39
@Lennart: Forgive my ignorance - I'm only a few months on Python now. So you asserted the consistency over time. What I meant is the (in)consistency between currying functions and methods. But don't get me wrong: I get your point, and - again - I see this as a feature (=positive surprise to me). If it's there for so long, is there any "official" (Guido, PEP, email thread?) reasoning why this has been introduced? Obviously it's useful. Can't remember seeing this in other languages (e.g. C++, Perl) – cfi Dec 01 '11 at 14:40
@cfi: To be honest I think it has always been like that, so there probably is no text on why it was implemented like that. And it really doesn't have anything to do with currying. It's just that when you call an instance method, self gets passed in automatically. The difference between C for example, is that when you get a reference to an instance method in Python, it stays an instance method, while in C++ it's just a pointer to a memory location, really. This is because C++ is not a high-level language, but an OO-wrapper around a semi-portable low-level language. – Lennart Regebro Dec 01 '11 at 21:16
@Lennart C++ is much more than an OO-wrapper. Maybe at it's conception it could be considered that, but it's evolved far beyond that. – MGZero Dec 05 '11 at 14:16
@MGZero: Yes, you are right, but I was being nice. :-) But I guess I could reformulate it like this, while still being nice to the language: C++ is a semi-portable object-oriented quasi-assembler. That's still nice, and more accurate. :-) – Lennart Regebro Dec 06 '11 at 07:45
Thanks for these explanations. Got here because I found this happy accident: given `mydict={'key1': 'value1', 'key2': 'value2'}` then `map('='.join, mydict.iteritems() )` gives `['key2=value2', 'key1=value1']` – Steve Carter Jul 05 '16 at 16:28

score 1 · Answer 2 · answered Dec 01 '11 at 13:36

When an instance method is called, that call will automatically pass in the instance as the first parameter. This is what happens here.

When you do

return self._wrapped

You will return an instance method. Calling it will pass in the instance as the first parameter, that is self. But in the second case you call the method on the class, and hence there exists no instance to get passed in, so no instance gets passed in.

The "storage" of this is simply that instance methods know which instance they belong to. If you don't want that behavior return the unbound class method instead.

class Wrapper:
    def wrap(self):
        return Wrapper._wrapped

    def _wrapped(self, *args, **kwargs):
        print('WRAPPED!', *args, **kwargs)
        print('..knows about self:', self)

thanks(!) for your valuable comments and this answer. I would have split the "answer" checkmark between Ethan's and your answer if that was possible. — cfi, Dec 02 '11 at 08:50

How does automatic currying with self when assigning a method into a var work in Python 3?

2 Answers2