Lesson on magic method access of Python new-style classes (from my failed Python3 port of Tomorrow)

I know the title is formidably long, but I can't find something more accurate (and my homegrown mini CMS doesn't support subtitle), so please bear with me.

So, I have madisonmay/Tomorrow — "magic decorator syntax for asynchronous code in Python 2.7" — bookmarked for a long time¹ without ever trying it, because I simply don't write Python 2 code any more (except when I try to maintain compatibililty). I felt kind of strange that a ~50-line project with ~1000 stars on GitHub hasn't been ported to Python 3 already, so I gave it a shot just now.

I thought it would be easy:

1. Modernize the old-style class Tomorrow;
2. Replace __getattr__ with __getattribute__ for unconditional attribute routing, then make a few exceptions to prevent infinite recursion;
3. 2to3 test cases;
4. Make meta changes, like removing the futures dependency.

However, after doing 1–3, I ran the tests, and out of the five test cases, three failed and one errored. I tried to isolate the problem, and ended up with the following piece of proof-of-concept:

class PassThrough(object):

    def __init__(self, obj):
        self._obj = obj

    def __getattribute__(self, name):
        if name == "_obj":
            return object.__getattribute__(self, name)
        print("Accessing '%s'" % name)
        return self._obj.__getattribute__(name)

This snippet is valid in both Python 2.7 and Python 3, but here's the surprise:

>>> g = PassThrough(0)
>>> print(g)
<__main__.PassThrough object at 0x10c662e48>
>>> str(g)
'<__main__.PassThrough object at 0x10c662e48>'
>>> hasattr(g, '__str__')
Accessing '__str__'
True
>>> g.__str__()
Accessing '__str__'
'0'

In addition, here's what happens if you try to "pass through" a function:

>>> def f(): return True
>>> g = PassThrough(f)
>>> g()
Accessing '__class__'
Accessing '__class__'
Traceback (most recent call last):
  File "<ipython-input-6-d65ffd94a45c>", line 1, in <module>
    g()
TypeError: 'PassThrough' object is not callable

>>> callable(g)
False
>>> hasattr(g, '__call__')
Accessing '__call__'
True
>>> g.__call__()
Accessing '__call__'
True

As you can tell, although __str__ or __call__ may have been implemented through __getattribute__, and hasattr (which in turn depends on getattr) has no trouble finding them, they are not picked up by str or function call (...). At this point, one would suspect that this is due to str or function call only looking at the class instance's __dict__. Compare this to the behavior of an old-style class:

class PassThrough():

    def __init__(self, obj):
        self._obj = obj

    def __getattr__(self, name):
        print("Acessing '%s'" % name)
        return self._obj.__getattribute__(name)

Now:

>>> g = PassThrough(0)
>>> print(g)
Acessing '__str__'
0
>>> def f(): return True
>>> g = PassThrough(f)
>>> g()
Acessing '__call__'
True

Note that magic method access is always routed through __getattr__.

After some digging, my suspicion was confirmed: indeed, for new-style classes, rather than invoking __getattribute__, the Python interpreter only looks for magic methods in __dict__. But is there a workaround for implementing something like the PassThrough class above? There's a nice answer on StackOverflow that uses a metaclass to "automatically add proxies for magic methods at the time of class creation", to quote the author. However, the thing about Tomorrow is that we don't have the result and don't know whatever magic methods it might have at class creation — after all, Python isn't a statically typed language. It is possible for programmers to offer hints, but then Tomorrow won't be as elegant and magical anymore. Therefore, unfortunately enough, Tomorrow isn't portable to Python 3 — at least not without a substantial hack that's beyond my knowledge, or a complete overhaul of its logic (haven't thought about that).