Unable to override __init__ class from Cython extension

Question

I am trying to subclass pysam Tabixfile and add additional attributes when instantiating.

 class MyTabixfile(pysam.Tabixfile): def __init__(self, filename, mode='r', *args, **kwargs): super().__init__(filename, mode=mode, *args, **kwargs) self.x = 'foo' 

When I try to instantiate my subclass MyTabixfile , I get TypeError: object.__init__() takes no parameters :

 >>> mt = MyTabixfile('actn2-oligos-forward.tsv.gz') Traceback (most recent call last): File "<ipython-input-11-553015ac7d43>", line 1, in <module> mt = MyTabixfile('actn2-oligos-forward.tsv.gz') File "mytabix.py", line 4, in __init__ super().__init__(filename, mode=mode, *args, **kwargs) TypeError: object.__init__() takes no parameters 

I also tried to explicitly call the Tabixfile constructor:

 class MyTabixfile(pysam.Tabixfile): def __init__(self, filename, mode='r', *args, **kwargs): pysam.Tabixfile.__init__(self, filename, mode=mode, *args, **kwargs) self.x = 'foo' 

but it still raises TypeError: object.__init__() takes no parameters .

This class is actually implemented in Cython; constructor code below:

 cdef class Tabixfile: '''*(filename, mode='r')* opens a :term:`tabix file` for reading. A missing index (*filename* + ".tbi") will raise an exception. ''' def __cinit__(self, filename, mode = 'r', *args, **kwargs ): self.tabixfile = NULL self._open( filename, mode, *args, **kwargs ) 

I read the Cython documentation on __cinit__ and __init__ which says

Any arguments passed to the constructor will be passed for both __cinit__() and the __init__() method. If you expect to subclass your extension type in Python, you might find it useful to enter the __cinit__() method * and ** arguments so that it can accept and ignore additional arguments. Otherwise, any Python subclass that has __init__() with a different signature will have to override __new__() 1 , as well as __init__() , which the author of the Python class would not expect to do.

The pysam developers took care to add *args and **kwargs to the Tabixfile.__cinit__ , and my subclass __init__ corresponds to the __cinit__ signature, so I don’t understand why I cannot redefine the Tabixfile initialization.

I am developing Python 3.3.1, Cython v.0.19.1 and pysam v.0.7.5.

Answer 1

The documentation here is a bit confusing as it assumes you are familiar with using __new__ and __init__ .

The __cinit__ method __cinit__ roughly equivalent to the __new__ method in Python. *

Like __new__ , __cinit__ not called by your super().__init__ ; it is called before Python even gets into your subclass __init__ . The reason __cinit__ must be the signature processing of your subclass __init__ . Exactly the same reason __new__ does.

If your subclass explicitly calls super().__init__ , which again looks for the __init__ method in the superclass, for example __new__ , and __cinit__ not __init__ . So, if you have not defined __init__ yet, it will go to object .

---

You can see the sequence with the following code.

cinit.pyx:

 cdef class Foo: def __cinit__(self, a, b, *args, **kw): print('Foo.cinit', a, b, args, kw) def __init__(self, *args, **kw): print('Foo.init', args, kw) 

init.py:

 import pyximport; pyximport.install() import cinit class Bar(cinit.Foo): def __new__(cls, *args, **kw): print('Bar.new', args, kw) return super().__new__(cls, *args, **kw) def __init__(self, a, b, c, d): print('Bar.init', a, b, c, d) super().__init__(a, b, c, d) b = Bar(1, 2, 3, 4) 

When you start, you will see something like:

 Bar.new (1, 2, 3, 4) {} Foo.cinit 1 2 (3, 4) {} Bar.init 1 2 3 4 Foo.init (1, 2, 3, 4) {} 

---

So, the correct fix here depends on what you are trying to do, but this is one of the following:

• Add the __init__ method to the Cython base class.
• Delete the entire super().__init__ .
• Modify super().__init__ to not pass any parameters.
• Add the appropriate __new__ method to the Python subclass.

I suspect that in this case you need 2.

---

* It is worth noting that __cinit__ definitely not identical to __new__ . Instead of getting the cls parameter, you get a partially constructed self object (where you can trust __class__ and C attributes, but not Python attributes or methods), the __new__ methods of all classes in the MRO have already been called before any __cinit__ ; __cinit__ your databases is automatically called automatically, and not manually; You cannot return another object than the one that was requested; etc. It is just that he called before __init__ and expected passing through parameters, just like __new__ .

Answer 2

I would comment, not send a response, but StackOverflow foo is still not enough.

The @abarnert post is excellent and very helpful. I would just add a few pysam features here, as I just subclassed pysam.AlignmentFile in a very similar way.

Option # 4 was the cleanest / easiest choice, which meant only changes in my own __new__ subclass to filter unknown parameters:

 def __new__(cls, file_path, mode, label=None, identifier=None, *args, **kwargs): # Suck up label and identifier unknown to pysam.AlignmentFile.__cinit__ return super().__new__(cls, file_path, mode, *args, **kwargs) 

It should also be noted that the pysam file classes do not seem to have an explicit __init__ method, so you also need to skip the param pass as this is directly related to the .__ init__ object, which does not accept parameters:

 def __init__(self, label=None, identifier=None, *args, **kwargs): # Handle subclass params/attrs here # pysam.AlignmentFile doesn't have an __init__ so passes straight through to # object which doesn't take params. __cinit__ via new takes care of params super(pysam.AlignmentFile, self).__init__()