The right way to create user extensible features

Question

What is the correct way in Python to allow a user to extend the types that a function can run on without changing the source code of the function?

Suppose I have a module with the function my_module.foo() , which was originally written to work with float types. Now I would like the same function to work with valid exact mpmath floats, but without changing the code in the source module.

In C ++, I would add extra overload (or rather, some trick of specializing a template with an auxiliary structure). How should I structure the source code of my_module.foo() so that the user can add their own custom interceptors inside?

I can come up with several ways to achieve this, but as a beginner Python programmer, I'm sure most of them will be terrible :)

EDIT : Thanks for all the answers so far, much appreciated.

I should probably clarify that one key requirement is the ability to handle types that I myself have not defined. For example, if I try to encode the generic cos function in my module, I want to call math.cos for built-in types, mpmath.cos for mpf types, sympy.cos for symbolic character types, etc. And of course, I would like the send logic not to be in my implementation of the cos module.

Answer 1

There are two ways to do this:

• Delegation Already in Python. Most pythons. Does not work for built-in types. Not quite what you are looking for.
• Separate shipping . Another PEP. Works for built-in types. Exactly what you are looking for.

Delegation

Usually you delegate responsibility for the object that you are affecting, and you do not implement the logic in your function.

Here is an example: len . The len implementation is very simple:

 def len(obj): return obj.__len__() 

Different types ( str , list , tuple ...) have different implementations, but they all work with the same function.

Now, if I want to define my own type that works with len , I can do:

 class MyTypeOfLength3(object): def __len__(self): return 3 o = MyTypeOfLength3() print len(o) # 3 

In your case, you will implement something similar to len .

(Note: this is not the actual code for len , but it is more or less equivalent.)

Single dispatch

Of course, in some cases this can be impractical. If this is your case, then the "Single Dispatch" PEP 443 is probably what you are looking for.

He offers a new decorator that will accomplish what you are looking for:

 >>> from functools import singledispatch >>> @singledispatch ... def fun(arg, verbose=False): ... if verbose: ... print("Let me just say,", end=" ") ... print(arg) ... >>> @fun.register(int) ... def _(arg, verbose=False): ... if verbose: ... print("Strength in numbers, eh?", end=" ") ... print(arg) ... >>> @fun.register(list) ... def _(arg, verbose=False): ... if verbose: ... print("Enumerate this:") ... for i, elem in enumerate(arg): ... print(i, elem) 

Once you have defined your function as such, you can call fun(something) and Python will find the correct implementation ( int or list here), disconnecting the default implementation def fun(...): ...

Therefore, you only need to decorate your original function, and you're done, your users can add their own types.

Note: as pointed out in the comments, singledispatch already implemented in Python, it is pkgutil.simplegeneric

Answer 2

It is possible to do what you want without waiting for PEP 443 - common functions to send one part , instead using abstract base classes that were added in Python 2.6. They allow you to create "virtual" metaclasses and add arbitrary subclasses to them without changing the existing code or rendering it harmless. Then your module can use the types registered in this metaclass to figure out what to do. You (or authors) of other types can register them as needed.

Here is a sample code illustrating the concept:

 import abc class Trigonometric(object): __metaclass__ = abc.ABCMeta _registry = {} @classmethod def register(cls, subclass, cos_func, sin_func): cls.__metaclass__.register(cls, subclass) if subclass not in cls._registry: # may or may not want this check... cls._registry[subclass] = {'cos': cos_func, 'sin': sin_func} @classmethod def call_func(cls, func_name, n): try: return cls._registry[n.__class__][func_name](n) except KeyError: raise RuntimeError( "Either type {} isn't registered or function {}() " "isn't known.".format(n.__class__.__name__, func_name)) # module-level functions def cos(n): return Trigonometric.call_func('cos', n) def sin(n): return Trigonometric.call_func('sin', n) if __name__ == '__main__': # avoid hardcoding this module filename into the source import sys my_module = sys.modules[__name__] # replaces import my_module # register the built-in float type import math print 'calling Trigonometric.register(float)' Trigonometric.register(float, math.cos, math.sin) # register mpmath arbitrary-precision mpf float type from mpmath import mp print 'calling Trigonometric.register(mp.mpf)' Trigonometric.register(mp.mpf, mp.cos, mp.sin) f = 1.0 print 'isinstance(f, Trigonometric):', isinstance(f, Trigonometric) print 'my_module.cos(f):', my_module.cos(f), my_module.sin(f) v = mp.mpf(1) print 'isinstance(v, Trigonometric):', isinstance(v, Trigonometric) print 'my_module.cos(v):', my_module.cos(v), my_module.sin(v) 

Answer 3

It depends on the code and the expected results. Normally you should not specify data types implicitly. Use Duck Typing .

But if a function expects only a float, you can wrap it. The simplest example:

 def bar(data): """Execute my_module.foo function with data converted to float""" return my_module.foo(float(data))