Why do many Python built-in / standard libraries actually execute classes

Question

Many of Python's built-in "functions" are actually classes, although they also have a direct implementation of functions. Even very simple ones, like itertools.repeat . What is the motivation for this? It seems to me that this is too complicated.

Edit: I am not asking about the purpose of itertools.repeat or any other specific function. This was just an example of a very simple function with a very simple possible implication:

 def repeat(x): while True: yield x 

But itertools.repeat is not really a function, it is implemented as a class. My question is why? This seems like unnecessary overhead.

I also understand that classes are function calls and how you can emulate the behavior of a function using a class. But I do not understand why it is so widely used through the standard library.

Answer 1

Implementing as a class for itertools has some advantages that do not have generator functions. For instance:

• CPython implements these built-in modules at the C level, and at the C level, the generator function is best implemented as a class that implements __next__ , which stores state as instance attributes; The yield -based generators are Python level, and indeed, they are just an instance of the generator class (so they are actually still instances of the class, like everything else in Python).
• Generators are not legible or copied and do not have a “history” in order to force them to support either behavior (the internal state is too complex and opaque to generalize); the class can define __reduce__ / __copy__ / __deepcopy__ (and if it is a Python level class, it probably doesn’t even need to, it will work automatically) and make pickleable / copyable instances (so if you have already generated 5 elements from the range iterator, you can copy or expand / expand it and get an iterator at the same distance along the iteration)

For non-generating tools, the reasons are usually similar. Classes can be given state and customized behavior that the function cannot. They can be inherited from (if necessary, but C classes can prohibit subclassing if they are "logically" functions).

It is also useful for creating dynamic instances; if you have an instance of an unknown class, but a known prototype (say, sequence constructors that accept iterability, or chain or something else), and you want to convert some other type to this class, you can do type(unknown)(constructorarg) ; if it is a generator, type(unknown) useless, you cannot use it to do more, because you cannot understand where it came from (not in sensible ways).

And besides, even if you never use functions for programming logic, what would you rather see in the interactive interpreter or debug the fingerprint type(myiter) , <class 'generator'> , which does not give any hint of origin, or <class 'itertools.repeat'> , which tells you exactly what you have and where it came from?

Answer 2

Both functions and classes are callers, so they can be used interchangeably in higher functions, for example.

 $ python2 ... >>> map(dict, [["ab"], ["cd"], ["ef"]]) [{'a': 'b'}, {'c': 'd'}, {'e': 'f'}] >>> map(lambda x: dict(x), [["ab"], ["cd"], ["ef"]]) [{'a': 'b'}, {'c': 'd'}, {'e': 'f'}] 

However, classes can also define methods that you can later call on returned objects. For example, the dict class defines .get() for dictionaries, etc.

Answer 3

In the case of itertools.repeat (and most iterators), using a suitable class that implements the iterator protocol has several advantages from POV implementation / maintenance - for example, you can better control iteration, you can specialize the class, etc. I also suspect that there are some optimizations that can be done at C level for the right iterators that don't apply to generators.

Also remember that classes and functions are also objects - the def statement is basically syntactic sugar for creating an instance of function and fills it with compiled code, local namespace, cells, closure and whatnot (somehow the FWIW task involved, I just did it from because of curiosity, and it was a big PITA), and the class operator is also syntactic sugar for creating a new instance of type (doing it manually, in fact, is really trivial). From this, POV yield is a syntax-like sugar that turns your function into a returned factory instance of a typical generator builtin type - IOW, which forces your function to act as a class, without the hassle, but also without the fine controls and possible optimizations you can get. by writing a full-blown class.

In a more general left, sometimes when writing your “function” as a custom called type, instead, similar wins are offered - subtle control, possible optimizations, and sometimes just improved readability (think of two-story decorators, custom descriptors, etc.).

Finally, wrt / builtin types ( int , str , etc.) IIRC (please someone correct me if I'm wrong), they were originally functions that acted as factory functions (before the revolution of new-style classes when building types and custom types are different types of objects). Of course, it now makes sense to have them as simple classes, but they should have supported the all_lower naming scheme for compatibility.