Why doesn't Python include the pickle loading function from the file name?

Question

I often include this or something close to it in Python scripts and IPython laptops.

import cPickle def unpickle(filename): with open(filename) as f: obj = cPickle.load(f) return obj 

This seems like a fairly common case where the standard library should provide a function that does the same. Is there such a function? If not, how?

Answer 1

Most serialization libraries in stdlib and PyPI have a similar API. I am sure it is marshal that install standard, ^* and pickle , json , PyYAML , etc. just followed in the footsteps.

So the question is, why was marshal designed this way?

Well, you obviously need loads / dumps ; you could not create them on top of the function based on the file name, and to create them on top of the function based on file-based objects, you need StringIO , which did not appear until the end.

You do not need to load / dump , because they can be built on top of loads / dumps , but this can have serious performance implications: you can save nothing on the file until you build it all in memory, and vice versa, which can be a problem for huge objects.

You definitely don't need the loadf / dumpf function based on file names, because they can be built trivially on top of load / dump , without any performance implications, and there are no complicated considerations that the user might be wrong.

On the one hand, it would be convenient to have them anyway - and there are some libraries, such as ElementTree , that have similar functions. This can save only a few seconds and a few lines per project, but multiply this by thousands of projects ...

On the other hand, this will make Python larger. There is not much extra 1K to download and install if you added these two functions to each module (although it meant a lot more in 1.x days than these days ...), but for more documentation, learn more, remember more . And, of course, more support code - every time you need to fix a bug in marshal.dumpf , you need to remember that you need to go through the pickle.dumpf and json.dumpf to make sure they don't need to be changed, and sometimes you won 'Don't remember.

Balancing these two considerations is indeed a call to judgment. One person did decades ago, and probably no one ever discussed with him. If you think that today there is a good option for changing it, you can always send a function request to track problems or start the flow on python-ideas .

---

_{* Not in the original version of the marshal.c version in the 1991 version; which had only load and dump . Guido added loads and dumps in 1993 as part of a change whose main description was “Add a standalone core Mac program: macmain.c”. Presumably because something inside the Python interpreter needed to be flushed and loaded into strings. ^**}

_{** marshal used as the basis for things like importing .pyc files. It also means (at least in CPython) that it is not only implemented in C, but statically built into the core of the interpreter itself. Although I think that in fact it could be turned into a regular module, since the changes are 3.4 import , but it definitely could not return in the early days. Therefore, it is an additional motivation to keep it small and simple.}