Create an object that cannot be pickled

Question

How can I easily create an object that cannot be etched to check for edge cases in my rpc code?

It should be:

• Plain
• Reliable (not expected to be torn in future versions of python or pickle)
• Cross platform

Edit: The intended use looks something like this:

class TestRPCServer: def foo(self): return MagicalUnpicklableObject() def test(): with run_rpc_server_and_connect_to_it() as proxy: with nose.assert_raises(pickle.PickleError): proxy.foo() 

Answer 1

If all you need is an object that will throw an exception when you catch it, to navel the test, you can detonate the __getstate__ method.

 >>> class C: ... def __getstate__(self): ... raise Exception ... >>> pickle.dumps(C()) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1374, in dumps Pickler(file, protocol).dump(obj) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 224, in dump self.save(obj) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 723, in save_inst stuff = getstate() File "<stdin>", line 3, in __getstate__ Exception 

This is hardly easier!

---

If you want to use a less artificial scenario, think of objects that use OS resources, such as file descriptors, or sockets, or streams, etc.

 >>> with open('spam.txt', 'w') as f: ... pickle.dumps(f) ... Traceback (most recent call last): File "<stdin>", line 2, in <module> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1374, in dumps Pickler(file, protocol).dump(obj) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 224, in dump self.save(obj) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 306, in save rv = reduce(self.proto) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/copy_reg.py", line 70, in _reduce_ex raise TypeError, "can't pickle %s objects" % base.__name__ TypeError: can't pickle file objects 

Answer 2

If you need an explicit list of objects that can be pickled compared to objects that cannot be pickled using pickle and more advanced serializers, such as dill , this file contains a somewhat exhaustive list for standard library objects, It provides an easy way to build each object (as usually single-line) and shows options for different versions of python, if applicable.

https://github.com/uqfoundation/dill/blob/cccbea9b715e16b742288e1e5a21a687a4d4081b/dill/_objects.py#L255

For example, pickle will fail on the following object, while extended serializers, such as dill , will not:

 >>> import dill >>> dill.dumps(Ellipsis) b'\x80\x03cdill.dill\n_eval_repr\nq\x00X\x08\x00\x00\x00Ellipsisq\x01\x85q\x02Rq\x03.' 

Extended serializers can also work with file objects, etc., btw:

 >>> dill.dumps(open('foo.pkl', 'w')) b'\x80\x03cdill.dill\n_create_filehandle\nq\x00(X\x07\x00\x00\x00foo.pklq\x01X\x01\x00\x00\x00wq\x02K\x00\x89cdill.dill\n_get_attr\nq\x03cdill.dill\n_import_module\nq\x04X\x02\x00\x00\x00ioq\x05\x85q\x06Rq\x07X\x04\x00\x00\x00openq\x08\x86q\tRq\n\x89K\x00X\x00\x00\x00\x00q\x0btq\x0cRq\r.' 

However, pickle and dill (and other advanced serializers) will fail on any type that is directly connected to the FrameType python, like a generator:

 >>> dill.dumps((i for i in [])) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/mmckerns/lib/python3.4/site-packages/dill-0.2.6.dev0-py3.4.egg/dill/dill.py", line 243, in dumps dump(obj, file, protocol, byref, fmode, recurse)#, strictio) File "/Users/mmckerns/lib/python3.4/site-packages/dill-0.2.6.dev0-py3.4.egg/dill/dill.py", line 236, in dump pik.dump(obj) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/pickle.py", line 412, in dump self.save(obj) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/pickle.py", line 499, in save rv = reduce(self.proto) TypeError: can't pickle generator objects