Python: can I safely unzip untrusted data?

Question

The brine documentation says right at the start:

Caution : The brine module is not designed to protect against erroneous or malicious data. Never print data received from an unreliable or unauthenticated source.

However, further limiting global variables seems to describe how to securely access data securely using a white list of allowed objects.

Does this mean that I can safely scatter untrusted data if I use RestrictedUnpickler , which allows only some "elementary" types, or are there additional security problems that are not addressed by this method? If there is, is there any other way to make the spill safe (obviously because you cannot spill every thread)?

With "elementary types" I mean exactly the following:

• bool
• str , bytes , bytearray
• int , float , complex
• tuple , list , dict , set and frozenset

Answer 1

I would say that a safe way to use a brine to handle unreliable data does not exist.

Even with limited global dynamics, the dynamic nature of Python is such that a certain hacker still has a chance to find a way to display __builtins__ and from there to Jewel Crown.

See the Ned Batchelder blog post for restrictions on eval() that apply equally to pickle .

Remember that pickle is still the language of the stack, and you cannot foresee all possible objects created by allowing arbitrary calls even to a limited set of global variables. The pickle documentation also does not mention EXT* opcodes, which allow you to call copyreg extensions; You will also have to consider everything that is installed in this registry. All that is required is a single vector, allowing you to turn the call of the object into the equivalent of getattr for your protection to crash.

At the very least, use a cryptographic signature for your data so you can verify integrity. You will limit the risks, but if an attacker manages to steal your subscription secrets (keys), then they can again dump you with hacked pickling.

Instead, I would use an existing harmless format, such as JSON, and add type annotations; for example, store data in dictionaries using a type key and convert it when loading data.

Answer 2

This idea was also discussed on the python-ideas mailing list when solving the problem of adding a safe pickle alternative to the standard library. For example here :

To make it safer, I would have a limited unickler by default (for upload / download) and force people to override it if they want to relax the restrictions. To be explicit, I would do the upload / download with only the built-in types.

And also here :

I always wanted a version of pickle.loads () that accepts a list of classes that are allowed to instantiate.

Is the following enough for you: http://docs.python.org/3.4/library/pickle.html#restricting-globals ?

In fact, it is. Thanks for pointing! I never passed a part of the module interface in documents. Perhaps a warning at the top of the page could also mean that there are ways to mitigate security issues and point out # global restrictions?

Yes, that would be a good idea :-)

Therefore, I don’t know why the documentation was not changed, but in my opinion, using RestrictedUnpickler to restrict the types that can be printed is a safe solution. Of course, errors may occur in the library in compromising the system, but there may be an error in OpenSSL that displays random memory data to anyone who asks.