NumPy: consequences of using 'np.save ()' with 'allow_pickle = False'

Question

According to the NumPy documentation here , by default, the matrix is ​​saved with allow_pickle=True , and they also report what might be problematic with this default behavior:

allow_pickle: bool, optional
Allow saving arrays of objects using Python pickles. Reasons for not pickling include security (loading pickled data may execute arbitrary code) and portability (pickled objects may not load on different Python installations, for example, if stored objects require libraries that are not available, and not all pickled data is compatible between Python 2 and Python 3).
Default: True

After reading, I would prefer to use allow_pickle=False - but they do not say what makes it different when it is used that way. For some reason, they should use allow_pickel=True by default, despite their drawbacks.

Could you tell me if you allow_pickle=False and how it behaves differently?

Answer 1

An array of objects is just a regular numpy array, where dtype is an object ; this happens if the contents of the array are not ordinary numeric types (for example, int or float , etc.). We can try to save a numpy array with objects, just to test how it works. A simple view of an object would be a dict :

 >>> import numpy as np >>> a = np.array([{x: 1} for x in range(4)]) >>> a array([{0: 1}, {1: 1}, {2: 1}, {3: 1}], dtype=object) >>> np.save('test.pkl', a) 

Downloading this function works fine:

 >>> np.load('test.pkl.npy') array([{0: 1}, {1: 1}, {2: 1}, {3: 1}], dtype=object) 

An array cannot be stored without using a brine:

 >>> np.save('test.pkl', a, allow_pickle=False) ... ValueError: Object arrays cannot be saved when allow_pickle=False 

The rule of thumb for pickles is that you are safe if you load pickled cucumbers, but you must be careful when loading pickles that you got from another place. Firstly, if you do not have the same libraries (or library versions) that were used to prepare the brine, you may not be able to load the brine (this is what is meant with portability above). Security is another potential concern; you can read a little about how pickles can be abused in this article , for example.