NumPy Array Copy-On-Write

Question

I have a class that returns large NumPy arrays. These arrays are cached inside the class. I would like the returned arrays to be copy to write arrays. If the caller finishes reading only from the array, no copy is made. This will mean that additional memory will not be used. However, the array is “modified”, but does not change the internal cached arrays.

My solution at the moment is to make any readonly cached arrays (a.flags.writeable = False). This means that if the caller may need to create their own copy of the array if they want to change it. Of course, if the source was not from the cache, and the array was already writable, then they would duplicate the data unnecessarily.

So, optimally I would like something like that a.view(flag=copy_on_write). There seems to be a flag for the opposite for this UPDATEIFCOPY, which forces the copy to update the original after being released.

Thanks!

Answer 1

Copy-on-write - , , -, " NumPy". " ", .

, copy-on-write. . "get_array (readonly)", ( ) numpy. , "readonly = False", . , .

numpy numpy, (, 2 ) .

Answer 2

, base, data, strides ndarray. , Python. Cython .

IPython:

%load_ext cythonmagic

Cython define copy_view():

%%cython
cimport numpy as np

np.import_array()
np.import_ufunc()

def copy_view(np.ndarray a):
    cdef np.ndarray b
    cdef object base
    cdef int i
    base = np.get_array_base(a)
    if base is None or isinstance(base, a.__class__):
        return a
    else:
        print "copy"
        b = a.copy()
        np.set_array_base(a, b)
        a.data = b.data
        for i in range(b.ndim):
            a.strides[i] = b.strides[i]

ndarray:

class cowarray(np.ndarray):
    def __setitem__(self, key, value):
        copy_view(self)
        np.ndarray.__setitem__(self, key, value)

    def __array_prepare__(self, array, context=None):
        if self is array:
            copy_view(self)
        return array

    def __array__(self):
        copy_view(self)
        return self

:

a = np.array([1.0, 2, 3, 4])
b = a.view(cowarray)
b[1] = 100 #copy 
print a, b
b[2] = 200 #no copy
print a, b

c = a[::2].view(cowarray)
c[0] = 1000 #copy
print a, c

d = a.view(cowarray)
np.sin(d, d) #copy
print a, d           

:

copy
[ 1.  2.  3.  4.] [   1.  100.    3.    4.]
[ 1.  2.  3.  4.] [   1.  100.  200.    4.]
copy
[ 1.  2.  3.  4.] [ 1000.     3.]
copy
[ 1.  2.  3.  4.] [ 0.84147098  0.90929743  0.14112001 -0.7568025 ]