The average value of the average density in a large array of masks

Question

What is a suitable way to get the average of a large array of masks? Usually I just called .mean(), but for a very large array this did not work for me.

Consider creating an array of a million elements, all with a value of 500, for example:

a = np.ones(1000000, dtype=np.int16) * 500

Then create a random mask and combine both in a new one masked array:

mask = np.random.randint(0, 2, a.size)
b = np.ma.masked_array(a, mask=mask)

An array binherits dtypeas well int16.

Obtaining the average value from bcan be carried out in different ways, but they all give the same result. But functions non-maignore the mask and should not be used.

print(np.average(b), np.mean(b), np.ma.average(b), np.ma.mean(b))
(500.0, 500.0, 500.0, 500.0)

But if I increase the size of the original array from a million to ten million, the result will be:

print(np.average(b), np.mean(b), np.ma.average(b), np.ma.mean(b))
(500.0, -359.19365132075774, -359.19365132075774, -359.19365132075774)

np.average , , , , b[b.mask] = 1000 . , np.mean .

b float32 :

b = b.astype(np.float32)
print(np.average(b), np.mean(b), np.ma.average(b), np.ma.mean(b))
(511.18945, 510.37895680000003, 510.37895680000003, 510.37895680000003)

b float64 ( ) :

b = b.astype(np.float64)
print(np.average(b), np.mean(b), np.ma.average(b), np.ma.mean(b))
(500.0, 500.0, 500.0, 500.0)

float64, , , , .

, np.ma.average (a) , , , np.ma.mean .

- dtype size ? , .

Numpy 1.8.1 64- Win 7. .

, , :

http://nbviewer.ipython.org/gist/RutgerK/69b60da73f464900310a

Answer 1

b[b.mask] = 1000, . , np.mean , .

, b.mask - True, . , , , , b[np.invert(b.mask)].

, :

import numpy as np

a = np.ones(10000000, dtype=np.int64) * 500

mask = np.random.randint(0, 2, a.size)
b = np.ma.masked_array(a, mask=mask)

b[np.invert(b.mask)] = 1000
print(np.average(b), np.mean(b), np.ma.average(b), np.ma.mean(b))

, np.average.

, / , . dtype=np.int64,

: - Python dtype=object , , np.average, .

2: , , np.mean(b, dtype=np.float64), np.mean , .