Comparison of layered structured arrays

Question

Quick problem

I would like to be able to compare specific fields of type dtype from two massive numpy arrays, which are guaranteed to have the same type of dtype. I would like to do this in such a way that the fields that we compare with each other, every time the function is called based on the given inputs (i.e. I cannot easily copy comparisons for each individual field)

Long problem with examples

I am trying to compare specific fields from two massive numpy arrays with the same type. for example, say that

import numpy as np
from io import BytesIO

a = np.genfromtxt(BytesIO('12 23 0|23.2|17.9|0\n12 23 1|13.4|16.9|0'.encode()),dtype=[('id','U7'),('pos',[('x',float),('y',float)]),('flag','U1')],delimiter='|')

b = np.genfromtxt(BytesIO(' |23.0|17.91|0'.encode()),dtype=[('id','U7'),('pos',[('x',float),('y',float)]),('flag','U1')],delimiter='|')

which gives

In[156]: a
Out[154]: 
array([('12 23 0', (23.2, 17.9), '0'), ('12 23 1', (13.4, 16.9), '0')], 
      dtype=[('id', '<U7'), ('pos', [('x', '<f8'), ('y', '<f8')]), ('flag', '<U1')])

and

In[153]: b
Out[151]: 
array([('', (23.0, 17.91), '0')], 
      dtype=[('id', '<U7'), ('pos', [('x', '<f8'), ('y', '<f8')]), ('flag', '<U1')])

Now let's say that I want to check and find any entries in awhose field is a['pos']['x']larger than the field b['pos']['x']and return these entries to a new numpy array, something like this will work

newArr = a[a["pos"]["x"]>b["pos"]["x"]]

, a, x y , b. ,

newArr = a[np.array([np.array([a['pos']['x']>b['pos']['x']),a['pos']['y']>b['pos']['y'])).all(axis=0)]

, .

, , , dtype (, 34 - . dtype, ), , , , ( , dtype , ). , , , run to run ( , ). .

()

, , . - (, ):

mask = np.ones(z.shape,dtype=[('id',bool),('pos',[('x',bool),('y',bool)]),('flag',bool)])
# unmask the x and y fields so we can compare them 
mask['pos']['x']=0
mask['pos']['y']=0

maskedA = np.ma.masked_array(a, mask=mask)
# We need to do this or the masked array gets angry (at least in python 3)
b.shape = (1,)

maskedB = np.ma.masked_array(b, mask=mask)

-

test = (maskedA>maskedB).any(axis=1)

, , -

TypeError: unorderable types: MaskedArray() > MaskedArray()

test = (maskedA.compressed()>maskedB.compressed()).any(axis=1)

TypeError: ufunc 'logical_not' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

, , , , , , . - ?

, , , , , ...

, , , , , b. numpy , a. , , , .

.

, , , , , , , - , / numpy, , () , - , , . , MWE, " ", , .

Answer 1

, . numpy . . , . /usr/lib/python3/dist-packages/numpy/ma/core.py.

, .

"" :

In [116]: a['pos'][['y','x']]
Out[116]: 
array([(17.9, 23.2), (16.9, 13.4)], 
      dtype=[('y', '<f8'), ('x', '<f8')])

:

In [117]: a['pos'][['y','x']]=0
...
IndexError: unsupported iterator index

(, , ) .

In [123]: a['pos'][['y','x']]>b['pos'][['y','x']]
...
TypeError: unorderable types: numpy.ndarray() > numpy.ndarray()

unutbu :

In [127]: [a['pos'][name]>b['pos'][name] for name in ['x','y']]
Out[127]: [array([ True, False], dtype=bool), array([False, False], dtype=bool)]

dtype . recarray , , ( ). genfromtxt, , - , , dtype.

. , ('x','y') (2,):

In [141]: a1=np.array([('12 23 0', (23.2, 17.9), '0'), ('12 23 1', (13.4, 16.9), '0')], 
      dtype=[('id', '<U7'), ('pos', '<f8',(2,)), ('flag', '<U1')])
In [142]: b1=np.array([('', (23.0, 17.91), '0')], dtype=a1.dtype)
In [143]: a1['pos']>b1['pos']
Out[143]: 
array([[ True, False],
       [False, False]], dtype=bool)
In [145]: a1['pos']
Out[145]: 
array([[ 23.2,  17.9],
       [ 13.4,  16.9]])

a - copy, view reshape. copy , view dtype ( ).

In [150]: a['pos'].copy().view(float)
Out[150]: array([ 23.2,  17.9,  13.4,  16.9])

In [153]: a['pos'].copy().view(float).reshape(-1,2)>b['pos'].copy().view(float)
Out[153]: 
array([[ True, False],
       [False, False]], dtype=bool)

Answer 2

, Python. , :

In [87]: np.all([a['pos'][key] > b['pos'][key] for key in a['pos'].dtype.names], axis=0)
Out[87]: array([False, False], dtype=bool)

a['pos'][key] > b['pos'][key] a['pos'], np.all 0.

, , , a['pos'].dtype.names .