Numpy.concatenate on record arrays fails when the array has different string lengths

Question

When trying to combine arrays of records that have a dtype string field but have different lengths, concatenation is not performed.

As you can see in the following example, concatenate works if "f1" is the same length, but fails if not.

In [1]: import numpy as np

In [2]: a = np.core.records.fromarrays( ([1,2], ["one","two"]) )

In [3]: b = np.core.records.fromarrays( ([3,4,5], ["three","four","three"]) )

In [4]: c = np.core.records.fromarrays( ([6], ["six"]) )

In [5]: np.concatenate( (a,c) )
Out[5]: 
array([(1, 'one'), (2, 'two'), (6, 'six')], 
      dtype=[('f0', '<i8'), ('f1', '|S3')])

In [6]: np.concatenate( (a,b) )
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)

/u/jegannas/<ipython console> in <module>()

TypeError: expected a readable buffer object

But then again, if we just concatenate arrays (rather than records), this succeeds, although the strings are of different sizes.

In [8]: np.concatenate( (a['f1'], b['f1']) )
Out[8]: 
array(['one', 'two', 'three', 'four', 'three'], 
      dtype='|S5')

Is this a mistake in concatenation when concatenating records or is this the expected behavior. I decided only the following way to overcome this.

In [10]: np.concatenate( (a.astype(b.dtype), b) )
Out[10]: 
array([(1, 'one'), (2, 'two'), (3, 'three'), (4, 'four'), (5, 'three')], 
      dtype=[('f0', '<i8'), ('f1', '|S5')]

, , , . , .

, , ?

Answer 1

. Pierre GM :

import numpy.lib.recfunctions

. , , , :

numpy.lib.recfunctions.stack_arrays((a,b), autoconvert=True, usemask=False)

(usemask=False , , , , . autoconvert=True a dtype "|S3" "|S5").

Answer 2

dtype, np.rec.fromarrays (aka np.core.records.fromarrays) dtype . ,

In [4]: a = np.core.records.fromarrays( ([1,2], ["one","two"]) )

In [5]: a
Out[5]: 
rec.array([(1, 'one'), (2, 'two')], 
      dtype=[('f0', '<i4'), ('f1', '|S3')])

, dtype f1 3- .

np.concatenate( (a,b) ), numpy , dtpes a b dtype , .

, , dtype :

In [9]: a = np.rec.fromarrays( ([1,2], ["one","two"]), dtype = [('f0', '<i4'), ('f1', '|S8')])

In [10]: b = np.core.records.fromarrays( ([3,4,5], ["three","four","three"]), dtype = [('f0', '<i4'), ('f1', '|S8')])

:

In [11]: np.concatenate( (a,b))
Out[11]: 
array([(1, 'one'), (2, 'two'), (3, 'three'), (4, 'four'), (5, 'three')], 
      dtype=[('f0', '<i4'), ('f1', '|S8')])

---

, dtype "":

In [35]: a = np.core.records.fromarrays( ([1,2], ["one","two"]), dtype = [('f0', '<i4'), ('f1', 'object')])

In [36]: b = np.core.records.fromarrays( ([3,4,5], ["three","four","three"]), dtype = [('f0', '<i4'), ('f1', 'object')])

In [37]: np.concatenate( (a,b))
Out[37]: 
array([(1, 'one'), (2, 'two'), (3, 'three'), (4, 'four'), (5, 'three')], 
      dtype=[('f0', '<i4'), ('f1', '|O4')])

, dtype '|Sn' ( n), , , concatenate.

Answer 3

Will it numpy.lib.recfunctions.merge_arrayswork for you? recfunctions- a little-known subpackage that has not been advertised much, it is a bit clumsy, but can sometimes be useful.