Creating a typed array from an array of bytes stored in a node buffer

Question

From node docs regarding creating typed arrays from buffers:

Buffer memory is interpreted as an array, not a byte array. What is, new Uint32Array(new Buffer([1,2,3,4])) creates a 4-element Uint32Array with elements [1,2,3,4] , not a Uint32Array with one element [0x1020304] or [0x4030201] .

This contrasts with simple javascript, where creating a typed representation of an array from ArrayBuffer uses ArrayBuffer memory as bytes (e.g. reinterpret_cast in C ++). I need this behavior in node when working with node buffers.

I could convert the buffer to an ArrayBuffer, but this is too slow for my application. (I tried a lot of methods - but they are all O (n) time.) (Edit: the fastest method I found is this , which is memmove op one and pretty fast, but still has at least short-term memory consumption until then until the reference to the source buffer is freed.)

Is there a way (fast / O (1)) to get a typed array from a buffer using the contents of the buffer as bytes instead of elements? (the required size of the array element is> 1 byte, of course.)

Answer 1

As far as I know, this cannot be done without making copies of the data in memory. Even your new Uint32Array(new Buffer([1,2,3,4])) example new Uint32Array(new Buffer([1,2,3,4])) internally does this (which means it is not O (1)).

Note that typed arrays are just ArrayBuffer views (not Buffer , which is not possible). new Uint32Array(array) creates an ArrayBuffer of 4 * array.length bytes. You can access it using uint32Array.buffer . The constructor considers your Buffer different from the usual Array .

The best solution I know is the one you already found .

Another problem with using Uint32Array for what you are trying to do is that it depends on the platform byte order . You can either iterate over Buffer as, or use a DataView if you want to be safe.

Answer 2

As with node 4.0, buffers are Uint8Arrays, and you can directly create new views on them:

 var b = new Buffer([1,2,3,4]); new Uint32Array(b.buffer, b.byteOffset); 

This mini-example assumes that b greater than ~ 4096 (a threshold value for a buffer that owns its own piece of memory and exchange a larger fragment). See Converting a NodeJS binary buffer to JavaScript ArrayBuffer for more information - usually you should call b.slice() to work only with your own memory.