Why is numpy.float16 breaking OpenBlas / Atlas functions?

Question

Well, I know that float16it is not a real primitive type, but it is imitated by Python / numpy. However, the question arises: if this exists, and Python allows it to be used when multiplying arrays using a function numpy.dot(), why does OpenBlas (or ATLAS) not work? I mean that multiplication works, but there are no parallel computations. Or, again, in a different way (better, in my opinion), why does Python / numpy allow us to use float16it if we cannot use the advanced functions offered by OpenBlas / ATLAS?

Answer 1

Numpy float16is a weird and possibly evil beast. This is an IEEE 754 half-precision floating-point number with a 1-bit sign, 5 bits of the exponent, and 10 bits of the mantissa.

While this is a standard floating point number, it is new and not widely used. Some GPUs support it, but hardware support is not common on the CPU. Newer processors have instructions for converting between 16-bit and 32-bit floats, but there is no support to use it directly in mathematical operations. Because of this, and due to the lack of suitable data types in ordinary lower-level languages, the 16-bit float is slower than its 32-bit counterpart.

. 16- float , 32- float.

:

In [60]: r=random.random(1000000).astype('float32')

In [61]: %timeit r*r
1000 loops, best of 3: 435 us per loop

In [62]: r=random.random(1000000).astype('float16')

In [63]: %timeit r*r
100 loops, best of 3: 10.9 ms per loop

, . :

In [72]: array([3001], dtype='float16') - array([3000], dtype='float16')
Out[72]: array([ 0.], dtype=float32)