All geek questions in one place

Extract unsigned char from numpy.uint8 array

I have code to extract a numeric value from a python sequence, and it works well in most cases, but not for a numpy array.

When I try to retrieve an unsigned char, I do the following

unsigned char val = boost::python::extract<unsigned char>(sequence[n]);

where sequence is any python sequence and n is an index. I get the following error:

 TypeError: No registered converter was able to produce a C++ rvalue of type unsigned char from this Python object of type numpy.uint8

How can I successfully retrieve unsigned char in C ++? Should I write / register special converters for numpy types? I would prefer to use the same code that I use for other python sequences, and should not write special code that uses PyArrayObject* .

+5
source share
2 answers

You can register a custom-from-python converter with Boost.Python, which handles conversions from scanned NumPy arrays, such as numpy.uint8 , to C ++, scalars such as unsigned char . The python converter user registration consists of three parts:

  • A function that checks if PyObject convertible. Returning NULL means PyObject cannot use the registered converter.
  • A build function that builds a C ++ type from PyObject . This function is called only if converter(PyObject) does not return NULL .
  • The type of C ++ that will be created.

Retrieving a value from a NumPy scan array requires several calls to the NumPy C API:

Here is a complete example demonstrating the use of the enable_numpy_scalar_converter helper enable_numpy_scalar_converter to register specific NumPy array scalars for the corresponding C ++ types.

 #include <boost/cstdint.hpp> #include <boost/python.hpp> #define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION #include <numpy/arrayobject.h> // Mockup functions. /// @brief Mockup function that will explicitly extract a uint8_t /// from the Boost.Python object. boost::uint8_t test_generic_uint8(boost::python::object object) { return boost::python::extract<boost::uint8_t>(object)(); } /// @brief Mockup function that uses automatic conversions for uint8_t. boost::uint8_t test_specific_uint8(boost::uint8_t value) { return value; } /// @brief Mokcup function that uses automatic conversions for int32_t. boost::int32_t test_specific_int32(boost::int32_t value) { return value; } /// @brief Converter type that enables automatic conversions between NumPy /// scalars and C++ types. template <typename T, NPY_TYPES NumPyScalarType> struct enable_numpy_scalar_converter { enable_numpy_scalar_converter() { // Required NumPy call in order to use the NumPy C API within another // extension module. import_array(); boost::python::converter::registry::push_back( &convertible, &construct, boost::python::type_id<T>()); } static void* convertible(PyObject* object) { // The object is convertible if all of the following are true: // - is a valid object. // - is a numpy array scalar. // - its descriptor type matches the type for this converter. return ( object && // Valid PyArray_CheckScalar(object) && // Scalar PyArray_DescrFromScalar(object)->type_num == NumPyScalarType // Match ) ? object // The Python object can be converted. : NULL; } static void construct( PyObject* object, boost::python::converter::rvalue_from_python_stage1_data* data) { // Obtain a handle to the memory block that the converter has allocated // for the C++ type. namespace python = boost::python; typedef python::converter::rvalue_from_python_storage<T> storage_type; void* storage = reinterpret_cast<storage_type*>(data)->storage.bytes; // Extract the array scalar type directly into the storage. PyArray_ScalarAsCtype(object, storage); // Set convertible to indicate success. data->convertible = storage; } }; BOOST_PYTHON_MODULE(example) { namespace python = boost::python; // Enable numpy scalar conversions. enable_numpy_scalar_converter<boost::uint8_t, NPY_UBYTE>(); enable_numpy_scalar_converter<boost::int32_t, NPY_INT>(); // Expose test functions. python::def("test_generic_uint8", &test_generic_uint8); python::def("test_specific_uint8", &test_specific_uint8); python::def("test_specific_int32", &test_specific_int32); }

Interactive use:

 >>> import numpy >>> import example >>> assert(42 == example.test_generic_uint8(42)) >>> assert(42 == example.test_generic_uint8(numpy.uint8(42))) >>> assert(42 == example.test_specific_uint8(42)) >>> assert(42 == example.test_specific_uint8(numpy.uint8(42))) >>> assert(42 == example.test_specific_int32(numpy.int32(42))) >>> example.test_specific_int32(numpy.int8(42)) Traceback (most recent call last): File "<stdin>", line 1, in <module> Boost.Python.ArgumentError: Python argument types in example.test_specific_int32(numpy.int8) did not match C++ signature: test_specific_int32(int) >>> example.test_generic_uint8(numpy.int8(42)) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: No registered converter was able to produce a C++ rvalue of type unsigned char from this Python object of type numpy.int8

A few things to consider when using interactively:

  • Boost.Python was able to extract boost::uint8_t from numpy.uint8 and int Python objects.
  • enable_numpy_scalar_converter does not support promotions. For example, for test_specific_int32() should be safe to accept a numpy.int8 object that advances to a larger scalar type, such as int . If you want to make promotions:
    • convertible() will need to check for NPY_TYPES compatibility
    • construct() should use PyArray_CastScalarToCtype() to attribute the selected massive array value to the desired C ++ type.
+4
source

Here is a slightly more general version of the accepted answer:
https://github.com/stuarteberg/printnum

(The converter is copied from VIGRA C ++ / Python.)

The accepted answer indicates that it does not support casting between scalar types. This converter will allow implicit conversion between any two scalar types (even, say, from int32 to int8 or float32 to uint8 ). I think that overall is better, but there is a slight compromise between convenience and security.

 #include <iostream> #include <boost/python.hpp> #define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION // http://docs.scipy.org/doc/numpy/reference/c-api.array.html#importing-the-api #define PY_ARRAY_UNIQUE_SYMBOL printnum_cpp_module_PyArray_API #include <numpy/arrayobject.h> #include <numpy/arrayscalars.h> /* * Boost python converter for numpy scalars, eg numpy.uint32(123). * Enables automatic conversion from numpy.intXX, floatXX * in python to C++ char, short, int, float, etc. * When casting from float to int (or wide int to narrow int), * normal C++ casting rules apply. * * Like all boost::python converters, this enables automatic conversion for function args * exposed via boost::python::def(), as well as values converted via boost::python::extract<>(). * * Copied from the VIGRA C++ library source code (MIT license). * http://ukoethe.imtqy.com/vigra * https://github.com/ukoethe/vigra */ template <typename ScalarType> struct NumpyScalarConverter { NumpyScalarConverter() { using namespace boost::python; converter::registry::push_back( &convertible, &construct, type_id<ScalarType>()); } // Determine if obj_ptr is a supported numpy.number static void* convertible(PyObject* obj_ptr) { if (PyArray_IsScalar(obj_ptr, Float32) || PyArray_IsScalar(obj_ptr, Float64) || PyArray_IsScalar(obj_ptr, Int8) || PyArray_IsScalar(obj_ptr, Int16) || PyArray_IsScalar(obj_ptr, Int32) || PyArray_IsScalar(obj_ptr, Int64) || PyArray_IsScalar(obj_ptr, UInt8) || PyArray_IsScalar(obj_ptr, UInt16) || PyArray_IsScalar(obj_ptr, UInt32) || PyArray_IsScalar(obj_ptr, UInt64)) { return obj_ptr; } return 0; } static void construct( PyObject* obj_ptr, boost::python::converter::rvalue_from_python_stage1_data* data) { using namespace boost::python; // Grab pointer to memory into which to construct the C++ scalar void* storage = ((converter::rvalue_from_python_storage<ScalarType>*) data)->storage.bytes; // in-place construct the new scalar value ScalarType * scalar = new (storage) ScalarType; if (PyArray_IsScalar(obj_ptr, Float32)) (*scalar) = PyArrayScalar_VAL(obj_ptr, Float32); else if (PyArray_IsScalar(obj_ptr, Float64)) (*scalar) = PyArrayScalar_VAL(obj_ptr, Float64); else if (PyArray_IsScalar(obj_ptr, Int8)) (*scalar) = PyArrayScalar_VAL(obj_ptr, Int8); else if (PyArray_IsScalar(obj_ptr, Int16)) (*scalar) = PyArrayScalar_VAL(obj_ptr, Int16); else if (PyArray_IsScalar(obj_ptr, Int32)) (*scalar) = PyArrayScalar_VAL(obj_ptr, Int32); else if (PyArray_IsScalar(obj_ptr, Int64)) (*scalar) = PyArrayScalar_VAL(obj_ptr, Int64); else if (PyArray_IsScalar(obj_ptr, UInt8)) (*scalar) = PyArrayScalar_VAL(obj_ptr, UInt8); else if (PyArray_IsScalar(obj_ptr, UInt16)) (*scalar) = PyArrayScalar_VAL(obj_ptr, UInt16); else if (PyArray_IsScalar(obj_ptr, UInt32)) (*scalar) = PyArrayScalar_VAL(obj_ptr, UInt32); else if (PyArray_IsScalar(obj_ptr, UInt64)) (*scalar) = PyArrayScalar_VAL(obj_ptr, UInt64); // Stash the memory chunk pointer for later use by boost.python data->convertible = storage; } }; /* * A silly function to test scalar conversion. * The first arg tests automatic function argument conversion. * The second arg is used to demonstrate explicit conversion via boost::python::extract<>() */ void print_number( uint32_t number, boost::python::object other_number ) { using namespace boost::python; std::cout << "The number is: " << number << std::endl; std::cout << "The other number is: " << extract<int16_t>(other_number) << std::endl; } /* * Instantiate the python extension module 'printnum'. * * Example Python usage: * * import numpy as np * from printnum import print_number * print_number( np.uint8(123), np.int64(-456) ) * * ## That prints the following: * # The number is: 123 * # The other number is: -456 */ BOOST_PYTHON_MODULE(printnum) { using namespace boost::python; // http://docs.scipy.org/doc/numpy/reference/c-api.array.html#importing-the-api import_array(); // Register conversion for all scalar types. NumpyScalarConverter<signed char>(); NumpyScalarConverter<short>(); NumpyScalarConverter<int>(); NumpyScalarConverter<long>(); NumpyScalarConverter<long long>(); NumpyScalarConverter<unsigned char>(); NumpyScalarConverter<unsigned short>(); NumpyScalarConverter<unsigned int>(); NumpyScalarConverter<unsigned long>(); NumpyScalarConverter<unsigned long long>(); NumpyScalarConverter<float>(); NumpyScalarConverter<double>(); // Expose our C++ function as a python function. def("print_number", &print_number, (arg("number"), arg("other_number"))); }
+1
source

Source: https://habr.com/ru/post/1205637/

More articles:


All Articles