An array of Cast NumPy to / from a custom C ++ Matrix class using pybind11

Question

I am trying to build my C ++ code with pybind11 . In C ++, I have a Matrix3D class that acts like a three-dimensional array (ie, with the form [n,m,p] ). It has the following basic signature:

 template <class T> class Matrix3D { public: std::vector<T> data; std::vector<size_t> shape; std::vector<size_t> strides; Matrix3D<T>(); Matrix3D<T>(std::vector<size_t>); Matrix3D<T>(const Matrix3D<T>&); T& operator() (int,int,int); }; 

To minimize the shell code, I would like to pass this class directly to and from the NumPy array (copies are not a problem). For example, I would like to directly transfer the function of the following signature:

 Matrix3D<double> func ( const Matrix3D<double>& ); 

using shell code

 #include <pybind11/pybind11.h> #include <pybind11/stl.h> #include <pybind11/numpy.h> namespace py = pybind11; PYBIND11_PLUGIN(example) { py::module m("example", "Module description"); m.def("func", &func, "Function description" ); return m.ptr(); } 

I currently have another function that takes and returns py::array_t<double> . But I would like to avoid the need to write a wrapper function for each function, replacing it with some kind of template.

This was done for the Eigen library (for arrays and (2-D) matrices). But the code is too confusing for me to get my own code. Also, I really only need to wrap only one, simple, class.

Answer 1

Using @kazemakase and @jagerman (the latter via the pybind11 forum ) I figured this out. I changed the class signature a bit so that it can copy data from a pointer:

 template <class T> class Matrix3D { public: std::vector<T> data; std::vector<size_t> shape; std::vector<size_t> strides; Matrix3D<T>(); Matrix3D<T>(std::vector<size_t>, const T *data=NULL); Matrix3D<T>(const Matrix3D<T>&); T& operator() (int,int,int); }; 

The constructor of the class is as follows:

 template <class T> Matrix3D<T>::Matrix3D (std::vector<size_t> shape_, const T *data_ ) { int size; while ( shape.size()<3 ) shape.push_back(1); while ( strides.size()<3 ) strides.push_back(1); for ( int i=0 ; i<3 ; i++ ) shape[i] = shape_[i]; size = shape_[0]*shape_[1]*shape_[2]; strides[0] = shape[2]*shape[1]; strides[1] = shape[2]; strides[2] = 1; while ( data.size()<size ) data.push_back((T)0); if ( data_!=NULL ) for ( int i=0 ; i<size ; i++ ) data[i] = data_[i]; } 

Directly wrap the function of the following signature:

 Matrix3D<double> func ( const Matrix3D<double>& ); 

the following shell code is needed

 #include <pybind11/pybind11.h> #include <pybind11/stl.h> #include <pybind11/numpy.h> namespace py = pybind11; namespace pybind11 { namespace detail { template <typename T> struct type_caster<Matrix3D<T>> { public: PYBIND11_TYPE_CASTER(Matrix3D<T>, _("Matrix3D<T>")); // Conversion part 1 (Python -> C++) bool load(py::handle src, bool convert) { if (!convert && !py::array_t<T>::check_(src)) return false; auto buf = py::array_t<T, py::array::c_style | py::array::forcecast>::ensure(src); if (!buf) return false; auto dims = buf.ndim(); if (dims != 3 ) return false; std::vector<size_t> shape(3); for ( int i=0 ; i<3 ; i++ ) shape[i] = buf.shape()[i]; value = Matrix3D<T>(shape,buf.data()); return true; } //Conversion part 2 (C++ -> Python) static py::handle cast(const Matrix3D<T>& src, py::return_value_policy policy, py::handle parent) { std::vector<size_t> shape (3); std::vector<size_t> strides(3); for ( int i=0 ; i<3 ; i++ ) { shape [i] = src.shape [i]; strides[i] = src.strides[i]*sizeof(T); } py::array a(std::move(shape), std::move(strides), src.data.data() ); return a.release(); } }; }} // namespace pybind11::detail PYBIND11_PLUGIN(example) { py::module m("example", "Module description"); m.def("func", &func, "Function description" ); return m.ptr(); } 

Please note that function overloading is now possible. For example, if an overloaded function existed with the following signature:

 Matrix3D<int > func ( const Matrix3D<int >& ); Matrix3D<double> func ( const Matrix3D<double>& ); 

The following shell function definition is required:

 m.def("func", py::overload_cast<Matrix3D<int >&>(&func), "Function description" ); m.def("func", py::overload_cast<Matrix3D<double>&>(&func), "Function description" ); 

Answer 2

I am not familiar with pybind11, but became interested after reading this question. From a documented document, it looks like you have to write your own type of caster . This is apparently a pretty advanced topic, but seems to be with some effort.

Separated from the documentation, this is the shell of such a converter for converting a C ++ inty :

 namespace pybind11 { namespace detail { template <> struct type_caster<inty> { public: PYBIND11_TYPE_CASTER(inty, _("inty")); // Conversion part 1 (Python->C++) bool load(handle src, bool); //Conversion part 2 (C++ -> Python) static handle cast(inty src, return_value_policy, handle); }; }} // namespace pybind11::detail 

It seems that all you need to do is replace inty with Matrix3D<double> and implement load() and cast() .

Let's see how they did it for Eigen ( eigen.h, line 236 ):

 bool load(handle src, bool) { auto buf = array_t<Scalar>::ensure(src); if (!buf) return false; auto dims = buf.ndim(); if (dims < 1 || dims > 2) return false; auto fits = props::conformable(buf); if (!fits) return false; // Non-comformable vector/matrix types value = Eigen::Map<const Type, 0, EigenDStride>(buf.data(), fits.rows, fits.cols, fits.stride); return true; } 

It does not look too complicated. First make sure the input is of type array_t<Scalar> (maybe array_t<double> in your case). Then they check the size and some conformity (maybe you can skip the last one). And finally, create the Eigen matrix. Since copying is not a problem, at this point just create a new instance of Martix3D<double> and populate it with data from the numpy array.

There are various implementations of cast() for different cases of l-values ​​and constants. I think it's enough to make only one implementation that creates a copy of the data in the new numpy array, if that is normal. See eigen_array_cast() Function eigen_array_cast() how to return an array as the return type of handle .

I have not tested any of this, and there may be more to the process than it seems. Hope this serves as a starting point.