Write boost :: multi_array in hdf5 dataset

Question

Are there any libraries or headers that make it easy to write C ++ or boost :: multi_arrays vectors to HDF5 datasets?

I looked at HDF5 C ++ examples, and they just use C ++ syntax to call c-functions, and they only write static c-arrays to their data sets (see create.cpp).

Am I missing a point !?

Thanks a lot in advance, Adam

Answer 1

Here's how to record the size of N multi_array in HDF5 format

Here is a quick example:

 #include <boost/multi_array.hpp> using boost::multi_array; using boost::extents; // allocate array int NX = 5, NY = 6, NZ = 7; multi_array<double, 3> float_data(extents[NX][NY][NZ]); // initialise the array for (int ii = 0; ii != NX; ii++) for (int jj = 0; jj != NY; jj++) for (int kk = 0; kk != NZ; kk++) float_data[ii][jj][kk] = ii + jj + kk; // // write to HDF5 format // H5::H5File file("SDS.h5", H5F_ACC_TRUNC); write_hdf5(file, "doubleArray", float_data ); 

Here is the code for write_hdf5() .

First we must map the C ++ types to the HDF5 types (from the H5 C ++ api). I commented out the lines that lead to duplicate definitions, because some of the <stdint.h> types (e.g. uint8_t ) are aliases of standard types (e.g. unsigned char )

 #include <cstdint> //!_______________________________________________________________________________________ //! //! map types to HDF5 types //! //! //! \author lg (04 March 2013) //!_______________________________________________________________________________________ template<typename T> struct get_hdf5_data_type { static H5::PredType type() { //static_assert(false, "Unknown HDF5 data type"); return H5::PredType::NATIVE_DOUBLE; } }; template<> struct get_hdf5_data_type<char> { H5::IntType type { H5::PredType::NATIVE_CHAR }; }; //template<> struct get_hdf5_data_type<unsigned char> { H5::IntType type { H5::PredType::NATIVE_UCHAR }; }; //template<> struct get_hdf5_data_type<short> { H5::IntType type { H5::PredType::NATIVE_SHORT }; }; //template<> struct get_hdf5_data_type<unsigned short> { H5::IntType type { H5::PredType::NATIVE_USHORT }; }; //template<> struct get_hdf5_data_type<int> { H5::IntType type { H5::PredType::NATIVE_INT }; }; //template<> struct get_hdf5_data_type<unsigned int> { H5::IntType type { H5::PredType::NATIVE_UINT }; }; //template<> struct get_hdf5_data_type<long> { H5::IntType type { H5::PredType::NATIVE_LONG }; }; //template<> struct get_hdf5_data_type<unsigned long> { H5::IntType type { H5::PredType::NATIVE_ULONG }; }; template<> struct get_hdf5_data_type<long long> { H5::IntType type { H5::PredType::NATIVE_LLONG }; }; template<> struct get_hdf5_data_type<unsigned long long> { H5::IntType type { H5::PredType::NATIVE_ULLONG }; }; template<> struct get_hdf5_data_type<int8_t> { H5::IntType type { H5::PredType::NATIVE_INT8 }; }; template<> struct get_hdf5_data_type<uint8_t> { H5::IntType type { H5::PredType::NATIVE_UINT8 }; }; template<> struct get_hdf5_data_type<int16_t> { H5::IntType type { H5::PredType::NATIVE_INT16 }; }; template<> struct get_hdf5_data_type<uint16_t> { H5::IntType type { H5::PredType::NATIVE_UINT16 }; }; template<> struct get_hdf5_data_type<int32_t> { H5::IntType type { H5::PredType::NATIVE_INT32 }; }; template<> struct get_hdf5_data_type<uint32_t> { H5::IntType type { H5::PredType::NATIVE_UINT32 }; }; template<> struct get_hdf5_data_type<int64_t> { H5::IntType type { H5::PredType::NATIVE_INT64 }; }; template<> struct get_hdf5_data_type<uint64_t> { H5::IntType type { H5::PredType::NATIVE_UINT64 }; }; template<> struct get_hdf5_data_type<float> { H5::FloatType type { H5::PredType::NATIVE_FLOAT }; }; template<> struct get_hdf5_data_type<double> { H5::FloatType type { H5::PredType::NATIVE_DOUBLE }; }; template<> struct get_hdf5_data_type<long double> { H5::FloatType type { H5::PredType::NATIVE_LDOUBLE }; }; 

Then we can use a little magic redirecting template to make the function of the right type to output our data. Since this is the template code, it must live in the header file if you intend to output HDF5 arrays from several source files to your program:

 //!_______________________________________________________________________________________ //! //! write_hdf5 multi_array //! //! \author leo Goodstadt (04 March 2013) //! //!_______________________________________________________________________________________ template<typename T, std::size_t DIMENSIONS, typename hdf5_data_type> void do_write_hdf5(H5::H5File file, const std::string& data_set_name, const boost::multi_array<T, DIMENSIONS>& data, hdf5_data_type& datatype) { // Little endian for x86 //FloatType datatype(get_hdf5_data_type<T>::type()); datatype.setOrder(H5T_ORDER_LE); vector<hsize_t> dimensions(data.shape(), data.shape() + DIMENSIONS); H5::DataSpace dataspace(DIMENSIONS, dimensions.data()); H5::DataSet dataset = file.createDataSet(data_set_name, datatype, dataspace); dataset.write(data.data(), datatype); } template<typename T, std::size_t DIMENSIONS> void write_hdf5(H5::H5File file, const std::string& data_set_name, const boost::multi_array<T, DIMENSIONS>& data ) { get_hdf5_data_type<T> hdf_data_type; do_write_hdf5(file, data_set_name, data, hdf_data_type.type); } 

Answer 2

I do not know anything. C ++ HDF5 wrappers are not so good, especially because they do not allow combining with parallel HDF5. So, I wrote my own wrappers in about 2 hours, and everything works fine. Ultimately, you just need to call it directly (or indirectly if you decide to make bindings in C ++).

Fortunately, both vectors and multi_arrays are contiguous in the store, so you can simply pass data from them directly to HDF5 function calls.