XML file inside HDF5, h5py

Question

I use h5py to store data (floating point numbers) in groups. In addition to the data itself, I need to add an additional file (.xml file containing the necessary information) in hdf5. How can I do it? Is my approach wrong?

f = h5py.File('filename.h5') f.create_dataset('/data/1',numpy_array_1) f.create_dataset('/data/2',numpy_array_2) . . 

my h5 tree should look like this:

 / /data /data/1 (numpy_array_1) /data/2 (numpy_array_2) . . /morphology.xml (?) 

Answer 1

One option is to add it as a variable-length string dataset.

http://code.google.com/p/h5py/wiki/HowTo#Variable-length_strings

eg:.

 import h5py xmldata = """<xml> <something> <else>Text</else> </something> </xml> """ # Write the xml file... f = h5py.File('test.hdf5', 'w') str_type = h5py.new_vlen(str) ds = f.create_dataset('something.xml', shape=(1,), dtype=str_type) ds[:] = xmldata f.close() # Read the xml file back... f = h5py.File('test.hdf5', 'r') print f['something.xml'][0] 

Answer 2

If you just need to attach the XML file to the hdf5 file, you can add it as an attribute to the hdf5 file.

 xmlfh = open('morphology.xml', 'rb') h5f.attrs['xml'] = xmlfh.read() 

You can access the xml file, for example:

 h5f.attrs['xml'] 

Please note that you cannot store attributes larger than 64 KB, you may want to compress the file before connecting. You can look at library compression in the standard Python library.

However, this does not make the information in the XML file very accessible. If you want to associate the metadata of each dataset with some metadata in the XML file, you can map them as you need using an XML library such as lxml . You can also add each XML data field to a separate attribute so that you can query the data sets for the XML field, it all depends on what you have in the XML file. Try to think about how you want to get the data later.

You can also create groups for each xml file with its data sets and put it all in one hdf5 file. I do not know how large the files you manage are YMMV.