Unpacking hex floats

Question

I am trying to translate the following Python code in C ++:

import struct import binascii inputstring = ("0000003F" "0000803F" "AD10753F" "00000080") num_vals = 4 for i in range(num_vals): rawhex = inputstring[i*8:(i*8)+8] # <f for little endian float val = struct.unpack("<f", binascii.unhexlify(rawhex))[0] print val # Output: # 0.5 # 1.0 # 0.957285702229 # -0.0 

Thus, it reads the 32-bit value of a hex-encoded string, turns it into a byte array using the unhexlify method unhexlify and interprets it as a little-endian floating-point value.

The following almost works, but the code looks like crappy (and the last 00000080 incorrectly):

 #include <sstream> #include <iostream> int main() { // The hex-encoded string, and number of values are loaded from a file. // The num_vals might be wrong, so some basic error checking is needed. std::string inputstring = "0000003F" "0000803F" "AD10753F" "00000080"; int num_vals = 4; std::istringstream ss(inputstring); for(unsigned int i = 0; i < num_vals; ++i) { char rawhex[8]; // The ifdef is wrong. It is not the way to detect endianness (it's // always defined) #ifdef BIG_ENDIAN rawhex[6] = ss.get(); rawhex[7] = ss.get(); rawhex[4] = ss.get(); rawhex[5] = ss.get(); rawhex[2] = ss.get(); rawhex[3] = ss.get(); rawhex[0] = ss.get(); rawhex[1] = ss.get(); #else rawhex[0] = ss.get(); rawhex[1] = ss.get(); rawhex[2] = ss.get(); rawhex[3] = ss.get(); rawhex[4] = ss.get(); rawhex[5] = ss.get(); rawhex[6] = ss.get(); rawhex[7] = ss.get(); #endif if(ss.good()) { std::stringstream convert; convert << std::hex << rawhex; int32_t val; convert >> val; std::cerr << (*(float*)(&val)) << "\n"; } else { std::ostringstream os; os << "Not enough values in LUT data. Found " << i; os << ". Expected " << num_vals; std::cerr << os.str() << std::endl; throw std::exception(); } } } 

(compiles on OS X 10.7 / gcc-4.2.1 with simple g++ blah.cpp )

In particular, I would like to get rid of the BIG_ENDIAN macro, as I am sure there is a better way to do this, as this post is being discussed.

A few other random data - I can't use Boost (too much dependency for the project). A string usually contains between 1536 (8 ³ * 3) and 98304 float values ​​(32 ³ * 3), at most 786432 (64 ³ * 3)

(edit2: another value added, 00000080 == -0.0 )

Answer 1

This is what we came across, OpenColorIO / src / core / FileFormatIridasLook.cpp

(Perhaps Amardeep's answer with unsigned uint32_t fix will also work)

  // convert hex ascii to int // return true on success, false on failure bool hexasciitoint(char& ival, char character) { if(character>=48 && character<=57) // [0-9] { ival = static_cast<char>(character-48); return true; } else if(character>=65 && character<=70) // [AF] { ival = static_cast<char>(10+character-65); return true; } else if(character>=97 && character<=102) // [af] { ival = static_cast<char>(10+character-97); return true; } ival = 0; return false; } // convert array of 8 hex ascii to f32 // The input hexascii is required to be a little-endian representation // as used in the iridas file format // "AD10753F" -> 0.9572857022285461f on ALL architectures bool hexasciitofloat(float& fval, const char * ascii) { // Convert all ASCII numbers to their numerical representations char asciinums[8]; for(unsigned int i=0; i<8; ++i) { if(!hexasciitoint(asciinums[i], ascii[i])) { return false; } } unsigned char * fvalbytes = reinterpret_cast<unsigned char *>(&fval); #if OCIO_LITTLE_ENDIAN // Since incoming values are little endian, and we're on little endian // preserve the byte order fvalbytes[0] = (unsigned char) (asciinums[1] | (asciinums[0] << 4)); fvalbytes[1] = (unsigned char) (asciinums[3] | (asciinums[2] << 4)); fvalbytes[2] = (unsigned char) (asciinums[5] | (asciinums[4] << 4)); fvalbytes[3] = (unsigned char) (asciinums[7] | (asciinums[6] << 4)); #else // Since incoming values are little endian, and we're on big endian // flip the byte order fvalbytes[3] = (unsigned char) (asciinums[1] | (asciinums[0] << 4)); fvalbytes[2] = (unsigned char) (asciinums[3] | (asciinums[2] << 4)); fvalbytes[1] = (unsigned char) (asciinums[5] | (asciinums[4] << 4)); fvalbytes[0] = (unsigned char) (asciinums[7] | (asciinums[6] << 4)); #endif return true; } 

Answer 2

I think the whole istringstring business is redundant. It is much easier to parse this on your own one digit at a time.

First, create a function to convert the hexadecimal digit to an integer:

 signed char htod(char c) { c = tolower(c); if(isdigit(c)) return c - '0'; if(c >= 'a' && c <= 'f') return c - 'a'; return -1; } 

Then just convert the string to an integer. The code below does not check for errors and assumes greater reliability - but you should be able to fill in the details.

 unsigned long t = 0; for(int i = 0; i < s.length(); ++i) t = (t << 4) & htod(s[i]); 

Then your bobber

 float f = * (float *) &t; 

Answer 3

Below is the updated code modified to remove the #ifdef BIG_ENDIAN . It uses a read method that must be independent of the host byte. It does this by reading hexadecimal bytes (which are slightly oriented in the original string) in the format of the large end of the string, compatible with the iostream std :: hex operator. Once in this format it does not matter what the host byte order is.

In addition, it fixes the error that rawhex needs zero completion to be inserted into convert without interrupting garbage in some cases.

I do not have a large system for testing, so please check your platform. This has been compiled and tested under Cygwin.

 #include <sstream> #include <iostream> int main() { // The hex-encoded string, and number of values are loaded from a file. // The num_vals might be wrong, so some basic error checking is needed. std::string inputstring = "0000003F0000803FAD10753F00000080"; int num_vals = 4; std::istringstream ss(inputstring); size_t const k_DataSize = sizeof(float); size_t const k_HexOctetLen = 2; for (uint32_t i = 0; i < num_vals; ++i) { char rawhex[k_DataSize * k_HexOctetLen + 1]; // read little endian string into memory array for (uint32_t j=k_DataSize; (j > 0) && ss.good(); --j) { ss.read(rawhex + ((j-1) * k_HexOctetLen), k_HexOctetLen); } // terminate the string (needed for safe conversion) rawhex[k_DataSize * k_HexOctetLen] = 0; if (ss.good()) { std::stringstream convert; convert << std::hex << rawhex; uint32_t val; convert >> val; std::cerr << (*(float*)(&val)) << "\n"; } else { std::ostringstream os; os << "Not enough values in LUT data. Found " << i; os << ". Expected " << num_vals; std::cerr << os.str() << std::endl; throw std::exception(); } } }