Why is Haskell / unpack messing with my bytes?

Question

I built a tiny UDP / protobuf receiver and receiver. I spent the morning trying to track down why proto-bode decoding creates errors, only to discover that it is a transmitter (Spoke.hs) that was sending incorrect data.

The code uses unpack to include Lazy.ByteStrings in the lines that the network packet will send. I found unpack on Hoogle. It may not be the function I'm looking for, but its description is as follows: "O (n) Converts a byte string to a string."

Spoke.hs produces the following output:

 chris@gigabyte :~/Dropbox/haskell-workspace/hub/dist/build/spoke$ ./spoke 45 45 ["a","8","4a","6f","68","6e","20","44","6f","65","10","d2","9","1a","10","6a","64","6f","65","40","65","78","61","6d","70","6c","65","2e","63","6f","6d","22","c","a","8","35","35","35","2d","34","33","32","31","10","1"] 

So far, wirehark shows me the data in the packet:

 0a:08:4a:6f:68:6e:20:44:6f:65:10:c3:92:09:1a:10:6a:64:6f:65:40:65:78:61:6d:70:6c:65:2e:63:6f:6d:22:0c:0a:08:35:35:35:2d:34:33:32:31:10 

The length (45) is the same of Spoke.hs and Wireshark.

The last byte is missing in Wireshark (value Ox01), and the stream of central values ​​is different (and one byte is larger in Wireshark).

"65","10","d2","9" in Spoke.hs vs 65:10:c3:92:09 in Wireshark.

Since 0x10 is a DLE, it seemed to me that some acceleration was probably happening, but I don’t know why.

I have many years of trust in Wireshark and only a few tens of hours of Haskell experience, so I suggested that this is the code that is to blame.

Any suggestions appreciated.

 -- Spoke.hs: module Main where import Data.Bits import Network.Socket -- hiding (send, sendTo, recv, recvFrom) -- import Network.Socket.ByteString import Network.BSD import Data.List import qualified Data.ByteString.Lazy.Char8 as B import Text.ProtocolBuffers.Header (defaultValue, uFromString) import Text.ProtocolBuffers.WireMessage (messageGet, messagePut) import Data.Char (ord, intToDigit) import Numeric import Data.Sequence ((><), fromList) import AddressBookProtos.AddressBook import AddressBookProtos.Person import AddressBookProtos.Person.PhoneNumber import AddressBookProtos.Person.PhoneType data UDPHandle = UDPHandle {udpSocket :: Socket, udpAddress :: SockAddr} opensocket :: HostName -- ^ Remote hostname, or localhost -> String -- ^ Port number or name -> IO UDPHandle -- ^ Handle to use for logging opensocket hostname port = do -- Look up the hostname and port. Either raises an exception -- or returns a nonempty list. First element in that list -- is supposed to be the best option. addrinfos <- getAddrInfo Nothing (Just hostname) (Just port) let serveraddr = head addrinfos -- Establish a socket for communication sock <- socket (addrFamily serveraddr) Datagram defaultProtocol -- Save off the socket, and server address in a handle return $ UDPHandle sock (addrAddress serveraddr) john = Person { AddressBookProtos.Person.id = 1234, name = uFromString "John Doe", email = Just $ uFromString " [e-mail] ", phone = fromList [ PhoneNumber { number = uFromString "555-4321", type' = Just HOME } ] } johnStr = B.unpack (messagePut john) charToHex x = showIntAtBase 16 intToDigit (ord x) "" main::IO() main = do udpHandle <- opensocket "localhost" "4567" sent <- sendTo (udpSocket udpHandle) johnStr (udpAddress udpHandle) putStrLn $ show $ length johnStr putStrLn $ show sent putStrLn $ show $ map charToHex johnStr return () 

Answer 1

The documentation that I see in the bytestring package decompresses as converting ByteString to [Word8] , which does not match String . I would expect some byte difference between ByteString and String , because String is Unicode data and ByteString is just an efficient byte array, but unpack cannot create String in first place.

So, you are probably mistaken in Unicode conversion here, or at least interpret it as Unicode when the underlying data is really missing, and it rarely ends.

Answer 2

I think you will want toString and fromString from utf8-string instead of unpack and pack . This blog post was very helpful to me.