Structs and unpacking in Python

Question

I've been struggling to understand how unpacking works. More specifically this specific example in the documentation is confusing me the most because it won't give me the same output when I input it exactly the same. https://docs.python.org/2/library/struct.html#struct-examples

>>> from struct import *
>>> pack('hhl', 1, 2, 3)
'\x00\x01\x00\x02\x00\x00\x00\x03'
>>> unpack('hhl', '\x00\x01\x00\x02\x00\x00\x00\x03')
(1, 2, 3)
>>> calcsize('hhl')
8

8 byte size makes sense but when I do calcsize('hhl') it's returning 16. Storing pack('hhl', 1, 2, 3) as a variable and unpacking it works fine but using the hex values does not. I receive the error "unpack requires a string argument of length 16" , any idea what the problem is? Thanks

The doc notes that the example was done on a big-endian machine. Note that most (?) popular processors use little-endian byte ordering, so the output string may be different. As long as you unpack the string that was generated with pack, you'll be fine. You can also force a specific byte ordering (see the docs). — jedwards
– jedwards, Commented May 13, 2014 at 15:00
Yes this turned out to be the problem. Formatting it as big-endian gave the results I was expecting — John Smith
– John Smith, Commented May 13, 2014 at 15:03

zmo · Accepted Answer · 2014-05-13 15:24:49Z

2

it's because when you do:

>>> pack('hhl', 1, 2, 3)
'\x00\x01\x00\x02\x00\x00\x00\x03'

it's not returning a string with [ \, x, 0, ...] but a bytearray containing each number given in hex:

>>> pack('hhl', 1, 2, 3)[0]
'\x00'

so, when you're doing:

>>> unpack('hhl', '\x00\x01\x00\x02\x00\x00\x00\x03')

you're actually trying to unpack a string like I was saying previously. Whereas if you do:

>>> s = pack('hhl', 1, 2, 3)
'\x00\x01\x00\x02\x00\x00\x00\x03'
>>> unpack('hhl', s)
(1, 2, 3)

it's working as expected.

Now as @falstru is telling, a way to convert a string with hex into a bytearray that struct can understand is to use binascii.unhexlify, which handles the endianness and the conversion. You can also do it yourself:

>>> unpack('hhl', str(bytearray([1,0,2,0,0,0,0,0,3,0,0,0,0,0,0,0])))
(1, 2, 3)

edited May 13, 2014 at 15:24

answered May 13, 2014 at 15:05

zmo

24.9k4 gold badges58 silver badges91 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

falsetru · Accepted Answer · 2014-05-13 14:56:50Z

1

If you mean the hex string like '0001000200000003', you need to unhexlify it first using binascii.unhexlify (or binascii.a2b_hex)

>>> import struct
>>> import binascii
>>> struct.unpack('>hhl', binascii.unhexlify('0001000200000003'))
(1, 2, 3)

answered May 13, 2014 at 14:56

falsetru

371k69 gold badges770 silver badges660 bronze badges

Collectives™ on Stack Overflow

Structs and unpacking in Python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related