Extract Bytes From String That Contains Bytes Data

Question

I'm new to python 3 and trying to extract a message from a bytes array that contains both string and bytes in the message.

I'm unable to extract the bytes message from the decoded bytes array.

Firstly, I decode the bytes array.
Then I do a split on the decoded array.
I get string values upon splitting the array.

I tried to use bytes(v) for v in rest.split() function to try and get the bytes array and then decode it, but wasn't able to.

# The message chunk:
chunk = b"1568077849\n522\nb'l5:d4:auth53:\xc3\x99\xc3\xac\x1fH\xc2\xa3ei6eli1eee'\n"

# I split the chunk into sub categories for further processing:
_, size, rest = (chunk.decode("utf-8")).split('\n', 2)

# _ contains "1568077849"
# size contains "522" 
# rest contains "b'l5:d4:auth53:\xc3\x99\xc3\xac\x1fH\xc2\xa3ei6eli1eee'"

I'm supposed to be able to decode the rest variable (rest.decode("utf-8")), but since it's getting stored as string, I'm having a hard time figuring out how can I convert that to bytes and then decode the value.

The expected result: l5:d4:auth53:ÙìH£ei6eli1eee

how did you get this string ? It seems someone create this string in wrong way. — furas
– furas, Commented Sep 10, 2019 at 2:29
It's coming in from a server request = reader._build_request(chunk_meta) chunk = urllib.request.urlopen(request).read() — Alok Nath Saha
– Alok Nath Saha, Commented Sep 10, 2019 at 2:33
as @nathancy mentioned you have to slice it and then you should have correct string l5:d4:auth53:ÙìH£ei6eli1eee — furas
– furas, Commented Sep 10, 2019 at 2:38
@nathancy I need to be able to get the value in bytes to be able to decode it correctly. Currently, your solution is still giving rest as a string ``` print(isinstance(rest, bytes)) -> False ``` — Alok Nath Saha
– Alok Nath Saha, Commented Sep 10, 2019 at 2:38

DaveStSomeWhere · Accepted Answer · 2019-09-10 03:09:47Z

2

This will print your final result:

chunk = b"1568077849\n522\nb'l5:d4:auth53:\xc3\x99\xc3\xac\x1fH\xc2\xa3ei6eli1eee'\n"

l1 = chunk.decode('utf-8').split()[2:]  # Initial decode
#  slice out the embedded byte string "b'  '" characters
l1_string = ''.join([x[:-2] if x[0] != 'b' else x[2:] for x in l1])
l1_bytes = l1_string.encode('utf-8')
l1_final = l1_bytes.decode('utf-8')

print('Results')
print(f'l1_string is {l1_string}')
print(f'l1_bytes is {l1_bytes}')
print(f'l1_final is {l1_final}')

Results
l1_string is l5:d4:auth53:ÙìH£ei6eli1ee
l1_bytes is b'l5:d4:auth53:\xc3\x99\xc3\xacH\xc2\xa3ei6eli1ee'
l1_final is l5:d4:auth53:ÙìH£ei6eli1ee

answered Sep 10, 2019 at 3:09

DaveStSomeWhere

2,5402 gold badges25 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Alok Nath Saha Over a year ago

Thanks Dave !! Took sometime to decode the answer :)

Alok Nath Saha Over a year ago

Dave, I just came across the data and its coming in as: chunk = b"1568077849\n522\nb'l5:d4:auth53:\\xc3\\x99\\xc3\\xac\\x1fH\\xc2\\xa3ei6eli1eee'\n". There are two double slashes in the input. So the above solution is not working as expected.

Alok Nath Saha · Accepted Answer · 2019-09-10 15:11:11Z

0

I was able to get the expected output this way:

 _, size, rest = (chunk.decode("utf-8")).split('\n', 2)
 rest = bytes(rest.replace("b'", "").replace("'", ""), "utf-8").decode("unicode_escape")

Got the clue from this post: Process escape sequences in a string in Python

answered Sep 10, 2019 at 15:11

Alok Nath Saha

3671 gold badge4 silver badges21 bronze badges

Collectives™ on Stack Overflow

Extract Bytes From String That Contains Bytes Data

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related