encoding string

Question

"afile" is a previously existing file.

handle=open("afile",'r+b')
data=handle.readline()
handle.close() 
#  signgenerator is a hashlib.md5() object

signgenerator.update(data)     
hex=signgenerator.hexdigest()
print(hex) # prints out 061e3f139c80d04f039b7753de5313ce

and write this to a file

 f=open("syncDB.txt",'a')
 #hex=hex.encode('utf-8')
 pickle.dump(hex,f)
 f.close()

But when i read back the file as

while True:
    data=f.readline()
    print(data)

This gives the output:

b'\x80\x03X \x00\x00\x00061e3f139c80d04f039b7753de5313ceq\x00.\x80\x03X \x00\x00\x00d9afd4bb6bc57679f6b10c0b9610d2e0q\x00.\x80\x03X \x00\x00\x008b70452c46285d825d3670d433151841q\x00.\x80\x03X \x00\x00\x00061e3f139c80d04f039b7753de5313ceq\x00.\x80\x03X \x00\x00\x00d9afd4bb6bc57679f6b10c0b9610d2e0q\x00.\x80\x03X \x00\x00\x008b70452c46285d825d3670d433151841q\x00.\x80\x03X \x00\x00\x00b857c3b319036d72cb85fe8a679531b0q\x00.\x80\x03X \x00\x00\x007532fb972cdb019630a2e5a1373fe1c5q\x00.\x80\x03X \x00\x00\x000126bb23767677d0a246d6be1d2e4d5cq\x00.'

How do i encode to get the same hexdigest back from these bytes?? Also I am getting some gibberish characters in syncDb.txt like "€X" after each line.How do I correctly write the data in a readable form??

Community · Accepted Answer · 2020-06-20 09:12:55Z

2

You need to unpickle the data:

pickle.load(open('syncDB.txt', 'r+b'))

What you have there is pickled data. Proof:

>>> import pickle

>>> pickle.loads(b'\x80\x03X \x00\x00\x00061e3f139c80d04f039b7753de5313ceq\x00.\x80\x03X \x00\x00\x00d9afd4bb6bc57679f6b10c0b9610d2e0q\x00.\x80\x03X \x00\x00\x008b70452c46285d825d3670d433151841q\x00.\x80\x03X \x00\x00\x00061e3f139c80d04f039b7753de5313ceq\x00.\x80\x03X \x00\x00\x00d9afd4bb6bc57679f6b10c0b9610d2e0q\x00.\x80\x03X \x00\x00\x008b70452c46285d825d3670d433151841q\x00.\x80\x03X \x00\x00\x00b857c3b319036d72cb85fe8a679531b0q\x00.\x80\x03X \x00\x00\x007532fb972cdb019630a2e5a1373fe1c5q\x00.\x80\x03X \x00\x00\x000126bb23767677d0a246d6be1d2e4d5cq\x00.') '061e3f139c80d04f039b7753de5313ce'

But there's no point in pickling a hex string. You can just put it in the file. The pickle module should be used with more complex structures, like arrays, dicts, or even classes.

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Aug 21, 2011 at 13:43

Gabi Purcaru

31.7k9 gold badges81 silver badges96 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

agf Over a year ago

Pretty sure you should always open a pickled file in binary mode.

agf · Accepted Answer · 2011-08-21 13:42:56Z

1

Don't pickle the hexdigest, just write it out as text.

with open("afile",'rb') as handle:
    data=handle.readline()

signgenerator.update(data)
hex=signgenerator.hexdigest()

with open("syncDB.txt",'ab') as f:
    f.write(hex + '\n')

with open("syncDB.txt",'rb') as f:
    for data in f:
        print(data)

If you really want to use pickle, you need to use the pickle.load function to read the data back from the file.

answered Aug 21, 2011 at 13:42

agf

178k45 gold badges300 silver badges241 bronze badges

2 Comments

Sankalp Over a year ago

f.write(hex + '\n') TypeError: 'str' does not support the buffer interface

agf Over a year ago

You should specify if you're on Python 3 -- my code works on 2. You need to hex.encode() and use b'\n' to convert the unicode to bytes on Python 3.

Collectives™ on Stack Overflow

encoding string

2 Answers 2

1 Comment

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related