convert list of strings to dictionary python

Question

Is it possible to convert a string representation of a dictionary where the keys aren't encapsulated in double quotes e.g.

'{output:{OUTPUT_PATH:hdfs://x.x.x.x:X/tmp/x/x,OUTPUT_TYPE:hdfs},''input:{INPUT_TEMPTABLE:sample,INPUT_TYPE:hdfs,INPUT_PATH:hdfs://x.x.x.x:X/sparkStream/sample1/},''process:{query.param:${http.query.param.name},PROCESS_SQL:1,PROCESS_TYPE:sql},''attributes:{path:./,restlistener.remote.source.host:127.0.0.1,filename:1211999192960535,restlistener.remote.user.dn:none,uuid:2b025f49-7d53-49db-8063-24ddda29fc4a}}'

into a dictionary like:

{"output":{"OUTPUT_PATH":"hdfs://x.x.x.x:X/tmp/x/x","OUTPUT_TYPE":"hdfs"},"input":{"INPUT_TEMPTABLE":"sample","INPUT_TYPE":"hdfs","INPUT_PATH":"hdfs://x.x.x.x:X/sparkStream/sample1/"},"process":{"query.param":"${http.query.param.name}","PROCESS_SQL":"1","PROCESS_TYPE":"sql"},"attributes":{"path":"./","restlistener.remote.source.host":"127.0.0.1","filename":"1211999192960535","restlistener.remote.user.dn":"none","uuid":"2b025f49-7d53-49db-8063-24ddda29fc4a"}}

The data can't be pickled or converted into a serialized format like JSON because it is coming from another process in the current format.

How did you get that string in the first place? That doesn't look like any commonly supported format. — deceze
– deceze ♦, Commented Feb 15, 2018 at 17:01
@EdwinvanMierlo the formatting is pretty different. I think this is more of an issue of needing to use pickle or json or yaml. — Grant Williams
– Grant Williams, Commented Feb 15, 2018 at 17:02
Are those strings actually all joined together or are the commas supposed to be between them? — FHTMitchell
– FHTMitchell, Commented Feb 15, 2018 at 17:02
@Ravi Kiran, Can you give us some information on where this is being passed into the list from? My intuition is saying that you are trying to pass a dictionary through a socket or subprocess or something, and if that's the case you might be better off pickling or serializing the data with some method like JSON/YAML/XML — Grant Williams
– Grant Williams, Commented Feb 15, 2018 at 17:04
I reckon this is something generated by OP. They'd be better off learning about JSON perhaps? — FHTMitchell
– FHTMitchell, Commented Feb 15, 2018 at 17:04

Edwin van Mierlo · Accepted Answer · 2018-02-15 18:47:07Z

Without re, which is probably more efficient, and with some assumptions:

the string always start with '{ and ends with }'
top level elements are separated by ,'' and no other ,'' in the data
sublevel elements are separated by , and no other , in the data

then this can be done like this:

ms = "'{output:{OUTPUT_PATH:hdfs://x.x.x.x:X/tmp/x/x,OUTPUT_TYPE:hdfs},''input:{INPUT_TEMPTABLE:sample,INPUT_TYPE:hdfs,INPUT_PATH:hdfs://x.x.x.x:X/sparkStream/sample1/},''process:{query.param:${http.query.param.name},PROCESS_SQL:1,PROCESS_TYPE:sql},''attributes:{path:./,restlistener.remote.source.host:127.0.0.1,filename:1211999192960535,restlistener.remote.user.dn:none,uuid:2b025f49-7d53-49db-8063-24ddda29fc4a}}'"
result = {}
# get rid of '{ and '}
# split on ,''
for e in ms[2:-2].split(",''"):
    # e is top level
    # split on {
    # e[0] is toplevel key
    # e[1] is sublevel
    e = e.split('{',1)
    # p is sublevel
    # if multiple sublevels split on ,
    p = e[1].split(',') if ',' in e[1] else [e[1]]
    i_dict = {}
    for v in p:
        # for each value in p get rid of trailing }
        v = v.rstrip('}')
        # split the value
        # i[0] is sublevel key
        # i[1] is sublevel value
        i = v.split(':',1)
        #add to sublevel dict
        i_dict[i[0]] = i[1]
    #add sublevel dict as value for toplevel
    result[e[0][:-1]] = i_dict
print(result)

output is a dictionary:

{'output': {'OUTPUT_PATH': 'hdfs://x.x.x.x:X/tmp/x/x', 'OUTPUT_TYPE': 'hdfs'}, 'input': {'INPUT_TEMPTABLE': 'sample', 'INPUT_TYPE': 'hdfs', 'INPUT_PATH': 'hdfs://x.x.x.x:X/sparkStream/sample1/'}, 'process': {'query.param': '${http.query.param.name', 'PROCESS_SQL': '1', 'PROCESS_TYPE': 'sql'}, 'attributes': {'path': './', 'restlistener.remote.source.host': '127.0.0.1', 'filename': '1211999192960535', 'restlistener.remote.user.dn': 'none', 'uuid': '2b025f49-7d53-49db-8063-24ddda29fc4a'}}

with the absence of other input strings, this code is not further tested, YMMV.

Collectives™ on Stack Overflow

convert list of strings to dictionary python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related