How can I serialize a Python dictionary into a string and then return to the dictionary?

Question

How to serialize a Python dictionary into a string and then return to the dictionary? The dictionary will have lists and other dictionaries inside it.

Answer 1

It depends on what you want to use it for. If you are just trying to save it, you should use pickle (or, if you use CPython 2.x, cPickle , which is faster).

 >>> import pickle >>> pickle.dumps({'foo': 'bar'}) b'\x80\x03}q\x00X\x03\x00\x00\x00fooq\x01X\x03\x00\x00\x00barq\x02s.' >>> pickle.loads(_) {'foo': 'bar'} 

If you want it to be readable, you can use json :

 >>> import json >>> json.dumps({'foo': 'bar'}) '{"foo": "bar"}' >>> json.loads(_) {'foo': 'bar'} 

json , however, is very limited in what it will support, while pickle can be used for arbitrary objects (if it doesn't work automatically, the class can define __getstate__ to specify exactly how it should be pickled).

 >>> pickle.dumps(object()) b'\x80\x03cbuiltins\nobject\nq\x00)\x81q\x01.' >>> json.dumps(object()) Traceback (most recent call last): ... TypeError: <object object at 0x7fa0348230c0> is not JSON serializable 

Answer 2

Use the Python json module, or simplejson if you do not have python 2.6 or higher.

Answer 3

If you fully trust the string and don't care about python injection attacks , then this is a very simple solution:

 d = { 'method' : "eval", 'safe' : False, 'guarantees' : None } s = str(d) d2 = eval(s) for k in d2: print k+"="+d2[k] 

If you are safer, then ast.literal_eval is the best bet.

Answer 4

Pickle is great, but I think it's worth mentioning literal_eval from the ast module for an even easier weighting solution if you only serialize the basic python types. This is basically a "safe" version of the notorious eval function, which allows us to evaluate the basic types of python, and not any valid python code.

Example:

 >>> d = {} >>> d[0] = range(10) >>> d['1'] = {} >>> d['1'][0] = range(10) >>> d['1'][1] = 'hello' >>> data_string = str(d) >>> print data_string {0: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], '1': {0: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], 1: 'hello'}} >>> from ast import literal_eval >>> d == literal_eval(data_string) True 

One of the benefits is that serialized data is just python code, so it is very human friendly. Compare this to what you would get with pickle.dumps :

 >>> import pickle >>> print pickle.dumps(d) (dp0 I0 (lp1 I0 aI1 aI2 aI3 aI4 aI5 aI6 aI7 aI8 aI9 asS'1' p2 (dp3 I0 (lp4 I0 aI1 aI2 aI3 aI4 aI5 aI6 aI7 aI8 aI9 asI1 S'hello' p5 ss. 

The disadvantage is that as soon as the data includes a type that is not supported by literal_ast , you will have to go to something else like etching.

Answer 5

One thing json can't do is dict with an index with numbers. Next snippet

 import json dictionary = dict({0:0, 1:5, 2:10}) serialized = json.dumps(dictionary) unpacked = json.loads(serialized) print unpacked[0] 

will throw

 KeyError: 0 

Because keys are converted to strings. cPickle saves the numeric type, and the unpacked dict can be used immediately.

Answer 6

Although not strictly serialization, json may be the wise approach here. This will process nested dicts and lists and data as long as your data is "simple": strings and basic numeric types.

Answer 7

pyyaml should also be mentioned here. It is human readable and can serialize any python object.
Here is pyyaml:
https://bitbucket.org/xi/pyyaml

Answer 8

If you are only trying to serialize, then pprint might also be a good option. It requires the object to be serialized and file stream.

Here is the code:

 from pprint import pprint my_dict = {1:'a',2:'b'} with open('test_results.txt','wb') as f: pprint(my_dict,f) 

I'm not sure if deserializing is easy. I used json for serialization and deserialze before, which works correctly in most cases.

 f.write(json.dumps(my_dict, sort_keys = True, indent = 2, ensure_ascii=True)) 

However, in one particular case, there were errors writing non-unicode data to json.