Python namedtuple serialization for json

Question

What is the recommended way to serialize namedtuple for json with saved field names?

Serializing a namedtuple to json only results in values ​​that are serialized, and field names are lost during translation. I would like the fields to be saved as well under json-ized and therefore do the following:

 class foobar(namedtuple('f', 'foo, bar')): __slots__ = () def __iter__(self): yield self._asdict() 

The above is serialized in json, as I expect, and behaves like namedtuple in other places that I use (access to attributes, etc.), unless they have no tuples when they are repeated (which works great for my use case).

What is the “proper way” to convert to json with saved field names?

Answer 1

This is pretty tricky since namedtuple() is a factory that returns a new type derived from tuple . One approach would be to have your class also inherit from UserDict.DictMixin , but tuple.__getitem__ already defined and expects an integer indicating the position of the element, and not the name of its attribute:

 >>> f = foobar('a', 1) >>> f[0] 'a' 

Essentially, namedtuple is an odd position for JSON, since it really is a custom type whose key names are fixed as part of the type definition , unlike a dictionary where key names are stored inside the instance. This prevents the "circular disconnect" of the named set, for example. you cannot decode the dictionary back to namedtuple without any other piece of information, such as an application type marker in dict {'a': 1, '#_type': 'foobar'} , which is a bit hacked.

This is not ideal, but if you only need to code namedtuples into dictionaries, another approach is to extend or modify your JSON encoder in special cases of these types. The following is an example of Python subclasses json.JSONEncoder . This solves the problem of ensuring the correct conversion of nested namedtuples into dictionaries:

 from collections import namedtuple from json import JSONEncoder class MyEncoder(JSONEncoder): def _iterencode(self, obj, markers=None): if isinstance(obj, tuple) and hasattr(obj, '_asdict'): gen = self._iterencode_dict(obj._asdict(), markers) else: gen = JSONEncoder._iterencode(self, obj, markers) for chunk in gen: yield chunk class foobar(namedtuple('f', 'foo, bar')): pass enc = MyEncoder() for obj in (foobar('a', 1), ('a', 1), {'outer': foobar('x', 'y')}): print enc.encode(obj) {"foo": "a", "bar": 1} ["a", 1] {"outer": {"foo": "x", "bar": "y"}} 

Answer 2

If this is just the one namedtuple you are looking for serialization, using its _asdict() method will work (with Python> = 2.7)

 >>> from collections import namedtuple >>> import json >>> FB = namedtuple("FB", ("foo", "bar")) >>> fb = FB(123, 456) >>> json.dumps(fb._asdict()) '{"foo": 123, "bar": 456}' 

Answer 3

It looks like you used a subclass of simplejson.JSONEncoder to do the job, but with the last simple word, this is no longer the case: you should really change the project code. I see no reason why simplejson should not support namedtuples, so I forked the project, added namedtuple support, and I'm currently waiting for my branch to return to the main project . If you need corrections, just pull it out of the plug.

EDIT . It seems that recent versions of simplejson now support this with the namedtuple_as_object option, which defaults to True .

Answer 4

I wrote a library for this: https://github.com/ltworf/typedload

This can go from and to a named tuple and back.

It supports fairly complex nested structures with lists, sets, enumerations, unions, default values. It should cover the most common cases.

edit: the library also supports data classes and attr classes.

Answer 5

It recursively converts namedTuple data to json.

 print(m1) ## Message(id=2, agent=Agent(id=1, first_name='asd', last_name='asd', mail='[e-mail]'), customer=Customer(id=1, first_name='asd', last_name='asd', mail='[e-mail]', phone_number=123123), type='image', content='text', media_url='h.com', la=123123, ls=4512313) def reqursive_to_json(obj): _json = {} if isinstance(obj, tuple): datas = obj._asdict() for data in datas: if isinstance(datas[data], tuple): _json[data] = (reqursive_to_json(datas[data])) else: print(datas[data]) _json[data] = (datas[data]) return _json data = reqursive_to_json(m1) print(data) {'agent': {'first_name': 'asd', 'last_name': 'asd', 'mail': '[e-mail]', 'id': 1}, 'content': 'text', 'customer': {'first_name': 'asd', 'last_name': 'asd', 'mail': '[e-mail]', 'phone_number': 123123, 'id': 1}, 'id': 2, 'la': 123123, 'ls': 4512313, 'media_url': 'h.com', 'type': 'image'} 

Answer 6

There is a more convenient solution - use a decorator (it uses the _fields protected field).

Python 2.7+:

 import json from collections import namedtuple, OrderedDict def json_serializable(cls): def as_dict(self): yield OrderedDict( (name, value) for name, value in zip( self._fields, iter(super(cls, self).__iter__()))) cls.__iter__ = as_dict return cls #Usage: C = json_serializable(namedtuple('C', 'ab c')) print json.dumps(C('abc', True, 3.14)) # or @json_serializable class D(namedtuple('D', 'ab c')): pass print json.dumps(D('abc', True, 3.14)) 

Python 3.6 6+:

 import json from typing import TupleName def json_serializable(cls): def as_dict(self): yield {name: value for name, value in zip( self._fields, iter(super(cls, self).__iter__()))} cls.__iter__ = as_dict return cls # Usage: @json_serializable class C(NamedTuple): a: str b: bool c: float print(json.dumps(C('abc', True, 3.14)) 

Answer 7

This is an old question. However:

Suggestion for all those who have the same question, think carefully about using any private or internal NamedTuple functions because they were earlier and will change over time.

For example, if your NamedTuple is an object with a flat value and you are only interested in serializing it, and not when it is nested in another object, you can avoid the problems that __dict__ can remove _as_dict() or change _as_dict() and just do something like (and yes, this is Python 3, because this answer is for now):

 from typing import NamedTuple class ApiListRequest(NamedTuple): group: str="default" filter: str="*" def to_dict(self): return { 'group': self.group, 'filter': self.filter, } def to_json(self): return json.dumps(self.to_dict()) 

I tried using the default kwarg called for dumps to do to_dict() if available, but it was not called, because NamedTuple can be converted to a list.

Answer 8

The jsonplus library provides a serializer for NamedTuple instances. Use compatibility mode to output simple objects if necessary, but prefer the default values, as this is useful for reverse decoding.