Is there a way to override the python json handler?

Question

I have a problem with infinity encoding in json.

json.dumps converts this value to "Infinity" , but I would like it to convert it to null or another value of my choice.

Unfortunately, the default parameter seems to work only if the dumps no longer understand the object, otherwise the default handler seems to bypass it.

Is there a way to pre-encode an object, change the default method for encoding a type / class, or convert a specific type / class to another object before normal encoding?

Answer 1

Take a look at the source here: http://hg.python.org/cpython/file/7ec9255d4189/Lib/json/encoder.py

If you subclass JSONEncoder, you can only override the iterencode(self, o, _one_shot=False) method iterencode(self, o, _one_shot=False) , which has an explicit special case for Infinity (inside an internal function).

To make this reusable, you'll also want to modify __init__ to take some new parameters and store them in the class.

Alternatively, you can choose the json library from pypi that has the corresponding extensibility you are looking for: https://pypi.python.org/pypi?%3Aaction=search&term=json&submit=search

Here is an example:

 import json class FloatEncoder(json.JSONEncoder): def __init__(self, nan_str = "null", **kwargs): super(FloatEncoder,self).__init__(**kwargs) self.nan_str = nan_str #uses code from official python json.encoder module. Same licence applies. def iterencode(self, o, _one_shot=False): """Encode the given object and yield each string representation as available. For example:: for chunk in JSONEncoder().iterencode(bigobject): mysocket.write(chunk) """ if self.check_circular: markers = {} else: markers = None if self.ensure_ascii: _encoder = json.encoder.encode_basestring_ascii else: _encoder = json.encoder.encode_basestring if self.encoding != 'utf-8': def _encoder(o, _orig_encoder=_encoder, _encoding=self.encoding): if isinstance(o, str): o = o.decode(_encoding) return _orig_encoder(o) def floatstr(o, allow_nan=self.allow_nan, _repr=json.encoder.FLOAT_REPR, _inf=json.encoder.INFINITY, _neginf=-json.encoder.INFINITY, nan_str = self.nan_str): # Check for specials. Note that this type of test is processor # and/or platform-specific, so do tests which don't depend on the # internals. if o != o: text = nan_str elif o == _inf: text = 'Infinity' elif o == _neginf: text = '-Infinity' else: return _repr(o) if not allow_nan: raise ValueError( "Out of range float values are not JSON compliant: " + repr(o)) return text _iterencode = json.encoder._make_iterencode( markers, self.default, _encoder, self.indent, floatstr, self.key_separator, self.item_separator, self.sort_keys, self.skipkeys, _one_shot) return _iterencode(o, 0) example_obj = {'name': 'example', 'body': [1.1, {"3.3": 5, "1.1": float('Nan')}, [float('inf'), 2.2]]} print json.dumps(example_obj, cls=FloatEncoder) 

ideone: http://ideone.com/dFWaNj

Answer 2

No, there is no easy way to achieve this. In fact, NaN and Infinity floating point values ​​should not be serialized using json at all, in accordance with the standard. Python uses an extension of the standard. You can make the python encoding of the standard compatible pass allow_nan=False dumps parameter, but this will raise a ValueError for infinity / NaNs, even if you provide a default function.

You have two ways to do what you want:

• Subclass JSONEncoder and change how these values ​​are encoded. Please note that you will need to consider cases where the sequence may contain an infinity value, etc. AFAIK does not have an API to override how objects of a particular class are encoded.

• Make a copy of the object to encode and replace any infinity / nan appearance with None or another object that is encoded the way you want.

A less reliable, but much simpler solution is to change the encoded data, for example, replacing all Infinity substrings with null :

 >>> import re >>> infty_regex = re.compile(r'\bInfinity\b') >>> def replace_infinities(encoded): ... regex = re.compile(r'\bInfinity\b') ... return regex.sub('null', encoded) ... >>> import json >>> replace_infinities(json.dumps([1, 2, 3, float('inf'), 4])) '[1, 2, 3, null, 4]' 

Obviously, you have to consider Infinity text inside lines, etc., so even here a reliable solution is not immediate, but also elegant.

Answer 3

You can do something in this direction:

 import json import math target=[1.1,1,2.2,float('inf'),float('nan'),'a string',int(2)] def ffloat(f): if not isinstance(f,float): return f if math.isnan(f): return 'custom NaN' if math.isinf(f): return 'custom inf' return f print 'regular json:',json.dumps(target) print 'customized:',json.dumps(map(ffloat,target)) 

Print

 regular json: [1.1, 1, 2.2, Infinity, NaN, "a string", 2] customized: [1.1, 1, 2.2, "custom inf", "custom NaN", "a string", 2] 

If you want to process nested data structures, this is also not so difficult:

 import json import math from collections import Mapping, Sequence def nested_json(o): if isinstance(o, float): if math.isnan(o): return 'custom NaN' if math.isinf(o): return 'custom inf' return o elif isinstance(o, basestring): return o elif isinstance(o, Sequence): return [nested_json(item) for item in o] elif isinstance(o, Mapping): return dict((key, nested_json(value)) for key, value in o.iteritems()) else: return o nested_tgt=[1.1,{1.1:float('inf'),3.3:5},(float('inf'),2.2),] print 'regular json:',json.dumps(nested_tgt) print 'nested json',json.dumps(nested_json(nested_tgt)) 

Print

 regular json: [1.1, {"3.3": 5, "1.1": Infinity}, [Infinity, 2.2]] nested json [1.1, {"3.3": 5, "1.1": "custom inf"}, ["custom inf", 2.2]] 

Answer 4

Context

I ran into this problem and did not want to add extra dependency to the project to handle this. In addition, my project supports Python 2.6, 2.7, 3.3, and 3.4 and the simplejson user. Unfortunately, between these versions there are three different implementations of iterencode , so hard coding of a particular version was undesirable.

Hope this helps someone with similar requirements!

Qualifiers

If the coding time / processing power surrounding your json.dumps call is small compared to other components of your project, you can not code / recode JSON to get the desired result using parse_constant kwarg.

Benefits

• It doesn't matter if the end user has Python 2.x json , Python 3.x json or uses simplejson (e.g. import simplejson as json )
• It uses only public json interfaces, which are unlikely to change.

Warnings

• It takes ~ 3 times to encode things
• This implementation does not handle object_pairs_hook, because then it will not work for python 2.6
• Invalid delimiters fail

code

 class StrictJSONEncoder(json.JSONEncoder): def default(self, o): """Make sure we don't instantly fail""" return o def coerce_to_strict(self, const): """ This is used to ultimately *encode* into strict JSON, see `encode` """ # before python 2.7, 'true', 'false', 'null', were include here. if const in ('Infinity', '-Infinity', 'NaN'): return None else: return const def encode(self, o): """ Load and then dump the result using parse_constant kwarg Note that setting invalid separators will cause a failure at this step. """ # this will raise errors in a normal-expected way encoded_o = super(StrictJSONEncoder, self).encode(o) # now: # 1. `loads` to switch Infinity, -Infinity, NaN to None # 2. `dumps` again so you get 'null' instead of extended JSON try: new_o = json.loads(encoded_o, parse_constant=self.coerce_to_strict) except ValueError: # invalid separators will fail here. raise a helpful exception raise ValueError( "Encoding into strict JSON failed. Did you set the separators " "valid JSON separators?" ) else: return json.dumps(new_o, sort_keys=self.sort_keys, indent=self.indent, separators=(self.item_separator, self.key_separator))