Check for a key in nested dictionaries

Question

I work with API calls and therefore using Python dictionaries.

However, for the same request, I do not always use the same keys, and I would like to know when I can call the key without exception ...

Say I have

test = {'a':{'b':{'c':{'d':'e'}}}} 

Sometimes the key d will exist, sometimes it will not. Sometimes c doesn't even exist.

I would like to somehow check if test['a']['b']['c']['d'] exists on the same line.

What I have tried so far:

• Using test.get('a', {}).get('b', {}).get('c', {}).get('d', {}) . Works well, but it's a mess, sometimes I have 5-6 nested dictionaries with really long names ...

• Using try / except block, which is good, but usually if test['a']['b']['c']['d'] does not exist, I will try calling test['a']['b']['e']['f'] to check if it exists, so I will need to add try / catch for each of my if statements, as if I'm not mistaken if the exception is catch, try block is no longer performed.

Perhaps I was trying to find something to do with the reflexive way of doing this, calling the function with the name of my “object” as a string that checks if each key exists, and if so, return the object by itself.

Any thoughts?

The use behind would be, omitting a useless case and suggesting that sometimes the information is in the test ['a'] ['b'] ['c'] ['d'], sometimes in the test ['a'] ['b' ] ['f']:

 if test['a']['b']['c']['d'] **exists**: do sthg with the value of test['a']['b']['c']['d'] elif test['a']['b']['f'] **exists**: do sthg else with the value of test['a']['b']['f'] else: do sthg different 

If I put try / except there, would the first exception not stop executing and not let me execute elif?

Also, I really like the way invoking test['a']['b']['c']['d'] better than providing a list of keys. In fact, I want it to be as transparent as possible for me and for the people who will read / use my code.

Thanks!

Answer 1

If you work with JSON, you can write a simple class to use with dict, as it is imported.

Given the following JSON bit:

 >>> js='{"a": {"b": {"c": {"d": "e"}}}}' 

It will usually be decoded in a Python dict if it consists of pairs of objects:

 >>> import json >>> json.loads(js) {u'a': {u'b': {u'c': {u'd': u'e'}}}} 

Like a regular Python dict, it obeys KeyError missing keys. You can use __missing__ to override KeyErrors and achieve the original structure:

 class Mdict(dict): def __missing__(self, key): return False 

Now check that:

 >>> md=Mdict({'a':Mdict({'b':Mdict({'c':Mdict({'d':'e'})})})}) >>> if md['a']['b']['d']: ... print md['a']['b']['d'] ... elif md['a']['b']['c']: ... print 'elif', md['a']['b']['c'] ... elif {'d': 'e'} 

Each dict level should be Mdict compared to a regular Python dict . However, if you work with JSON, this is very easy to achieve. Just apply object_pairs_hook when decoding JSON:

 >>> js '{"a": {"b": {"c": {"d": "e"}}}}' >>> md=json.loads(js, object_pairs_hook=Mdict) 

And instead, the Mdict class is Mdict , and Python defaults to dict as JSON is decoded.

 >>> md {u'a': {u'b': {u'c': {u'd': u'e'}}}} >>> md['a'] {u'b': {u'c': {u'd': u'e'}}} >>> md['a']['c'] False 

The rest of the example remains unchanged.

Answer 2

You can write a recursive function to check:

 def f(d, keys): if not keys: return True return keys[0] in d and f(d[keys[0]], keys[1:]) 

If the function returns True, the keys exist:

 In [10]: f(test,"abcd") Out[10]: True In [11]: f(test,"abce") Out[11]: False 

If you want to test several key combinations:

 for keys in ("abce","abcr","abcd"): if f(test,keys): print(keys) break abcd 

To return a value, this is pretty simple:

 def f(d, keys): if len(keys) == 1: return d[keys[0]] if keys[0] in d else False return keys[0] in d and f(d[keys[0]], keys[1:]) print(f(test,"abcd")) e 

Try again for a few key combinations:

 def test_keys(keys): for keys in keys: val = f(test,keys) if val: return val return False print(test_keys(("abce","abcr","abc"))) 

You can also write the function iteratively:

 def f(d, keys): obj = object for k in keys: d = d.get(k, obj) if d is obj: return False return d print(f(test,"abcd")) e 

If you want to run a condition based on the return values:

 def f(d, keys): obj = object for k in keys: d = d.get(k, obj) if d is obj: return False return d from operator import mul my_actions = {"c": mul(2, 2), "d": lambda: mul(3, 3), "e": lambda: mul(3, 3)} for st in ("abce", "abcd", "abcf"): val = f(test, st) if val: print(my_actions[val]()) 9 

Just check the key combination in the same order as with if / elif, etc.

Answer 3

This is not exactly what you want, because it does not check for existence, but here is a one-line, similar to the dict.get method:

 In [1]: test = {'a':{'b':{'c':{'d':'e'}}}} In [2]: keys = 'abcd' # or ['a', 'b', 'c', 'd'] In [3]: reduce(lambda d, k: d.get(k) if d else None, keys, test) Out[3]: 'e' In [4]: keys = 'abcf' In [5]: reduce(lambda d, k: d.get(k) if d else None, keys, test) 

Unfortunately, it is not very efficient because it does not stop as soon as one of the keys is missing.

Answer 4

I would create a recursive function.

 def get_key(d, *args): if not args: return None val = d.get(args[0], None) if len(args) == 1: return val if isinstance(val, dict): return get_key(val, *args[1:]) 

Answer 5

You can nest try blocks to handle exceptions from both types of missing keys. Relying on try blocks conforms to the EAFP (easier to apologize than permission) Python philosophy is more than an LBYL (look before jumping) test sample before use. In a multi-threaded program, this also helps to avoid the unexpected behavior of TOCTTOU (time to check usage time) caused by another thread modifying the test dictionary between the existence test and using the value.

 try: value_abcd = test['a']['b']['c']['d'] except KeyError: try: value_abf = test['a']['b']['f'] except KeyError: print("do something different") else: print("value_abf is", value_abf) else: print("value_abcd is", value_abcd) 

Since then, I became aware that you have much more than two keys. Locking try with this large number of keys will create an anti-pattern arrowhead . Instead, you can try the following construct to handle all keys at the same level of indentation if access occurs in a function or in a for loop so that it can return or continue . If not, extract the method .

 try: value_abcd = test['a']['b']['c']['d'] except KeyError: pass else: print("value_abcd is", value_abcd) return # or continue if doing this in a loop try: value_abf = test['a']['b']['f'] except KeyError: pass else: print("value_abf is", value_abf) return print("do something different")