Navigating multidimensional JSON arrays in Python

Question

I am trying to figure out how to request a JSON array in Python. Can someone show me how to do a simple search and print through a rather complex array?

An example that I use is given here: http://eu.battle.net/api/wow/realm/status

I would like to see, for example, how to find the "Silvermoon" server and print its "population", and then the "control-fraction" in the "Wintergrasp" array.

Now the fragment of the array looks like this:

{"type":"pve","population":"high","queue":false,"wintergrasp":{"area":1,"controlling-faction":0,"status":0,"next":1382350068792},"tol-barad":{"area":21,"controlling-faction":0,"status":0,"next":1382349141932},"status":true,"name":"Silvermoon","slug":"silvermoon","battlegroup":"Cyclone / Wirbelsturm","locale":"en_GB","timezone":"Europe/Paris"} 

Right now, I can access the main array, but it seems I can’t access the sub-arrays without copying all this into another new variable that seems wasteful. I would like to be able to do something like

 import urllib2 import json req = urllib2.Request("http://eu.battle.net/api/wow/realm/status", None, {}) opener = urllib2.build_opener() f = opener.open(req) x = json.load(f) # open the file and read into a variable # search and find the Silvermoon server silvermoon_array = ???? # print the population print silvermoon_array.???? # access the Wintergrasp sub-array wintergrasp_sub = ???? print wintergrasp_sub.???? # print the controlling-faction variable 

It would really help me understand how to access other things.

Answer 1

Python interactive mode is a great way to gradually learn structured data. It is easy to find how to access, say, silvermoon server data:

 >>> data=json.load(urllib2.urlopen("http://eu.battle.net/api/wow/realm/status")) >>> type(data) <type 'dict'> >>> data.keys() [u'realms'] >>> type(data['realms']) <type 'list'> >>> type(data['realms'][0]) <type 'dict'> >>> data['realms'][0].keys() [u'status', u'wintergrasp', u'battlegroup', u'name', u'tol-barad', u'locale', u'queue', u'timezone', u'type', u'slug', u'population'] >>> data['realms'][0]['name'] u'Aegwynn' >>> [realm['name'] for realm in data['realms']].index('Silvermoon') 212 >>> silvermoon= data['realms'][212] >>> silvermoon['population'] u'high' >>> type(silvermoon['wintergrasp']) <type 'dict'> >>> silvermoon['wintergrasp'].keys() [u'status', u'next', u'controlling-faction', u'area'] >>> silvermoon['wintergrasp']['controlling-faction'] >>> silvermoon['population'] u'high' 

If you do not already know about them, you should read dictionary.keys , list.index and a list of concepts to understand what is happening.

After figuring out the data structure, you can finally rewrite the data access to be more readable and efficient:

 realms= data['realms'] realm_from_name= dict( [(realm['name'], realm) for realm in realms]) print realm_from_name['Silvermoon']['population'] print realm_from_name['Silvermoon']['wintergrasp']['controlling-faction'] 

As for copying the array to another variable that is wasteful, you should know that python passes the value by reference . This means that there is no copying when you assign something to a new variable. Here is a simple explanation of passing by vs value following a link

Finally, you seem to worry too much about performance. Python Philosophy First Start Again, Optimize Later . When you are working properly, and if you need better performance, optimize it if it's worth the time.

Answer 2

This does what you want:

 # -*- coding: utf-8 -*- import json import urllib2 def searchListOfDicts(listOfDicts, attr, value): """ Loops through a list of dictionaries and returns matching attribute value pair You can also pass it slug, silvermoon or type, pve returns a list containing all matching dictionaries """ matches = [record for record in listOfDicts if attr in record and record[attr] == value] return matches def myjsonDict(): """ Opens url, grabs json and puts it inside a dictionary """ req = urllib2.Request("http://eu.battle.net/api/wow/realm/status", None, {}) opener = urllib2.build_opener() f = opener.open(req) json_dict = json.load(f) return json_dict jsonDict = myjsonDict() #we want to search inside realms list silverMoonServers = searchListOfDicts(jsonDict["realms"], "name", "Silvermoon") #access first dictionary that matched "name, Silvermoon" query print silverMoonServers[0] print silverMoonServers[0]["wintergrasp"] print silverMoonServers[0]["wintergrasp"]["controlling-faction"] 

Answer 3

In Python, json.loads displays json objects for the python and Arrays to list dictionaries, so further manipulation is the same as with the regular python dict and list structures.

Here's how you can do it with requests and lamdbas :

  import json import requests response = requests.get("http://eu.battle.net/api/wow/realm/status") json_data = json.loads(response.text) # loop through the list of realms to find the one you need (realm_name) get_realm = lambda realm_name, jd: [r for r in jd['realms'] if r['name'] == realm_name] # extract data you need, if there is a match in the list of realms, # return None otherwise get_your_data = lambda realm: ( realm[0]['name'], realm[0]['wintergrasp']['controlling-faction'] ) if realm else None print get_your_data(get_realm('Silvermoon', json_data))