Iterating through a JSON object

Python Problem Overview

I am trying to iterate through a JSON object to import data, i.e. title and link. I can't seem to get to the content that is past the :.

JSON:

[    {        "title": "Baby (Feat. Ludacris) - Justin Bieber",        "description": "Baby (Feat. Ludacris) by Justin Bieber on Grooveshark",        "link": "http://listen.grooveshark.com/s/Baby+Feat+Ludacris+/2Bqvdq",        "pubDate": "Wed, 28 Apr 2010 02:37:53 -0400",        "pubTime": 1272436673,        "TinyLink": "http://tinysong.com/d3wI",        "SongID": "24447862",        "SongName": "Baby (Feat. Ludacris)",        "ArtistID": "1118876",        "ArtistName": "Justin Bieber",        "AlbumID": "4104002",        "AlbumName": "My World (Part II);\nhttp://tinysong.com/gQsw",        "LongLink": "11578982",        "GroovesharkLink": "11578982",        "Link": "http://tinysong.com/d3wI"    },    {        "title": "Feel Good Inc - Gorillaz",        "description": "Feel Good Inc by Gorillaz on Grooveshark",        "link": "http://listen.grooveshark.com/s/Feel+Good+Inc/1UksmI",        "pubDate": "Wed, 28 Apr 2010 02:25:30 -0400",        "pubTime": 1272435930    }]

I tried using a dictionary:

def getLastSong(user,limit):
    base_url = 'http://gsuser.com/lastSong/'
    user_url = base_url + str(user) + '/' + str(limit) + "/"
    raw = urllib.urlopen(user_url)
    json_raw= raw.readlines()
    json_object = json.loads(json_raw[0])
    
    #filtering and making it look good.
    gsongs = []
    print json_object
    for song in json_object[0]:   
        print song

This code only prints the information before :. (ignore the Justin Bieber track :))

Python Solutions

Solution 1 - Python

I believe you probably meant:

from __future__ import print_function

for song in json_object:
    # now song is a dictionary
    for attribute, value in song.items():
        print(attribute, value) # example usage

NB: You could use song.iteritems instead of song.items if in Python 2.

Solution 2 - Python

Your loading of the JSON data is a little fragile. Instead of:

json_raw= raw.readlines()
json_object = json.loads(json_raw[0])

you should really just do:

json_object = json.load(raw)

You shouldn't think of what you get as a "JSON object". What you have is a list. The list contains two dicts. The dicts contain various key/value pairs, all strings. When you do json_object[0], you're asking for the first dict in the list. When you iterate over that, with for song in json_object[0]:, you iterate over the keys of the dict. Because that's what you get when you iterate over the dict. If you want to access the value associated with the key in that dict, you would use, for example, json_object[0][song].

None of this is specific to JSON. It's just basic Python types, with their basic operations as covered in any tutorial.

Solution 3 - Python

This question has been out here a long time, but I wanted to contribute how I usually iterate through a JSON object. In the example below, I've shown a hard-coded string that contains the JSON, but the JSON string could just as easily have come from a web service or a file.

import json

def main():

    # create a simple JSON array
    jsonString = '{"key1":"value1","key2":"value2","key3":"value3"}'

    # change the JSON string into a JSON object
    jsonObject = json.loads(jsonString)

    # print the keys and values
    for key in jsonObject:
        value = jsonObject[key]
        print("The key and value are ({}) = ({})".format(key, value))

    pass

if __name__ == '__main__':
    main()

Solution 4 - Python

After deserializing the JSON, you have a python object. Use the regular object methods.

In this case you have a list made of dictionaries:

json_object[0].items()

json_object[0]["title"]

etc.

Solution 5 - Python

I would solve this problem more like this

import json
import urllib2

def last_song(user, limit):
    # Assembling strings with "foo" + str(bar) + "baz" + ... generally isn't 
    # as nice as using real string formatting. It can seem simpler at first, 
    # but leaves you less happy in the long run.
    url = 'http://gsuser.com/lastSong/%s/%d/' % (user, limit)

    # urllib.urlopen is deprecated in favour of urllib2.urlopen
    site = urllib2.urlopen(url)

    # The json module has a function load for loading from file-like objects, 
    # like the one you get from `urllib2.urlopen`. You don't need to turn 
    # your data into a string and use loads and you definitely don't need to 
    # use readlines or readline (there is seldom if ever reason to use a 
    # file-like object's readline(s) methods.)
    songs = json.load(site)

    # I don't know why "lastSong" stuff returns something like this, but 
    # your json thing was a JSON array of two JSON objects. This will 
    # deserialise as a list of two dicts, with each item representing 
    # each of those two songs.
    #
    # Since each of the songs is represented by a dict, it will iterate 
    # over its keys (like any other Python dict). 
    baby, feel_good = songs

    # Rather than printing in a function, it's usually better to 
    # return the string then let the caller do whatever with it. 
    # You said you wanted to make the output pretty but you didn't 
    # mention *how*, so here's an example of a prettyish representation
    # from the song information given.
    return "%(SongName)s by %(ArtistName)s - listen at %(link)s" % baby

Solution 6 - Python

for iterating through JSON you can use this:

json_object = json.loads(json_file)
for element in json_object: 
    for value in json_object['Name_OF_YOUR_KEY/ELEMENT']:
        print(json_object['Name_OF_YOUR_KEY/ELEMENT']['INDEX_OF_VALUE']['VALUE'])

Solution 7 - Python

For Python 3, you have to decode the data you get back from the web server. For instance I decode the data as utf8 then deal with it:


    # example of json data object group with two values of key id
    jsonstufftest = '{"group": {"id": "2", "id": "3"}}
    # always set your headers
    headers = {"User-Agent": "Moz & Woz"}
    # the url you are trying to load and get json from
    url = "http://www.cooljson.com/cooljson.json"
    # in python 3 you can build the request using request.Request 
    req = urllib.request.Request(url, None, headers)
    # try to connect or fail gracefully
    try:
    	response = urllib.request.urlopen(req) # new python 3 code -jc
    except:
    	exit('could not load page, check connection')
    # read the response and DECODE
    html=response.read().decode('utf8') # new python3 code
    # now convert the decoded string into real JSON
    loadedjson = json.loads(html)
    # print to make sure it worked
    print (loadedjson) # works like a charm
    # iterate through each key value
    for testdata in loadedjson['group']:
    	print (accesscount['id']) # should print 2 then 3 if using test json

If you don't decode you will get bytes vs string errors in Python 3.

Solution 8 - Python

Adding another solution (Python 3) - Iterating over json files in a directory and on each file iterating over all objects and printing relevant fields.

See comments in the code.

import os,json

data_path = '/path/to/your/json/files'  

# 1. Iterate over directory
directory = os.fsencode(data_path)
for file in os.listdir(directory):
    filename = os.fsdecode(file)
    # 2. Take only json files
    if filename.endswith(".json"):
        file_full_path=data_path+filename
        # 3. Open json file 
        with open(file_full_path, encoding='utf-8', errors='ignore') as json_data:
            data_in_file = json.load(json_data, strict=False)
            # 4. Iterate over objects and print relevant fields
            for json_object in data_in_file:
                print("ttl: %s, desc: %s" % (json_object['title'],json_object['description']) )

Solution 9 - Python

If you can store the json string in a variable jsn_string

import json

jsn_list = json.loads(json.dumps(jsn_string)) 
   for lis in jsn_list:
       for key,val in lis.items():
           print(key, val)

Output :

title Baby (Feat. Ludacris) - Justin Bieber
description Baby (Feat. Ludacris) by Justin Bieber on Grooveshark
link http://listen.grooveshark.com/s/Baby+Feat+Ludacris+/2Bqvdq
pubDate Wed, 28 Apr 2010 02:37:53 -0400
pubTime 1272436673
TinyLink http://tinysong.com/d3wI
SongID 24447862
SongName Baby (Feat. Ludacris)
ArtistID 1118876
ArtistName Justin Bieber
AlbumID 4104002
AlbumName My World (Part II);
http://tinysong.com/gQsw
LongLink 11578982
GroovesharkLink 11578982
Link http://tinysong.com/d3wI
title Feel Good Inc - Gorillaz
description Feel Good Inc by Gorillaz on Grooveshark
link http://listen.grooveshark.com/s/Feel+Good+Inc/1UksmI
pubDate Wed, 28 Apr 2010 02:25:30 -0400
pubTime 1272435930

Content Type	Original Author	Original Content on Stackoverflow
Question	myusuf3	View Question on Stackoverflow
Solution 1 - Python	tzot	View Answer on Stackoverflow
Solution 2 - Python	Thomas Wouters	View Answer on Stackoverflow
Solution 3 - Python	Dale Moore	View Answer on Stackoverflow
Solution 4 - Python	jcea	View Answer on Stackoverflow
Solution 5 - Python	Mike Graham	View Answer on Stackoverflow
Solution 6 - Python	Keivan	View Answer on Stackoverflow
Solution 7 - Python	jamescampbell	View Answer on Stackoverflow
Solution 8 - Python	RtmY	View Answer on Stackoverflow
Solution 9 - Python	mayankSinha	View Answer on Stackoverflow