Why can't Python parse this JSON data?
PythonJsonParsingPython Problem Overview
I have this JSON in a file:
{
"maps": [
{
"id": "blabla",
"iscategorical": "0"
},
{
"id": "blabla",
"iscategorical": "0"
}
],
"masks": [
"id": "valore"
],
"om_points": "value",
"parameters": [
"id": "valore"
]
}
I wrote this script to print all of the JSON data:
import json
from pprint import pprint
with open('data.json') as f:
data = json.load(f)
pprint(data)
This program raises an exception, though:
Traceback (most recent call last):
File "<pyshell#1>", line 5, in <module>
data = json.load(f)
File "/usr/lib/python3.5/json/__init__.py", line 319, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.5/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.5/json/decoder.py", line 355, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 13 column 13 (char 213)
How can I parse the JSON and extract its values?
Python Solutions
Solution 1 - Python
Your data is not valid JSON format. You have []
when you should have {}
:
[]
are for JSON arrays, which are calledlist
in Python{}
are for JSON objects, which are calleddict
in Python
Here's how your JSON file should look:
{
"maps": [
{
"id": "blabla",
"iscategorical": "0"
},
{
"id": "blabla",
"iscategorical": "0"
}
],
"masks": {
"id": "valore"
},
"om_points": "value",
"parameters": {
"id": "valore"
}
}
Then you can use your code:
import json
from pprint import pprint
with open('data.json') as f:
data = json.load(f)
pprint(data)
With data, you can now also find values like so:
data["maps"][0]["id"]
data["masks"]["id"]
data["om_points"]
Try those out and see if it starts to make sense.
Solution 2 - Python
Your data.json
should look like this:
{
"maps":[
{"id":"blabla","iscategorical":"0"},
{"id":"blabla","iscategorical":"0"}
],
"masks":
{"id":"valore"},
"om_points":"value",
"parameters":
{"id":"valore"}
}
Your code should be:
import json
from pprint import pprint
with open('data.json') as data_file:
data = json.load(data_file)
pprint(data)
Note that this only works in Python 2.6 and up, as it depends upon the with
-statement. In Python 2.5 use from __future__ import with_statement
, in Python <= 2.4, see Justin Peel's answer, which this answer is based upon.
You can now also access single values like this:
data["maps"][0]["id"] # will return 'blabla'
data["masks"]["id"] # will return 'valore'
data["om_points"] # will return 'value'
Solution 3 - Python
Justin Peel's answer is really helpful, but if you are using Python 3 reading JSON should be done like this:
with open('data.json', encoding='utf-8') as data_file:
data = json.loads(data_file.read())
Note: use json.loads
instead of json.load
. In Python 3, json.loads
takes a string parameter. json.load
takes a file-like object parameter. data_file.read()
returns a string object.
To be honest, I don't think it's a problem to load all json data into memory in most cases. I see this in JS, Java, Kotlin, cpp, rust almost every language I use. Consider memory issue like a joke to me :)
On the other hand, I don't think you can parse json without reading all of it.
Solution 4 - Python
data = []
with codecs.open('d:\output.txt','rU','utf-8') as f:
for line in f:
data.append(json.loads(line))
Solution 5 - Python
"Ultra JSON" or simply "ujson" can handle having []
in your JSON file input. If you're reading a JSON input file into your program as a list of JSON elements; such as, [{[{}]}, {}, [], etc...]
ujson can handle any arbitrary order of lists of dictionaries, dictionaries of lists.
You can find ujson in the Python package index and the API is almost identical to Python's built-in json
library.
ujson is also much faster if you're loading larger JSON files. You can see the performance details in comparison to other Python JSON libraries in the same link provided.
Solution 6 - Python
If you're using Python3, you can try changing your (connection.json
file) JSON to:
{
"connection1": {
"DSN": "con1",
"UID": "abc",
"PWD": "1234",
"connection_string_python":"test1"
}
,
"connection2": {
"DSN": "con2",
"UID": "def",
"PWD": "1234"
}
}
Then using the following code:
connection_file = open('connection.json', 'r')
conn_string = json.load(connection_file)
conn_string['connection1']['connection_string_python'])
connection_file.close()
>>> test1
Solution 7 - Python
Here you go with modified data.json
file:
{
"maps": [
{
"id": "blabla",
"iscategorical": "0"
},
{
"id": "blabla",
"iscategorical": "0"
}
],
"masks": [{
"id": "valore"
}],
"om_points": "value",
"parameters": [{
"id": "valore"
}]
}
You can call or print data on console by using below lines:
import json
from pprint import pprint
with open('data.json') as data_file:
data_item = json.load(data_file)
pprint(data_item)
Expected output for print(data_item['parameters'][0]['id'])
:
{'maps': [{'id': 'blabla', 'iscategorical': '0'},
{'id': 'blabla', 'iscategorical': '0'}],
'masks': [{'id': 'valore'}],
'om_points': 'value',
'parameters': [{'id': 'valore'}]}
Expected output for print(data_item['parameters'][0]['id'])
:
valore
Solution 8 - Python
There are two types in this parsing.
- Parsing data from a file from a system path
- Parsing JSON from remote URL.
From a file, you can use the following
import json
json = json.loads(open('/path/to/file.json').read())
value = json['key']
print(json['value'])
This arcticle explains the full parsing and getting values using two scenarios.[Parsing JSON using Python][1]
[1]: https://scriptcrunch.com/parse-json-file-using-python/ "Parsing JSON using Python"
Solution 9 - Python
As a python3 user,
The difference between load
and loads
methods is important especially when you read json data from file.
As stated in the docs:
json.load: > Deserialize fp (a .read()-supporting text file or binary > file containing a JSON document) to a Python object using this > conversion table.
json.loads:
> json.loads: Deserialize s (a str, bytes or bytearray instance > containing a JSON document) to a Python object using this conversion > table.
json.load method can directly read opened json document since it is able to read binary file.
with open('./recipes.json') as data:
all_recipes = json.load(data)
As a result, your json data available as in a format specified according to this conversion table:
https://docs.python.org/3.7/library/json.html#json-to-py-table