Encoding nested python object in JSON
PythonJsonEncodePython Problem Overview
I want to encode objects in JSON. But, I can not figure out how to make the output without the string escaping.
import json
class Abc:
def __init__(self):
self.name="abc name"
def toJSON(self):
return json.dumps(self.__dict__, cls=ComplexEncoder)
class Doc:
def __init__(self):
self.abc=Abc()
def toJSON(self):
return json.dumps(self.__dict__, cls=ComplexEncoder)
class ComplexEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, Abc) or isinstance(obj, Doc):
return obj.toJSON()
else:
return json.JSONEncoder.default(self, obj)
doc=Doc()
print doc.toJSON()
The result is (the dumps returns a string representation, that's why the " are escaped)
{"abc": "{\"name\": \"abc name\"}"}
I want something a little bit different. The expected result is
{"abc": {"name": "abc name"}"}
But I don't see how to... Any hint ?
thanks in advance.
Python Solutions
Solution 1 - Python
my previous sample, with another nested object and your advices :
import json
class Identity:
def __init__(self):
self.name="abc name"
self.first="abc first"
self.addr=Addr()
def reprJSON(self):
return dict(name=self.name, firstname=self.first, address=self.addr)
class Addr:
def __init__(self):
self.street="sesame street"
self.zip="13000"
def reprJSON(self):
return dict(street=self.street, zip=self.zip)
class Doc:
def __init__(self):
self.identity=Identity()
self.data="all data"
def reprJSON(self):
return dict(id=self.identity, data=self.data)
class ComplexEncoder(json.JSONEncoder):
def default(self, obj):
if hasattr(obj,'reprJSON'):
return obj.reprJSON()
else:
return json.JSONEncoder.default(self, obj)
doc=Doc()
print "Str representation"
print doc.reprJSON()
print "Full JSON"
print json.dumps(doc.reprJSON(), cls=ComplexEncoder)
print "Partial JSON"
print json.dumps(doc.identity.addr.reprJSON(), cls=ComplexEncoder)
produces the expected result :
Str representation
{'data': 'all data', 'id': <__main__.Identity instance at 0x1005317e8>}
Full JSON
{"data": "all data", "id": {"name": "abc name", "firstname": "abc first", "address": {"street": "sesame street", "zip": "13000"}}}
Partial JSON
{"street": "sesame street", "zip": "13000"}
Thanks.
Solution 2 - Python
So, the immediate problem is that you're passing the json module a JSON value, which will get encoded as just another string in the JSON value.
The broader problem is that you're greatly overcomplicating this.
Drawing on https://stackoverflow.com/questions/455580/json-datetime-between-python-and-javascript/2680060#2680060, I'd go with something closer to this:
import json
class Abc:
def __init__(self):
self.name="abc name"
def jsonable(self):
return self.name
class Doc:
def __init__(self):
self.abc=Abc()
def jsonable(self):
return self.__dict__
def ComplexHandler(Obj):
if hasattr(Obj, 'jsonable'):
return Obj.jsonable()
else:
raise TypeError, 'Object of type %s with value of %s is not JSON serializable' % (type(Obj), repr(Obj))
doc=Doc()
print json.dumps(doc, default=ComplexHandler)
which gets you:
~$ python nestjson.py
{"abc": "abc name"}
~$
This can be made cleaner/saner/safer (in particular, just grabbing __dict__ isn't generally a recommended thing to do outside debugging/troubleshooting), but it should get the point across. All you need, fundamentally, is a way to get a json-compatible object (whether that's a simple string or number, or a list or dict) out of each "node" in the tree. That object should not be an already-JSON-serialized object, which is what you were doing.
Solution 3 - Python
To avoid repetition of code like in Fred Laurent's answer I overloaded the __iter__() method as follows. This also permits to 'jsonize' list elements, datetime and decimal with no extra dependencies, just use dict().
import datetime
import decimal
class Jsonable(object):
def __iter__(self):
for attr, value in self.__dict__.iteritems():
if isinstance(value, datetime.datetime):
iso = value.isoformat()
yield attr, iso
elif isinstance(value, decimal.Decimal):
yield attr, str(value)
elif(hasattr(value, '__iter__')):
if(hasattr(value, 'pop')):
a = []
for subval in value:
if(hasattr(subval, '__iter__')):
a.append(dict(subval))
else:
a.append(subval)
yield attr, a
else:
yield attr, dict(value)
else:
yield attr, value
class Identity(Jsonable):
def __init__(self):
self.name="abc name"
self.first="abc first"
self.addr=Addr()
class Addr(Jsonable):
def __init__(self):
self.street="sesame street"
self.zip="13000"
class Doc(Jsonable):
def __init__(self):
self.identity=Identity()
self.data="all data"
def main():
doc=Doc()
print "-Dictionary- \n"
print dict(doc)
print "\n-JSON- \n"
print json.dumps(dict(doc), sort_keys=True, indent=4)
if __name__ == '__main__':
main()
The output:
-Dictionary-
{'data': 'all data', 'identity': {'first': 'abc first', 'addr': {'street': 'sesame street', 'zip': '13000'}, 'name': 'abc name'}}
-JSON-
{
"data": "all data",
"identity": {
"addr": {
"street": "sesame street",
"zip": "13000"
},
"first": "abc first",
"name": "abc name"
}
}
Hope it helps! Thanks
Solution 4 - Python
Although all the other solutions I assume they work I find they do have a lot of boilerplate code, when the goal is to only encode nested python objects.
In an article I found an elegant solution, which does exactly what you asked for but without the boilerplate code. As you can even have the de-serialization part for free as well I will show you first a solution to your exact question and then give a cleaner version where the de-serialization will work as well.
Exact solution to your question
import json
class Abc(object):
def __init__(self):
self.name = "abc name"
class Doc(object):
def __init__(self):
self.abc = Abc()
doc = Doc()
# Serialization
json_data = json.dumps(doc, default=lambda o: o.__dict__)
print(json_data)
This will output exactly what you where asking for:
{"abc": {"name": "abc name"}}
More elegant solution to enable serializing and de-seralizing
import json
class Abc(object):
def __init__(self, name: str):
self.name = name
class Doc(object):
def __init__(self, abc):
self.abc = abc
abc = Abc("abc name")
doc = Doc(abc)
# Serialization
json_data = json.dumps(doc, default=lambda o: o.__dict__)
print(json_data)
# De-serialization
decoded_doc = Doc(**json.loads(json_data))
print(decoded_doc)
print(vars(decoded_doc))
This will output the following:
{"abc": {"name": "abc name"}}
<__main__.Doc object at 0x7ff75366f250>
{'abc': {'name': 'abc name'}}
The whole magic works by defining a default lambda function: json_data = json.dumps(doc, default=lambda o: o.__dict__).
Solution 5 - Python
I could not add this as a comment and adding as answer. Fred's final sample was useful for me.I was told jsonpickle does this, but could not get the module to install and run properly. So used the code here. Minor tweak though, I had way too many variables to add by hand to some of the objects. So this little loop simplified things:
def reprJSON(self):
d = dict()
for a, v in self.__dict__.items():
if (hasattr(v, "reprJSON")):
d[a] = v.reprJSON()
else:
d[a] = v
return d
It can be used in any object that has a subclass that is too busy to hand encode. Or can be made a helper for all classes. This also works for the full JSON presentation of member arrays that contain other classes (as long as they implement reprJSON() of course).
Solution 6 - Python
This is what you're looking for: https://github.com/jsonpickle/jsonpickle
It does nested serialization of Python objects and can easily be extended to serialize custom types.
Solution 7 - Python
For more complex serialization I would use jsons, it was published in 2022.
-
Turn Python objects into dicts or (JSON)strings and back
-
No changes are required to your objects
-
Easily customizable and extendable
-
Works with data classes, attrs, and POPOs
pip install jsons class Person: name:str birthday:datetime personObject = Person("Tony", date_of_birth) import jsons json_data = jsons.dumps(personObject, indent=4)