Difference between data and json parameters in python requests package

PythonJsonPython Requests

Python Problem Overview


What is the difference between the data and json parameters in the python Requests package?

It is unclear from the documentation

Does this code:

import requests
import json
d = {'a': 1}
response = requests.post(url, data=json.dumps(d))

Note that we convert the dict to JSON here ☝️ !

Do anything different than:

import requests
import json
d = {'a': 1}
response = requests.post(url, json=d)

If so, what? Does the latter automatically set the content-type in the header to application/json?

Python Solutions


Solution 1 - Python

To answer my own question, it appears my two examples above do the same thing and that using the json parameter does indeed set the content-type in the headers to application/json. In my first example above using the data parameter, the content-type in the headers would need to be set manually.

Solution 2 - Python

As of Dec 2021 the requests documentation about using data vs json is now pretty clear about the difference.

(With our small contribution to it - my PR and your upvotes that confirmed that this used to be an issue. Thank you!).


PS This does NOT answer the OP question but if the FIRST piece of code would be a bit different:

import requests
import json
d = {'a': 1}
response = requests.post(url, data=d)

(note that the dict d is NOT converted to JSON string here!)

And if the second code would be the same (copying it for completeness):

import requests
import json
d = {'a': 1}
response = requests.post(url, json=d)

...then the result would be quite a lot different.

First code would generate a request with content type set to application/x-www-form-urlencoded and the data in this format, so: "a=1"

The second code would generate a request with content type set to application/json and in fact the data in this format, so {"a": 1} - a JSON string.

Solution 3 - Python

Talking only from my experience here, but please note that it should be preferred to use the json field with the dict, rather than dumping the dict in the data field.

Again, talking only from experience, I did not study the code itself, but it seems that the requests library does some more clever json serialization than just json.dumps. When using json.dumps in data field, I have encountered several instances where this resulted in an "value is not a valid dict" error response from the (FastAPI) server. Using the json field instead fixed these issues.

EDIT: I went through the code today. If you use the json parameter, it seems the requests library really only sets Content-Type and dumps it:

from .compat import json as complexjson
content_type = 'application/json'
body = complexjson.dumps(json)
if not isinstance(body, bytes):
    body = body.encode('utf-8')

where in the requests.compat, the json is just:

try:
    import simplejson as json
except ImportError:
    import json

... therefore I really cannot figure out why doing this manually using the data parameter just sometimes fails to work. ¯\_(ツ)_/¯

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
Questionuser1507844View Question on Stackoverflow
Solution 1 - Pythonuser1507844View Answer on Stackoverflow
Solution 2 - PythonGreg DubickiView Answer on Stackoverflow
Solution 3 - PythonTiriarView Answer on Stackoverflow