Python json.loads fails with `ValueError: Invalid control character at: line 1 column 33 (char 33)`
PythonJsonPython Problem Overview
I have a string like this:
s = u"""{"desc": "\u73cd\u54c1\u7f51-\u5168\u7403\u6f6e\u6d41\u5962\u54c1\u7f51\u7edc\u96f6\u552e\u5546 <br \/>\r\nhttp:\/\/www.zhenpin.com\/ <br \/>\r\n<br \/>\r\n200\u591a\u4e2a\u56fd\u9645\u4e00\u7ebf\u54c1\u724c\uff0c\u9876\u7ea7\u4e70\u624b\u5168\u7403\u91c7\u8d2d\uff0c100%\u6b63\u54c1\u4fdd\u969c\uff0c7\u5929\u65e0\u6761\u2026"}"""
json.loads(s) returns error message like this:
ValueError: Invalid control character at: line 1 column 33 (char 33)
Why does this error occur? How can I solve this problem?
Python Solutions
Solution 1 - Python
Another option, perhaps, is to use the strict=False argument
According to http://docs.python.org/2/library/json.html
"If strict is False (True is the default), then control characters will be allowed inside strings. Control characters in this context are those with character codes in the 0-31 range, including '\t' (tab), '\n', '\r' and '\0'."
For example:
json.loads(json_str, strict=False)
Solution 2 - Python
The problem is your unicode string contains carriage returns (\r) and newlines (\n) within a string literal in the JSON data. If they were meant to be part of the string itself, they should be escaped appropriately. If they weren't meant to be part of the string, they shouldn't be in your JSON either.
If you can't fix where you got this JSON string to produce valid JSON, you could either remove the offending characters:
>>> json.loads(s.replace('\r\n', ''))
or escape them manually:
>>> json.loads(s.replace('\r\n', '\\r\\n'))
Solution 3 - Python
The problem is that the character at index 33 is a carriage return control character.
>>> s[33]
u'\r'
According to the JSON spec, valid characters are:
-
Any Unicode character except:
",\, and control-characters (ord(char) < 32). -
The following character sequences are allowed:
\",\\,\/,\b(backspace),\f(form feed),\n(line-feed/new-line),\r(carriage return),\t(tab), or\ufollowed by four hexadecimal digits.
However, in Python you're going to have to double escape control characters (unless the string is raw) because Python also interprets those control characters.
>>> s = ur"""{"desc": "\u73cd\u54c1\u7f51-\u5168\u7403\u6f6e\u6d41\u5962\u54c1\u7f51\u7edc\u96f6\u552e\u5546 <br \/>\r\nhttp:\/\/www.zhenpin.com\/ <br \/>\r\n<br \/>\r\n200\u591a\u4e2a\u56fd\u9645\u4e00\u7ebf\u54c1\u724c\uff0c\u9876\u7ea7\u4e70\u624b\u5168\u7403\u91c7\u8d2d\uff0c100%\u6b63\u54c1\u4fdd\u969c\uff0c7\u5929\u65e0\u6761\u2026"}"""
>>> json.loads(s)
{u'desc': u'\u73cd\u54c1\u7f51-\u5168\u7403\u6f6e\u6d41\u5962\u54c1\u7f51\u7edc\u96f6\u552e\u5546 <br />\r\nhttp://www.zhenpin.com/ <br />\r\n<br />\r\n200\u591a\u4e2a\u56fd\u9645\u4e00\u7ebf\u54c1\u724c\uff0c\u9876\u7ea7\u4e70\u624b\u5168\u7403\u91c7\u8d2d\uff0c100%\u6b63\u54c1\u4fdd\u969c\uff0c7\u5929\u65e0\u6761\u2026'}
References:
Solution 4 - Python
Try to escape your \n and \r:
s = s.replace('\r', '\\r').replace('\n', '\\n')
json.loads(s)
>>> {u'desc': u'\u73cd\u54c1\u7f51-\u5168\u7403\u6f6e\u6d41\u5962\u54c1\u7f51\u7edc\u96f6\u552e\u5546 <br />\r\nhttp://www.zhenpin.com/ <br />\r\n<br />\r\n200\u591a\u4e2a\u56fd\u9645\u4e00\u7ebf\u54c1\u724c\uff0c\u9876\u7ea7\u4e70\u624b\u5168\u7403\u91c7\u8d2d\uff0c100%\u6b63\u54c1\u4fdd\u969c\uff0c7\u5929\u65e0\u6761\u2026'}
Solution 5 - Python
In some cases, this error will be raised when the file actually contains a string with a whitespace in it. Deleting the whitespace will solve the problem.