Python string to unicode

PythonStringUnicodePython 2.xPython Unicode

Python Problem Overview


> Possible Duplicate:
> How do I treat an ASCII string as unicode and unescape the escaped characters in it in python?
> How do convert unicode escape sequences to unicode characters in a python string

I have a string that contains unicode characters e.g. \u2026 etc. Somehow it is not received to me as unicode, but is received as a str. How do I convert it back to unicode?

>>> a="Hello\u2026"
>>> b=u"Hello\u2026"
>>> print a
Hello\u2026
>>> print b
Hello…
>>> print unicode(a)
Hello\u2026
>>> 

So clearly unicode(a) is not the answer. Then what is?

Python Solutions


Solution 1 - Python

Unicode escapes only work in unicode strings, so this

 a="\u2026"

is actually a string of 6 characters: '', 'u', '2', '0', '2', '6'.

To make unicode out of this, use decode('unicode-escape'):

a="\u2026"
print repr(a)
print repr(a.decode('unicode-escape'))

## '\\u2026'
## u'\u2026'

Solution 2 - Python

Decode it with the unicode-escape codec:

>>> a="Hello\u2026"
>>> a.decode('unicode-escape')
u'Hello\u2026'
>>> print _
Hello…

This is because for a non-unicode string the \u2026 is not recognised but is instead treated as a literal series of characters (to put it more clearly, 'Hello\\u2026'). You need to decode the escapes, and the unicode-escape codec can do that for you.

Note that you can get unicode to recognise it in the same way by specifying the codec argument:

>>> unicode(a, 'unicode-escape')
u'Hello\u2026'

But the a.decode() way is nicer.

Solution 3 - Python

>>> a="Hello\u2026"
>>> print a.decode('unicode-escape')
Hello…

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionprongsView Question on Stackoverflow
Solution 1 - PythongeorgView Answer on Stackoverflow
Solution 2 - PythonChris MorganView Answer on Stackoverflow
Solution 3 - PythonjamylakView Answer on Stackoverflow