How to convert unicode string into normal text in python

Question

Consider I have a Unicode string (Not the real unicode but the string that looks like unicode). and I want to get it's utf-8 variant. How can I do it in Python? For example If I have String like:

title = "\\u10d8\\u10e1\\u10e0\\u10d0\\u10d4\\u10da\\u10d8 == \\u10d8\\u10d4\\u10e0\\u10e3\\u10e1\\u10d0\\u10da\\u10d8\\u10db\\u10d8"

How Can I do it so that I get its utf-8 variant (Georgian symbols):

ისრაელი == იერუსალიმი

To say it simply I want to Have code like:

title = "\\u10d8\\u10e1\\u10e0\\u10d0\\u10d4\\u10da\\u10d8 == \\u10d8\\u10d4\\u10e0\\u10e3\\u10e1\\u10d0\\u10da\\u10d8\\u10db\\u10d8"
utfTitle = title.TurnToUTF()
print(utfTitle)

And I want this code to have output:

ისრაელი == იერუსალიმი

See stackoverflow.com/questions/1473577/…

ozking
– ozking

2017-12-28 11:04:21 +00:00
Commented Dec 28, 2017 at 11:04 — ozking
– ozking, Commented Dec 28, 2017 at 11:04
I think this link can help you...

Moshe Slavin
– Moshe Slavin

2017-12-28 11:04:37 +00:00
Commented Dec 28, 2017 at 11:04 — Moshe Slavin
– Moshe Slavin, Commented Dec 28, 2017 at 11:04
@MosheSlavin It doesn't help

Giorgi Tsertsvadze
– Giorgi Tsertsvadze

2017-12-28 11:07:09 +00:00
Commented Dec 28, 2017 at 11:07 — Giorgi Tsertsvadze
– Giorgi Tsertsvadze, Commented Dec 28, 2017 at 11:07
@ozking It doesn't help

Giorgi Tsertsvadze
– Giorgi Tsertsvadze

2017-12-28 11:07:20 +00:00
Commented Dec 28, 2017 at 11:07 — Giorgi Tsertsvadze
– Giorgi Tsertsvadze, Commented Dec 28, 2017 at 11:07

snakecharmerb · Accepted Answer · 2017-12-28 11:18:34Z

7

You can use the unicode-escape codec to get rid of the doubled-backslashes and use the string effectively.

Assuming that title is a str, you will need to encode the string first before decoding back to unicode(str).

>>> t = title.encode('utf-8').decode('unicode-escape')
>>> t
'ისრაელი == იერუსალიმი'

If title is a bytes instance you can decode directly:

>>> t = title.decode('unicode-escape')
>>> t
'ისრაელი == იერუსალიმი'

answered Dec 28, 2017 at 11:18

snakecharmerb

57.2k13 gold badges137 silver badges200 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Chanda Korat · Accepted Answer · 2017-12-28 12:01:43Z

6

Here, you go. Just use decode method and apply unicode_escape

For Python 2.x

title = "\\u10d8\\u10e1\\u10e0\\u10d0\\u10d4\\u10da\\u10d8 == \\u10d8\\u10d4\\u10e0\\u10e3\\u10e1\\u10d0\\u10da\\u10d8\\u10db\\u10d8"
utfTitle = title.decode('unicode_escape')
print(utfTitle)

#output :ისრაელი == იერუსალიმი

For python 3.x

title = "\\u10d8\\u10e1\\u10e0\\u10d0\\u10d4\\u10da\\u10d8 == \\u10d8\\u10d4\\u10e0\\u10e3\\u10e1\\u10d0\\u10da\\u10d8\\u10db\\u10d8"
print(title.encode('ascii').decode('unicode-escape'))

edited Dec 28, 2017 at 12:01

answered Dec 28, 2017 at 11:18

Chanda Korat

2,5812 gold badges22 silver badges24 bronze badges

Comments

MohamadHossein · Accepted Answer · 2019-01-01 17:25:06Z

0

let assume the unicode be str type and convert using decode and unicode-escape method

title="\\u10d8\\u10e1\\u10e0\\u10d0\\u10d4\\u10da\\u10d8 == \\u10d8\\u10d4\\u10e0\\u10e3\\u10e1\\u10d0\\u10da\\u10d8\\u10db\\u10d8"

res1 = title.encode('utf-8')

res2 = res1.decode('unicode-escape')

print(res2)

edited Jan 1, 2019 at 17:25

MohamadHossein

6407 silver badges23 bronze badges

answered Dec 28, 2017 at 14:11

nbm

697 bronze badges

1 Comment

Muhammad Tahir Qaiser Over a year ago

UnicodeEncodeError: 'cp932' codec can't encode character '\u10d8' in position 0: illegal multibyte sequence

Collectives™ on Stack Overflow

How to convert unicode string into normal text in python

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related