Url decode UTF-8 in Python

Problem

I have spent plenty of time as far as I am newbie in Python. How could I ever decode such a URL:

example.com?title=%D0%BF%D1%80%D0%B0%D0%B2%D0%BE%D0%B2%D0%B0%D1%8F+%D0%B7%D0%B0%D1%89%D0%B8%D1%82%D0%B0

to the following in Python 2.7: example.com? title==прaвовая+защита

url=urllib.unquote(url.encode(“utf8”)) returns a very unattractive result.

There is still no solution; any assistance would be greatly appreciated.

Asked by swordholder

Solution #1

The data is UTF-8 encoded bytes escaped with URL quoting, so you want to decode, with urllib.parse.unquote(), which handles decoding from percent-encoded data to UTF-8 bytes and then to text, transparently:

from urllib.parse import unquote

url = unquote(url)

Demo:

>>> from urllib.parse import unquote
>>> url = 'example.com?title=%D0%BF%D1%80%D0%B0%D0%B2%D0%BE%D0%B2%D0%B0%D1%8F+%D0%B7%D0%B0%D1%89%D0%B8%D1%82%D0%B0'
>>> unquote(url)
'example.com?title=правовая+защита'

urllib.unquote() is the Python 2 counterpart, however it returns a bytestring, which you’d have to decode manually:

from urllib import unquote

url = unquote(url).decode('utf8')

Answered by Martijn Pieters

Solution #2

You can use urllib.parse if you’re running Python 3.

url = """example.com?title=%D0%BF%D1%80%D0%B0%D0%B2%D0%BE%D0%B2%D0%B0%D1%8F+%D0%B7%D0%B0%D1%89%D0%B8%D1%82%D0%B0"""

import urllib.parse
urllib.parse.unquote(url)

gives:

'example.com?title=правовая+защита'

Answered by pavan

Solution #3

With the requests library, you can also achieve the desired result:

import requests

url = "http://www.mywebsite.org/Data%20Set.zip"

print(f"Before: {url}")
print(f"After:  {requests.utils.unquote(url)}")

Output:

$ python3 test_url_unquote.py

Before: http://www.mywebsite.org/Data%20Set.zip
After:  http://www.mywebsite.org/Data Set.zip

If you’re currently using requests and don’t want to use another library, this could be useful.

Answered by ivanleoncz

Solution #4

HTML entities can be included in URLs. This also replaces them.

#from urllib import unquote #earlier python version
from urllib.request import unquote
from html import unescape
unescape(unquote('https://v.w.xy/p1/p22?userId=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx&confirmationToken=7uAf%2fxJoxRTFAZdxslCn2uwVR9vV7cYrlHs%2fl9sU%2frix9f9CnVx8uUT%2bu8y1%2fWCs99INKDnfA2ayhGP1ZD0z%2bodXjK9xL5I4gjKR2xp7p8Sckvb04mddf%2fiG75QYiRevgqdMnvd9N5VZp2ksBc83lDg7%2fgxqIwktteSI9RA3Ux9VIiNxx%2fZLe9dZSHxRq9AA'))

Answered by Roland Puntaier

Post is based on https://stackoverflow.com/questions/16566069/url-decode-utf-8-in-python