Unable to parse Non ASCII URL on Python?

Question

I wanted to query Freebase API to get the list of teams José Mourinho has played for.

So, the URL i used on my browser is

https://www.googleapis.com/freebase/v1/mqlread?query=[{"name": "José Mourinho","/sports/pro_athlete/teams": [{"mid": null,"team": null,"to": null,"optional": true}]}]

However,

import json
import urllib

service_url="https://www.googleapis.com/freebase/v1/mqlread"
query = '[{"name": "' + "José Mourinho" + '","/sports/pro_athlete/teams": [{"mid": null,"team": null,"to": null,"optional": true}]}]'
url = service_url + '?' + 'query='+query
response = json.loads(urllib.urlopen(url).read())

Gives me an error saying,

UnicodeError: URL u'https://www.googleapis.com/freebase/v1/mqlread?query=[{"name": "Jos\xe9 Mourinho","/sports/pro_athlete/teams": [{"mid": null,"team": null,"to": null,"optional": true}]}]' contains non-ASCII characters

What is the solution to this?

I know that actually. But is there any hack/bypass for this in Python? — John Oliver
– John Oliver, Commented Nov 9, 2015 at 18:56
@RNar I saw that before posting it. The solution gives me the error where I decode it twice. — John Oliver
– John Oliver, Commented Nov 9, 2015 at 18:57

Kirk Strauser · Accepted Answer · 2015-11-09 19:16:13Z

1

I think you skipped over a little bit of the docs. Try this instead:

# coding=UTF-8

import json
import urllib

service_url = "https://www.googleapis.com/freebase/v1/mqlread"
query = [{
    '/sports/pro_athlete/teams': [
        {
            'to': None,
            'optional': True,
            'mid': None,
            'team': None
        }
    ],
    'name': 'José Mourinho'
}]

url = service_url + '?' + urllib.urlencode({'query': json.dumps(query)})
response = json.loads(urllib.urlopen(url).read())

print response

Rather than building the query string yourself, use json.dumps and urllib.urlencode to create it for you. They're good at this.

Note: if you can use the requests package, that last bit could be:

import requests
response = requests.get(service_url, params={'query': json.dumps(query)})

Then you get to skip the URL construction and escaping altogether!

edited Nov 9, 2015 at 19:16

answered Nov 9, 2015 at 19:08

Kirk Strauser

31.1k5 gold badges53 silver badges69 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

John Oliver Over a year ago

Fantastic Solution. Bravo

Collectives™ on Stack Overflow

Unable to parse Non ASCII URL on Python?

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related