I am trying to open a URL using mechanize in python. The code executes with no errors, but nothing actually happens. What am I missing? Also, is there a way to set the browser? This is python 2.7.
import mechanize
url='http://www.google.com/'
op = mechanize.Browser() # use mecahnize's browser
op.set_handle_robots(False) #tell the webpage you're not a robot
op.open(url)
mechanize doesn't use real browsers - it is a tool for programmatic web-browsing.
For example, print out the page title after opening the url:
>>> import mechanize
>>> url='http://www.google.com/'
>>> op = mechanize.Browser()
>>> op.set_handle_robots(False)
>>> op.open(url)
<response_seek_wrapper at 0x10247ebd8 whose wrapped object = <closeable_response at 0x102479a70 whose fp = <socket._fileobject object at 0x101903950>>>
>>> op.title()
'Google'
Here's a follow up, how you can submit the Google search form:
import mechanize
url='http://www.google.com/'
op = mechanize.Browser()
op.set_handle_equiv(True)
op.set_handle_gzip(True)
op.set_handle_redirect(True)
op.set_handle_referer(True)
op.set_handle_robots(False)
# pretend you are a real browser
op.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]
op.open(url)
op.select_form(nr=1)
op.form['q'] = 'Does mechanize use a real browser?'
op.submit()
print op.geturl()
Prints:
http://www.google.com/search?hl=en&source=hp&q=Does+mechanize+use+a+real+browser%3F&btnG=Google+Search&gbv=1
If your goal is to open a page in an actual web browser, I suggest instead of using mechanize that you use the webbrowser module included by default in Python 2.7. The simplest use of this module can be demonstrated by the command
>>> import antigravity
which opens up http://xkcd.com/353/ in your browser. The code for this joke module is
import webbrowser
webbrowser.open("http://xkcd.com/353/")
There are many options for customizing the behavior including which browser opens available. You can read up on these in the webbrowser docs.
