how to make automation google search in python

Question

i have a 100 list in spreadsheet that content many keywords. i want to search in google automatically. i have try this code but does not work. what should i do ?

    from selenium import webdriver 

# Taking input from user 
search_string = input("Input the URL or string you want to search for:") 

# This is done to structure the string 
# into search url.(This can be ignored) 
search_string = search_string.replace(' ', '+') 

# Assigning the browser variable with chromedriver of Chrome. 
# Any other browser and its respective webdriver 
# like geckodriver for Mozilla Firefox can be used 
browser = webdriver.Chrome('chromedriver') 

for i in range(1): 
    matched_elements = browser.get("https://www.google.com/search?q=" +
                                    search_string + "&start=" + str(i))

when i try this code, there was error :

 File "C:\Users\iyyaj\script.py", line 13, in <module>
    browser = webdriver.Chrome('chromedriver')
  File "C:\Users\iyyaj\anaconda3\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 45, in __init__
    super().__init__(
  File "C:\Users\iyyaj\anaconda3\lib\site-packages\selenium\webdriver\chromium\webdriver.py", line 51, in __init__
    self.service.path = DriverFinder.get_path(self.service, options)
  File "C:\Users\iyyaj\anaconda3\lib\site-packages\selenium\webdriver\common\driver_finder.py", line 40, in get_path
    msg = f"Unable to obtain driver for {options.capabilities['browserName']} using Selenium Manager."
AttributeError: 'str' object has no attribute 'capabilities'

i just want to make in one code and the input are in excel or csv file

It is against Google's Webmaster Guidelines and terms of service to submit programmatic search queries. Running this code against Google is likely to cause Google to show captcha for searches from your IP address. — Stephen Ostermiller
– Stephen Ostermiller ♦, Commented Nov 29, 2023 at 15:37

Divyansh Rai · Accepted Answer · 2023-11-29 03:43:46Z

0

The first thing you need to fix that that instead of passing string object as an argument to webdriver.Chrome() you should pass of an instance of Options. Using Options is better and will make it easier for you to automate things when passing headless options as shown below.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

search_string = input("Input the URL or string you want to search for:")

search_string = search_string.replace(' ', '+')

# Chrome Options - This is how you're going to add options and location to the binary.

chrome_options = Options()

# Path to your Chrome WebDriver executable
chrome_driver_path = 'images/chromedriver_mac64'  # I've used mine for now.

# Set the executable path directly in options
chrome_options.binary_location = chrome_driver_path

# Assigning the browser variable with chromedriver of Chrome using Chrome Options
browser = webdriver.Chrome(options=chrome_options)

for i in range(1):
    matched_elements = browser.get("https://www.google.com/search?q=" +
                                  search_string + "&start=" + str(i))

This does work for me. Additionally, you might want to look into headless option which you can add using the below line of code. Headless option will allow you to run it without opening browser window as it may not be feasible for 100s of search results.

chrome_options.add_argument("--headless")

The ability to pass the Chrome WebDriver path directly to webdriver.Chrome() as a string argument (webdriver.Chrome(executable_path=chrome_driver_path)) was introduced in Selenium version 3.8.0. Before this version, the WebDriver path was set using a different method.

Link -> https://github.com/SeleniumHQ/selenium/blob/trunk/py/CHANGES

You might also get blocked for too many frequent requests if you do this often. You should give proper spacing and delay to not be blocked.

answered Nov 29, 2023 at 3:43

Divyansh Rai

1901 silver badge5 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

iyya Over a year ago

i have selenium 4.15.2 and at your link that wa written * Fix bug for unzipping downloaded files. how to fix the bug?

Divyansh Rai Over a year ago

@iyya That bug is already fixed. Did the above solution solve your problem? The link is a release change document and they list out things they fix with each version.

Collectives™ on Stack Overflow

how to make automation google search in python

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related