Python Selenium cannot click the close button

Question

I am trying to download the data from https://projects.propublica.org/nonprofits for my research. When the page is open, a notification window pops up. I tried to use python selenium to close it. My code is as follows,

from selenium.webdriver import Chrome
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

driver = Chrome()

driver.get('https://projects.propublica.org/nonprofits')
driver.find_element(By.XPATH, "/html/body/div[1]/div/div[2]/p[2]/a").click()

I got the error message: selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div/div[2]/p[2]/a"} (Session info: chrome=99.0.4844.51)

I revised my code as

driver.get('https://projects.propublica.org/nonprofits')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "/html/body/div[1]/div/div[2]/button"))).click()

The error message is TimeoutException(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message: Stacktrace: Backtrace: Ordinal0 [0x005E9943+2595139] ...

Any suggestion to overpass the notification windows is highly appreciated. Thank you.

BTW -- propublica has an API for this type of data. propublica.org/datastore/api/nonprofit-explorer-api — Marcel Wilson
– Marcel Wilson, Commented Mar 14, 2022 at 18:21

Prophet · Accepted Answer · 2022-03-13 22:32:47Z

The element you are trying to click is inside an iframe - you have to switch to that iframe first in order to access this element
You have to add waits to let the elements be loaded before accessing them.
This will work better:

from selenium.webdriver import Chrome
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

driver = Chrome()

driver.get('https://projects.propublica.org/nonprofits')
wait = WebDriverWait(driver, 20)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe.syndicated-modal")))
wait.until(EC.visibility_of_element_located((By.XPATH, "//a[contains(@href,'newsletter-roadblock')]"))).click()

When finished working inside the iframe you will need to switch to the default content with

driver.switch_to.default_content()

Collectives™ on Stack Overflow

Python Selenium cannot click the close button

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related