0

This image represents the web page from which I am trying to extract the hrefs, hiddens under the details button. On the right hand side you can see the inspection of the web page

I am using Selenium library in Python but despite several attempts I can't get the href. Code used:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import ElementClickInterceptedException
import time

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
chrome_options.add_argument('--disable-gpu')

driver = webdriver.Chrome('chromedriver',chrome_options=chrome_options)
driver.get("https://webgate.ec.europa.eu/rasff-window/screen/search")

cookies = WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.XPATH, "//a[text()='Accept all cookies']"))).click()
search = WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.XPATH,'//*[@id="search"]/div[3]/div[4]/button'))).click()
next_page = driver.find_element(By.XPATH,'//*[@id="main-content"]/app-search-component/div/app-list-component/div/div[3]/mat-paginator/div/div/div[2]/button[2]')

counter = 0
 #15075
while counter <= 50:

    details = WebDriverWait(driver, 30).until(EC.visibility_of_all_elements_located((By.XPATH, "//a[contains(text(),'Details')]")))
    for detail in details:
        print(detail.get_attribute("href"))

    counter += 25
    next_page.click()
    time.sleep(1)

print(details)

driver.close()

I get the following error: cookies = WebDriverWait(driver,30).until(EC.element_to_be_clickable((By.XPATH, "//a[text()='Accept all cookies']"))).click()

selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element ... is not clickable at point (422, 528). Other element would receive the click: ...

Improve this question
1
  • 1
    Show us your attempts. Commented Jul 3, 2023 at 15:20

1 Answer 1

Reset to default
0

Refer the code below:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
driver.maximize_window()
driver.get('https://webgate.ec.europa.eu/rasff-window/screen/search')
wait = WebDriverWait(driver, 30)

# accept cookies
wait.until(EC.element_to_be_clickable((By.XPATH, "//a[text()='Accept all cookies']"))).click()
# Click on Search button
WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.XPATH, "//button[contains(text(),'Search')]"))).click()
# Store all 'Details' web elements
elements = wait.until(EC.visibility_of_all_elements_located((By.XPATH, "//a[contains(text(),'Details')]")))

for element in elements:
    print(element.get_attribute("href"))

Console output:

https://webgate.ec.europa.eu/rasff-window/screen/notification/620237
https://webgate.ec.europa.eu/rasff-window/screen/notification/620545
https://webgate.ec.europa.eu/rasff-window/screen/notification/620512
https://webgate.ec.europa.eu/rasff-window/screen/notification/620588
https://webgate.ec.europa.eu/rasff-window/screen/notification/620544
https://webgate.ec.europa.eu/rasff-window/screen/notification/620422
https://webgate.ec.europa.eu/rasff-window/screen/notification/620420
https://webgate.ec.europa.eu/rasff-window/screen/notification/620548
https://webgate.ec.europa.eu/rasff-window/screen/notification/620506
https://webgate.ec.europa.eu/rasff-window/screen/notification/620577
https://webgate.ec.europa.eu/rasff-window/screen/notification/620568
https://webgate.ec.europa.eu/rasff-window/screen/notification/620379
https://webgate.ec.europa.eu/rasff-window/screen/notification/620397
https://webgate.ec.europa.eu/rasff-window/screen/notification/620523
https://webgate.ec.europa.eu/rasff-window/screen/notification/620383
https://webgate.ec.europa.eu/rasff-window/screen/notification/620508
https://webgate.ec.europa.eu/rasff-window/screen/notification/618752
https://webgate.ec.europa.eu/rasff-window/screen/notification/620374
https://webgate.ec.europa.eu/rasff-window/screen/notification/620248
https://webgate.ec.europa.eu/rasff-window/screen/notification/620258
https://webgate.ec.europa.eu/rasff-window/screen/notification/620201
https://webgate.ec.europa.eu/rasff-window/screen/notification/620312
https://webgate.ec.europa.eu/rasff-window/screen/notification/620179
https://webgate.ec.europa.eu/rasff-window/screen/notification/620064
https://webgate.ec.europa.eu/rasff-window/screen/notification/620187

Process finished with exit code 0

UPDATE:

Root cause of the error: In headless mode, Accept All cookies button was overlapped by some other element. To fix this, you need to maximise the window using following code: chrome_options.add_argument("window-size=1920,1080")

There were more than one issues in your code. Check below the corrected code:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import ElementClickInterceptedException
import time

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument("window-size=1920,1080")
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
chrome_options.add_argument('--disable-gpu')

driver = webdriver.Chrome('chromedriver',chrome_options=chrome_options)
driver.get("https://webgate.ec.europa.eu/rasff-window/screen/search")

cookies = WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.XPATH, "//a[text()='Accept all cookies']"))).click()
search = WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.XPATH,'//*[@id="search"]/div[3]/div[4]/button'))).click()
next_page = WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.XPATH,"//button[@aria-label='Next page']//span")))

counter = 0
 #15075
while counter <= 50:

    details = WebDriverWait(driver, 30).until(EC.visibility_of_all_elements_located((By.XPATH, "//a[contains(text(),'Details')]")))
    for detail in details:
        print(detail.get_attribute("href"))

    counter += 25
    # next_page.click()
    driver.execute_script("arguments[0].click();", next_page)
    time.sleep(10)

print(details)

driver.close()
Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

Edit the question and share your code in the question.
Check the updated answer, use the code from the answer. You will notice that I have made few changes to your code.
The code you provided worked the first time then I got the same error reported in the question with the following small difference: not clickable at point (1183, 1032)
Asking multiple follow-up questions on a same question is a bad idea. If the original issue which you reported has resolved then accept this answer. And then ask a new question with the specific error and your code trials, this way you will get answers from a wider community.
I got the same error reported in the question with the following small difference: not clickable at point (1183, 1032 - Clearly its not same error. Its giving similar error on some other line of your code.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.