How to scroll in selenium (python) when only 1 part of the webpage is scrollable

Question

I would like to be able to scroll down to the bottom of a page. Reason for this is my Xpath elements are not visible until they have been scrolled to(if that makes sense). So, to be able to interact with all elements i want to scroll down to the bottom

However, only a small part of the page is scrollable (the reviews) instead of the entire page. My code for opening the page is as follows:

import pandas as pd
import re
import csv
from time import sleep
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import NoSuchElementException
from msedge.selenium_tools import Edge, EdgeOptions
from selenium.webdriver.common.action_chains import ActionChains

mobile_emulation = {
    "deviceMetrics": { "width": 360, "height": 640, "pixelRatio": 3.0 },
    "userAgent": "Mozilla/5.0 (iPhone; CPU iPhone OS 10_3 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.75 Mobile Safari/535.19" }

options = EdgeOptions()
options.add_experimental_option("mobileEmulation", mobile_emulation)
options.use_chromium = True
driver = Edge(options=options)

# navigate to login screen
driver.get('https://www.google.com/maps/place/AT5/@52.3556437,4.8470182,14z/data=!4m10!1m2!2m1!1sat5!3m6!1s0x47c60995371e3933:0x50f58efcb7962ea8!8m2!3d52.3462526!4d4.8306237!9m1!1b1')
#driver.maximize_window()
driver.set_window_size(250, 1080)
driver.find_element_by_xpath('//*[@id="app"]/div[20]/div/div[2]/div/div[3]/div/span[2]/button').click()

I have tried multiple solutions, but none of them seemed to work.

I have tried

browser.execute_script("window.scrollTo(0,document.body.scrollHeight)")

and

while driver.find_element_by_tag_name('div'):
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
    Divs=driver.find_element_by_xpath('//*[@id="app"]/div[11]/div[2]/div/div[2]/div[2]/div[43]/div[2]/div') ### This is the last element i want to interact with
    if 'End of Results' in Divs:
        print 'end'
        break
    else:
        continue

Nothing i have tried worked. Does anybody know how i can either

scroll down to the bottom of the page

or

be able to interact with the last element without it being visible at the beginning?

There no element to scroll in the google map other than reviews , could you add more details or screen shot on what lement to scrool to — PDHide
– PDHide, Commented Feb 8, 2021 at 13:24

PDHide · Accepted Answer · 2021-02-08 13:59:28Z

1

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

    
url = 'https://www.google.com/maps/place/AT5/@52.3556437,4.8470182,14z/data=!4m10!1m2!2m1!1sat5!3m6!1s0x47c60995371e3933:0x50f58efcb7962ea8!8m2!3d52.3462526!4d4.8306237!9m1!1b1'

browser.get(url)
scroll = WebDriverWait(browser, 15).until(EC.presence_of_element_located(
    (By.CSS_SELECTOR, '[class="section-layout section-scrollbox scrollable-y scrollable-show"]')))
browser.execute_script(
    "arguments[0].scrollBy(0,arguments[0].scrollHeight)", scroll)

use the above code to scroll through review, you have to do scroll on the scroll bar css locator which is

[class="section-layout section-scrollbox scrollable-y scrollable-show"]

edited Feb 8, 2021 at 13:59

answered Feb 8, 2021 at 13:32

PDHide

20.1k2 gold badges43 silver badges57 bronze badges

Sign up to request clarification or add additional context in comments.

13 Comments

PDHide Over a year ago

@Raymondvanzonneveld updated the answer , use the updated one jquery should be available in <script> to use $

PDHide Over a year ago

@Raymondvanzonneveld added wait s now try

PDHide Over a year ago

@Raymondvanzonneveld added full code please try

PDHide Over a year ago

i just tested the code it works fine cou;ld you just use that 4 lines

PDHide Over a year ago

could you try google chromeand see if it works

|

Collectives™ on Stack Overflow

How to scroll in selenium (python) when only 1 part of the webpage is scrollable

1 Answer 1

13 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

13 Comments

Your Answer

Sign up or log in

Post as a guest

Related