xpath selector returns empty values

Question

i'm trying to manipulate text from all spans under "critical-product-marquee-container" div using python, selenium and xpath selector.

<div data-v-936e71a8="" class="critical-product-marquee-container">
    <div data-v-936e71a8="" class="marquee-text-wrap">
        <div data-v-936e71a8="" class="marquee-text-content">
            <div data-v-936e71a8="" class="marquee-text-text"
                style="animation-duration: 40s;">
                <span data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Surgical &amp; Reusable
                            Masks:</span><span data-v-936e71a8="" class="line-item-bold available"
                        style="padding-right: 2em;"> 1,215,028,138 units </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Surface Wipes:</span><span
                        data-v-936e71a8="" class="line-item-bold available"
                        style="padding-right: 2em;"> 56,023,303 units </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Gowns and Coveralls:</span><span
                        data-v-936e71a8="" class="line-item-bold available"
                        style="padding-right: 2em;"> 22,899,306 units </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Respirators:</span><span
                        data-v-936e71a8="" class="line-item-bold available"
                        style="padding-right: 2em;"> 11,732,566 units </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Face Shields:</span><span
                        data-v-936e71a8="" class="line-item-bold available"
                        style="padding-right: 2em;"> 4,731,581 units </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Hand Sanitizer:</span><span
                        data-v-936e71a8="" class="line-item-bold available"
                        style="padding-right: 2em;"> 527,672 L </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Surface Solutions:</span><span
                        data-v-936e71a8="" class="line-item-bold available"
                        style="padding-right: 2em;"> 48,337 L </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Ventilators:</span><span
                        data-v-936e71a8="" class="line-item-bold critical"
                        style="padding-right: 2em;"> 500 units </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Thermometers:</span><span
                        data-v-936e71a8="" class="line-item-bold critical"
                        style="padding-right: 2em;"> 6,922 units </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Testing Kits:</span><span
                        data-v-936e71a8="" class="line-item-bold critical"
                        style="padding-right: 2em;"> 22,205 units </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Disposable Gloves:</span><span
                        data-v-936e71a8="" class="line-item-bold critical"
                        style="padding-right: 2em;"> 137,455,881 units </span></span></span>
            </div>
            <div data-v-936e71a8="" class="marquee-text-text"
                style="animation-duration: 40s;">
                <span data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Surgical &amp; Reusable
                            Masks:</span><span data-v-936e71a8="" class="line-item-bold available"
                        style="padding-right: 2em;"> 1,215,028,138 units </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Surface Wipes:</span><span
                        data-v-936e71a8="" class="line-item-bold available"
                        style="padding-right: 2em;"> 56,023,303 units </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Gowns and Coveralls:</span><span
                        data-v-936e71a8="" class="line-item-bold available"
                        style="padding-right: 2em;"> 22,899,306 units </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Respirators:</span><span
                        data-v-936e71a8="" class="line-item-bold available"
                        style="padding-right: 2em;"> 11,732,566 units </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Face Shields:</span><span
                        data-v-936e71a8="" class="line-item-bold available"
                        style="padding-right: 2em;"> 4,731,581 units </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Hand Sanitizer:</span><span
                        data-v-936e71a8="" class="line-item-bold available"
                        style="padding-right: 2em;"> 527,672 L </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Surface Solutions:</span><span
                        data-v-936e71a8="" class="line-item-bold available"
                        style="padding-right: 2em;"> 48,337 L </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Ventilators:</span><span
                        data-v-936e71a8="" class="line-item-bold critical"
                        style="padding-right: 2em;"> 500 units </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Thermometers:</span><span
                        data-v-936e71a8="" class="line-item-bold critical"
                        style="padding-right: 2em;"> 6,922 units </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Testing Kits:</span><span
                        data-v-936e71a8="" class="line-item-bold critical"
                        style="padding-right: 2em;"> 22,205 units </span></span></span><span
                    data-v-936e71a8=""><span data-v-936e71a8=""><span
                        data-v-936e71a8="" class="bold">Disposable Gloves:</span><span
                        data-v-936e71a8="" class="line-item-bold critical"
                        style="padding-right: 2em;"> 137,455,881 units </span></span></span>
            </div>
        </div>
    </div>
</div>

my code is as follows:

from selenium import webdriver 
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = webdriver.ChromeOptions();
options.add_argument('headless');
driver = webdriver.Chrome(options=options)
url = 'https://www.rrpcanada.org/#/' # site I'm scraping
driver.get(url)

item_elements = WebDriverWait(driver, 20).until(EC.presence_of_all_elements_located((By.XPATH, '//div[@class="critical-product-marquee-container"]//span[@class="bold"]')))
quantity_elements = WebDriverWait(driver, 20).until(EC.presence_of_all_elements_located((By.XPATH, '//div[@class="critical-product-marquee-container"]//span[contains(@class, "line-item-bold")]')))

it returns a lost of all span elements but when trying to print the span text it prints the first two spans only, the rest are empty. any advice?

'//div[@class="critical-product-marquee-container"]//span[contains(@class, "line-item-bold")] will returns 3 span elements. Is that correct? — Nandan A
– Nandan A, Commented Aug 23, 2021 at 8:44
there are more span elements, for some reason i cant post full DOM. lets assume they are 3 spans. thanks in advance — October Rain
– October Rain, Commented Aug 23, 2021 at 8:45

cruisepandey · Accepted Answer · 2021-08-23 09:15:39Z

basically that's marquee in HTML5, so you have to explicitly wait for each elements.

Code :

driver = webdriver.Chrome(driver_path)
driver.maximize_window()
driver.implicitly_wait(50)
driver.get("https://www.rrpcanada.org/#/")
wait = WebDriverWait(driver, 50)
try:
    wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[class$='unset-margin']"))).click()
except:
    pass

elements = driver.find_elements(By.XPATH, "//div[contains(@class, 'marquee-text-text')]//descendant::span[@class='bold']")
for ele in elements:
    wait.until(EC.visibility_of(ele))
    print(ele.text)
    new_ele = ele.find_element_by_xpath(".//following-sibling::span")
    wait.until(EC.visibility_of(new_ele))
    print(new_ele.text)

Output :

"C:\Program Files\Python39\python.exe" C:/Users/****/PycharmProjects/SeleniumSO/Chrome.py
Surgical & Reusable Masks:
1,215,028,138 units
Surface Wipes:
56,023,303 units
Gowns and Coveralls:
22,899,306 units
Respirators:
11,732,566 units
Face Shields:
4,731,581 units
Hand Sanitizer:
527,672 L
Surface Solutions:
48,337 L
Ventilators:
500 units
Thermometers:
6,922 units
Testing Kits:
22,205 units
Disposable Gloves:
137,455,881 units
Surgical & Reusable Masks:
1,215,028,138 units
Surface Wipes:
56,023,303 units
Gowns and Coveralls:
22,899,306 units
Respirators:

Collectives™ on Stack Overflow

xpath selector returns empty values

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related