How to get Javascript generated content

Question

I am currently having an issue getting the HTML of javascript generated content on this ( https://aca3.accela.com/MILARA/GeneralProperty/PropertyLookUp.aspx ) webpage. It generates the javascript on the page itself. I was wondering what I was doing wrong. The code I'm using is this:

import time
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver import ActionChains
from selenium.webdriver.common.keys import Keys
import selenium.webdriver.support.ui as ui
from bs4 import BeautifulSoup
driver = webdriver.Chrome(executable_path="/Users/MrPete/Downloads/chromedriver_win32/chromedriver")
driver.get('https://aca3.accela.com/MILARA/GeneralProperty/PropertyLookUp.aspx')

profession = Select(driver.find_element_by_xpath('//*[@id="ctl00_PlaceHolderMain_refLicenseeSearchForm_ddlLicenseType"]'))
profession.select_by_value("Pharmacist")
time.sleep(5) # Let the user actually see something!
lName = driver.find_element_by_xpath('//*[@id="ctl00_PlaceHolderMain_refLicenseeSearchForm_txtLastName"]')
lName.send_keys('roy')
search = driver.find_element_by_xpath('//*[@id="ctl00_PlaceHolderMain_btnNewSearch"]')
search.click()
time.sleep(5)
html = driver.execute_script("return document.getElementsByTagName('table')[38].innerHTML")
print(html)

Now, I'm not getting no output, the output I'm getting is:

<tbody><tr>
                                        <td style="white-space:nowrap;"><span class="ACA_SmLabel ACA_SmLabel_FontSize"> Showing 1-13 of 13 </span></td>
                                </tr>
                        </tbody>

Which is (kinda) the title of the table I'm trying to get. What I want as the output is the HTML of the entire table (posted a picture of the table that is generated by javascript. What I'm currently getting is the small title up at the top of the picture, the 'Showing 1-13 of 13', and what I want is the entire table.

@Jackfleeting I clarified what I want as output--the entire table not just that 'title' (it's a span tag really) — Gfellha
– Gfellha, Commented Mar 27, 2020 at 14:37

Jack Fleeting · Accepted Answer · 2020-03-27 14:56:14Z

1

Try changing

html = driver.execute_script("return document.getElementsByTagName('table')[38].innerHTML")
print(html)

To:

target = driver.find_element_by_xpath('//table[@class="ACA_GridView ACA_Grid_Caption"]')
print(target.text)

Output:

Showing 1-13 of 13
License Type
License Number
First Name
Middle Initial
Last Name
Organization Name
DBA/Trade Name
License Status
License Expiration Date
Pharmacist
5302017621
Arthur
James

etc.

answered Mar 27, 2020 at 14:56

Jack Fleeting

25k6 gold badges27 silver badges49 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to get Javascript generated content

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related