There is a table that is being rendered by javaScript on the page. I would like to retrieve that data in my python code. Below I have include
1. Python Code
2. Relavent HTML
3. The things that I have tried.
1. Python Code
<selenium.webdriver.firefox.webelement.FirefoxWebElement (session="70cabf50-eee7-486f-829c-1d4e9b0e0dfe", element="b3a93cc8-6b8f-431c-a117-3fcb97456631")>
I guess I should have played a little more before asking for help
after the second click
data=browser.page_source does the trick
1. Python Code
2. Relavent HTML
3. The things that I have tried.
1. Python Code
from selenium.webdriver import Firefox from selenium.webdriver.firefox.options import Options opts = Options() opts.set_headless() assert opts.headless # Operating in headless mode browser = Firefox(options=opts) url="https://www.sec.gov/cgi-bin/viewer?action=view&cik=1596532&accession_number=0001596532-19-000027&xbrl_type=v#" browser.get(url) ## first click link1=browser.find_elements_by_link_text("Financial Statements") link1[0].click() ## See below for what I have tried here ## second click link2=browser.find_elements_by_link_text("Consolidated Balance Sheets") link2[0].click() ## See below for what I have tried here browser.save_screenshot("file.png") ## file.png had the correct data so I know that the clicks are working correctly ## browser.close() quit() 2. Relavent HTML code # This is the HTML from the initial URL !-- cover_category = 0 --> <li class="accordion"> <a id="menu_cat1" href="#">Cover</a> <ul> <li class="accordion" id="r1" ><a class="xbrlviewer" onClick="javascript:highlight(\ this);" href="javascript:loadReport(1);">Document and Entity Information</a></li> </ul> </li> <li class="accordion"> <a id="menu_cat2" href="#">Financial Statements</a> <ul> <li class="accordion" id="r2" ><a class="xbrlviewer" onClick="javascript:highlight(\ this);" href="javascript:loadReport(2);">Consolidated Balance Sheets</a></li> <li class="accordion" id="r3" ><a class="xbrlviewer" onClick="javascript:highlight(\ this);" href="javascript:loadReport(3);">Consolidated Balance Sheets (Parenthetical)</a></li> <li class="accordion" id="r4" ><a class="xbrlviewer" onClick="javascript:highlight(\ this);" href="javascript:loadReport(4);">Consolidated Statements of Income</a></li> <li class="accordion" id="r5" ><a class="xbrlviewer" onClick="javascript:highlight(\ this);" href="javascript:loadReport(5);">Consolidated Statements of Comprehensive Income</a></l\ i> <li class="accordion" id="r6" ><a class="xbrlviewer" onClick="javascript:highlight(\ this);" href="javascript:loadReport(6);">Consolidated Statements of Stockholders' Equity</a></l\ i> <li class="accordion" id="r7" ><a class="xbrlviewer" onClick="javascript:highlight(\ this);" href="javascript:loadReport(7);">Consolidated Statements of Cash Flows</a></li> </ul> </li>3. what I have tried both after click1 and click2
data = browser.execute_script("javascript:loadReport(2)" ) result None data = browser.execute_script("loadReport(2)" )returns:
<selenium.webdriver.firefox.webelement.FirefoxWebElement (session="70cabf50-eee7-486f-829c-1d4e9b0e0dfe", element="b3a93cc8-6b8f-431c-a117-3fcb97456631")>
data= browser.find_element_by_link_text("Consolidated Balance Sheets")returns None
I guess I should have played a little more before asking for help
after the second click
data=browser.page_source does the trick