Jun-08-2018, 06:40 PM
(This post was last modified: Jun-08-2018, 06:41 PM by DoctorEvil.)
Let me answer my own question. This seems to be the fastest way for scraping.
from lxml import html # open some link and wait data = [] innerHTML = driver.execute_script("return document.body.innerHTML") htmlElem = html.document_fromstring(innerHTML) tables = htmlElem.xpath("//div[@id='match-history-content']/div[contains(@id, 'tab-mhistory-')]/table/tbody") for table in tables: this_table = [] for row in table.xpath(".//tr"): this_row = [] for elm in row.xpath(".//td"): this_row.append(elm.text_content()) this_table.append(this_row) data.append(this_table)