Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Dynamically changing Xpath
#4
(Jun-17-2021, 02:33 PM)AgileAVS Wrote: The code does only get data upto 7 columns or upto the year 2015 but I need the data from 2020 to 2005. Could you suggest on how do I extract more data covering the entire table? Anyways thank you so very much for the help.
The slider most move to activate more data,can show how to move and will probably need a slider loop to get all data.
This is of course a difficult and not so good way to get the data.
Many of theses stock sites has an API(search) then can get data in more normal format as JSON.
A good example of this is ALPHA VANTAGE.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.action_chains import ActionChains
import time
from io import StringIO
import pandas as pd
pd.set_option('expand_frame_repr', False)

#--| Setup
options = Options()
#options.add_argument("--headless")
#options.add_argument("--window-size=1980,1020")
browser = webdriver.Chrome(executable_path=r'C:\cmder\bin\chromedriver.exe', options=options)
#--| Parse or automation
url = "https://www.macrotrends.net/stocks/charts/BAC/bank-of-america/financial-ratios"
browser.get(url)
time.sleep(3)
# Find slider and move it
slider = browser.find_element_by_id('jqxScrollThumbhorizontalScrollBarjqxgrid')
ActionChains(browser).click_and_hold(slider).move_by_offset(350 , 10).release().perform()
time.sleep(3)
# Make a loop for 10 rows
lst = []
for number in range(1,11):
    row = browser.find_elements_by_css_selector(f'#row{number}jqxgrid')
    lst.append(row[0].text.replace('\n', ','))

# Make DataFrame
df = pd.read_csv(StringIO('\n'.join(lst)), sep=",")
print(df)
# Write to excel
#df.to_excel("output.xlsx", index=False)
Output:
Long-term Debt / Capital 0.4598 0.4489 0.4803 0.4997 0.5176 0.5377 0.618 0.6627 0.6545 0.6024 0.5736 0 Debt/Equity Ratio 1.9395 1.781 1.9757 2.2581 2.4808 2.8409 2.9697 3.6177 3.5804 3.8671 4.6823 1 Gross Margin - - - - - - - - - - - 2 Operating Margin - - - - - - - - - - - 3 EBIT Margin - - - - - - - - - - - 4 EBITDA Margin - - - - - - - - - - - 5 Pre-Tax Profit Margin 34.8895 31.2356 27.8019 9.523 18.9399 4.087 -0.2873 -1.6177 6.1345 9.6351 35.7993 6 Net Profit Margin 19.8471 20.1488 18.078 5.3529 11.8076 3.6719 0.1062 -4.4006 -3.101 5.4116 25.1369 7 Asset Turnover 0.0367 0.0366 0.0372 0.0397 0.0406 0.034 0.0376 0.0361 0.0319 0.0253 0.0341 8 Inventory Turnover Ratio - - - - - - - - - - -
Reply


Messages In This Thread
Dynamically changing Xpath - by AgileAVS - Jun-17-2021, 10:03 AM
RE: Dynamically changing Xpath - by snippsat - Jun-17-2021, 12:57 PM
RE: Dynamically changing Xpath - by AgileAVS - Jun-17-2021, 02:33 PM
RE: Dynamically changing Xpath - by snippsat - Jun-17-2021, 04:27 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  need help with xpath pythonprogrammer 1 2,828 Jan-18-2020, 11:28 PM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020