Jun-17-2021, 04:27 PM
(Jun-17-2021, 02:33 PM)AgileAVS Wrote: The code does only get data upto 7 columns or upto the year 2015 but I need the data from 2020 to 2005. Could you suggest on how do I extract more data covering the entire table? Anyways thank you so very much for the help.The slider most move to activate more data,can show how to move and will probably need a slider loop to get all data.
This is of course a difficult and not so good way to get the data.
Many of theses stock sites has an API(search) then can get data in more normal format as JSON.
A good example of this is ALPHA VANTAGE.
from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.common.action_chains import ActionChains import time from io import StringIO import pandas as pd pd.set_option('expand_frame_repr', False) #--| Setup options = Options() #options.add_argument("--headless") #options.add_argument("--window-size=1980,1020") browser = webdriver.Chrome(executable_path=r'C:\cmder\bin\chromedriver.exe', options=options) #--| Parse or automation url = "https://www.macrotrends.net/stocks/charts/BAC/bank-of-america/financial-ratios" browser.get(url) time.sleep(3) # Find slider and move it slider = browser.find_element_by_id('jqxScrollThumbhorizontalScrollBarjqxgrid') ActionChains(browser).click_and_hold(slider).move_by_offset(350 , 10).release().perform() time.sleep(3) # Make a loop for 10 rows lst = [] for number in range(1,11): row = browser.find_elements_by_css_selector(f'#row{number}jqxgrid') lst.append(row[0].text.replace('\n', ',')) # Make DataFrame df = pd.read_csv(StringIO('\n'.join(lst)), sep=",") print(df) # Write to excel #df.to_excel("output.xlsx", index=False)
Output: Long-term Debt / Capital 0.4598 0.4489 0.4803 0.4997 0.5176 0.5377 0.618 0.6627 0.6545 0.6024 0.5736
0 Debt/Equity Ratio 1.9395 1.781 1.9757 2.2581 2.4808 2.8409 2.9697 3.6177 3.5804 3.8671 4.6823
1 Gross Margin - - - - - - - - - - -
2 Operating Margin - - - - - - - - - - -
3 EBIT Margin - - - - - - - - - - -
4 EBITDA Margin - - - - - - - - - - -
5 Pre-Tax Profit Margin 34.8895 31.2356 27.8019 9.523 18.9399 4.087 -0.2873 -1.6177 6.1345 9.6351 35.7993
6 Net Profit Margin 19.8471 20.1488 18.078 5.3529 11.8076 3.6719 0.1062 -4.4006 -3.101 5.4116 25.1369
7 Asset Turnover 0.0367 0.0366 0.0372 0.0397 0.0406 0.034 0.0376 0.0361 0.0319 0.0253 0.0341
8 Inventory Turnover Ratio - - - - - - - - - - -