Jun-17-2021, 02:33 PM
(Jun-17-2021, 12:57 PM)snippsat Wrote: Your code what can i say could be a lot better.
There are many loops,but a many are unnecessary and code still repeating to much in the loops🥨
When code start to get over 200 lines or a lot less,it's about time to think some structure like eg function.
Can giver you some tips and using Pandas is for me an better an easier option than using xlsxwriter.
In Pandas you get similar look that Excel when finish then is just to usedf.to_excel()
.
from selenium import webdriver from selenium.webdriver.chrome.options import Options import time from io import StringIO import pandas as pd #--| Setup options = Options() options.add_argument("--headless") browser = webdriver.Chrome(executable_path=r'C:\cmder\bin\chromedriver.exe', options=options) #--| Parse or automation url = "https://www.macrotrends.net/stocks/charts/BAC/bank-of-america/financial-ratios" browser.get(url) time.sleep(3) # Make a loop for 10 rows lst = [] for number in range(1,11): row = browser.find_elements_by_css_selector(f'#row{number}jqxgrid') lst.append(row[0].text.replace('\n', ',')) # Make DataFrame df = pd.read_csv(StringIO('\n'.join(lst)), sep=",") print(df) # Write to excel df.to_excel("output.xlsx", index=False)
Output:Long-term Debt / Capital 0.4907 0.4763 ... 0.4598 0.4489 0.4803 0 Debt/Equity Ratio 1.9196 1.9389 ... 1.9395 1.781 1.9757 1 Gross Margin - - ... - - - 2 Operating Margin - - ... - - - 3 EBIT Margin - - ... - - - 4 EBITDA Margin - - ... - - - 5 Pre-Tax Profit Margin 25.597 37.3674 ... 34.8895 31.2356 27.8019 6 Net Profit Margin 22.1984 29.6598 ... 19.8471 20.1488 18.078 7 Asset Turnover 0.0263 0.036 ... 0.0367 0.0366 0.0372 8 Inventory Turnover Ratio - - ... - - -
Thank you so much for the suggestion. I am still a newbie to Python, so definitely will lean more about functions and Panda now.
However, the main problem still remains for the code you wrote as well. The code does only get data upto 7 columns or upto the year 2015 but I need the data from 2020 to 2005. Could you suggest on how do I extract more data covering the entire table? Anyways thank you so very much for the help.