Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Dynamically changing Xpath
#3
(Jun-17-2021, 12:57 PM)snippsat Wrote: Your code what can i say could be a lot better.
There are many loops,but a many are unnecessary and code still repeating to much in the loops🥨
When code start to get over 200 lines or a lot less,it's about time to think some structure like eg function.

Can giver you some tips and using Pandas is for me an better an easier option than using xlsxwriter.
In Pandas you get similar look that Excel when finish then is just to use df.to_excel().
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import time
from io import StringIO
import pandas as pd

#--| Setup
options = Options()
options.add_argument("--headless")
browser = webdriver.Chrome(executable_path=r'C:\cmder\bin\chromedriver.exe', options=options)
#--| Parse or automation
url = "https://www.macrotrends.net/stocks/charts/BAC/bank-of-america/financial-ratios"
browser.get(url)
time.sleep(3)

# Make a loop for 10 rows
lst = []
for number in range(1,11):
    row = browser.find_elements_by_css_selector(f'#row{number}jqxgrid')
    lst.append(row[0].text.replace('\n', ','))

# Make DataFrame
df = pd.read_csv(StringIO('\n'.join(lst)), sep=",")
print(df)
# Write to excel
df.to_excel("output.xlsx", index=False)
Output:
Long-term Debt / Capital 0.4907 0.4763 ... 0.4598 0.4489 0.4803 0 Debt/Equity Ratio 1.9196 1.9389 ... 1.9395 1.781 1.9757 1 Gross Margin - - ... - - - 2 Operating Margin - - ... - - - 3 EBIT Margin - - ... - - - 4 EBITDA Margin - - ... - - - 5 Pre-Tax Profit Margin 25.597 37.3674 ... 34.8895 31.2356 27.8019 6 Net Profit Margin 22.1984 29.6598 ... 19.8471 20.1488 18.078 7 Asset Turnover 0.0263 0.036 ... 0.0367 0.0366 0.0372 8 Inventory Turnover Ratio - - ... - - -
[Image: qXbKs3.png]

Thank you so much for the suggestion. I am still a newbie to Python, so definitely will lean more about functions and Panda now.
However, the main problem still remains for the code you wrote as well. The code does only get data upto 7 columns or upto the year 2015 but I need the data from 2020 to 2005. Could you suggest on how do I extract more data covering the entire table? Anyways thank you so very much for the help.
Reply


Messages In This Thread
Dynamically changing Xpath - by AgileAVS - Jun-17-2021, 10:03 AM
RE: Dynamically changing Xpath - by snippsat - Jun-17-2021, 12:57 PM
RE: Dynamically changing Xpath - by AgileAVS - Jun-17-2021, 02:33 PM
RE: Dynamically changing Xpath - by snippsat - Jun-17-2021, 04:27 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  need help with xpath pythonprogrammer 1 2,801 Jan-18-2020, 11:28 PM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020