Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
scrap macrotrends
#1
I was trying to collect some information about stocks I like.
example for stock : https://www.macrotrends.net/stocks/chart...ent?freq=A

I made a script to scrap the table contents in the website.

myURL = 'https://www.macrotrends.net/stocks/charts/MSFT/microsoft/income-statement?freq=A'
    page = requests.get(myURL)
    soup = bs4.BeautifulSoup(page.content, 'html.parser')
    stockData = soup.find_all(class_="jqx-grid-cell jqx-item jqx-grid-cell-selected jqx-fill-state-pressed")
    print(stockData)
The stockData is always empty.
I printed the soup variable and noticed that no values exist in the table.

I tried to check if the website load a json file that has the table content but couldn't find any.
I only found a json files that has the stock names and ticker which is not useful to me.

it seems the website is tricky.

I appreciate any help.
Reply


Messages In This Thread
scrap macrotrends - by mr_byte31 - Jul-31-2019, 09:19 AM
RE: scrap macrotrends - by ichabod801 - Jul-31-2019, 01:03 PM
RE: scrap macrotrends - by mr_byte31 - Jul-31-2019, 04:54 PM
RE: scrap macrotrends - by snippsat - Jul-31-2019, 01:57 PM
RE: scrap macrotrends - by ichabod801 - Jul-31-2019, 05:55 PM
RE: scrap macrotrends - by snippsat - Jul-31-2019, 08:34 PM
RE: scrap macrotrends - by mr_byte31 - Aug-01-2019, 10:13 AM
RE: scrap macrotrends - by mr_byte31 - Aug-02-2019, 12:02 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Web scrap --Need help Lizardpython 4 1,010 Oct-01-2023, 11:37 AM
Last Post: Lizardpython
  I tried every way to scrap morningstar financials data without success so far sparkt 2 8,233 Oct-20-2020, 05:43 PM
Last Post: sparkt
  Web scrap multiple pages anilacem_302 3 3,824 Jul-01-2020, 07:50 PM
Last Post: mlieqo
  Need logic on how to scrap 100K URLs goodmind 2 2,609 Jun-29-2020, 09:53 AM
Last Post: goodmind
  Scrap a dynamic span hefaz 0 2,688 Mar-07-2020, 02:56 PM
Last Post: hefaz
  scrap by defining 3 functions zarize 0 1,851 Feb-18-2020, 03:55 PM
Last Post: zarize
  Skipping anti-scrap zarize 0 1,876 Jan-17-2020, 11:51 AM
Last Post: zarize
  Cannot get selenium to scrap past the first two pages newbie_programmer 0 4,159 Dec-12-2019, 06:19 AM
Last Post: newbie_programmer
  Scrap data from not standarized page? zarize 4 3,298 Nov-25-2019, 10:25 AM
Last Post: zarize
  page impossible to scrap? :O zarize 2 3,929 Oct-03-2019, 02:44 PM
Last Post: zarize

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020