Dec-13-2016, 02:53 PM
Hi guys,
I am just getting into Python and web scraping, so I am very sorry if this is a basic question. So far, I have been able to scrape information from tables from various website. However, when I want to get access to the prices at this web site nordpoolspot.com/Market-data1/#/nordic/table I run into trouble.
Is it because the page and table are "fancy"? I can't seem to access the data in the usual way -- in particular, I do not seem to have access to the whole page in my "soup" variable. Below is some code which I have tried.
I am just getting into Python and web scraping, so I am very sorry if this is a basic question. So far, I have been able to scrape information from tables from various website. However, when I want to get access to the prices at this web site nordpoolspot.com/Market-data1/#/nordic/table I run into trouble.
Is it because the page and table are "fancy"? I can't seem to access the data in the usual way -- in particular, I do not seem to have access to the whole page in my "soup" variable. Below is some code which I have tried.
import requests, bs4 url = 'www.nordpoolspot.com/Market-data1/#/nordic/table' # <-- include http (I can't post links before I am a proven non-spammer) res = requests.get(url) res.raise_for_status() soup = bs4.BeautifulSoup(res.content,"lxml") tmp = soup.select('tr td') print(len(tmp)) # Output: 0