Python Forum
Beautifulsoup doesn't scrape page (python 2.7) - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html)
+--- Thread: Beautifulsoup doesn't scrape page (python 2.7) (/thread-28740.html)



Beautifulsoup doesn't scrape page (python 2.7) - Hikki - Aug-01-2020

Hi,

Python newb here so go easy on me, using 2.7 because I am following a video course that involves scraping web pages for data and the instructor is demonstrating with that version, but when he does it with Spyder, the console shows the output data. Mine doesn't.

from os import chdir 
import pandas as pd
import csv
#webscraping functions
from bs4 import BeautifulSoup
import urllib2

chdir("C:\\Users\\Admin\\Documents\\test")

istst = (r'http://financials.morningstar.com/finan/ajax/exportKR2CSV.html?&callback=?&t=XNAS:AAPL&region=usa&culture=en-US&cur=&order=asc')
header = {'User-Agent': 'Mozilla/5.0'}
req = urllib2.Request(istst, headers = header)
page = urllib2.urlopen(req)
soup = BeautifulSoup(page, "lxml")
soup
What I wanna do is save the data to a .csv but right now I'm trying to find out why beautifulsoup is failing.

Thanks.

EDIT: I am on Windows 7 x64.