Python Forum

Full Version: Beautifulsoup doesn't scrape page (python 2.7)
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hi,

Python newb here so go easy on me, using 2.7 because I am following a video course that involves scraping web pages for data and the instructor is demonstrating with that version, but when he does it with Spyder, the console shows the output data. Mine doesn't.

from os import chdir 
import pandas as pd
import csv
#webscraping functions
from bs4 import BeautifulSoup
import urllib2

chdir("C:\\Users\\Admin\\Documents\\test")

istst = (r'http://financials.morningstar.com/finan/ajax/exportKR2CSV.html?&callback=?&t=XNAS:AAPL&region=usa&culture=en-US&cur=&order=asc')
header = {'User-Agent': 'Mozilla/5.0'}
req = urllib2.Request(istst, headers = header)
page = urllib2.urlopen(req)
soup = BeautifulSoup(page, "lxml")
soup
What I wanna do is save the data to a .csv but right now I'm trying to find out why beautifulsoup is failing.

Thanks.

EDIT: I am on Windows 7 x64.