Python Forum
Beautifulsoup doesn't scrape page (python 2.7)
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Beautifulsoup doesn't scrape page (python 2.7)
#1
Hi,

Python newb here so go easy on me, using 2.7 because I am following a video course that involves scraping web pages for data and the instructor is demonstrating with that version, but when he does it with Spyder, the console shows the output data. Mine doesn't.

from os import chdir 
import pandas as pd
import csv
#webscraping functions
from bs4 import BeautifulSoup
import urllib2

chdir("C:\\Users\\Admin\\Documents\\test")

istst = (r'http://financials.morningstar.com/finan/ajax/exportKR2CSV.html?&callback=?&t=XNAS:AAPL&region=usa&culture=en-US&cur=&order=asc')
header = {'User-Agent': 'Mozilla/5.0'}
req = urllib2.Request(istst, headers = header)
page = urllib2.urlopen(req)
soup = BeautifulSoup(page, "lxml")
soup
What I wanna do is save the data to a .csv but right now I'm trying to find out why beautifulsoup is failing.

Thanks.

EDIT: I am on Windows 7 x64.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
Photo How do I scrape a web page? oradba4u 2 165 Dec-23-2020, 12:35 PM
Last Post: codeto
  Python beautifulsoup pagination error The61 5 825 Apr-09-2020, 09:17 PM
Last Post: Larz60+
  use Xpath in Python :: libxml2 for a page-to-page skip-setting apollo 2 850 Mar-19-2020, 06:13 PM
Last Post: apollo
  scrape data 1 go to next page scrape data 2 and so on alkaline3 6 1,061 Mar-13-2020, 07:59 PM
Last Post: alkaline3
  Beautifulsoup don't get me the page mariolopes 5 1,427 Oct-23-2019, 04:44 PM
Last Post: mariolopes
  Scrape ASPX data with python... hoff1022 0 2,171 Feb-26-2019, 06:16 PM
Last Post: hoff1022
  How do i scrape website whose page changes using javsacript _dopostback function and Prince_Bhatia 1 3,211 Aug-06-2018, 09:45 AM
Last Post: wavic
  How to clean html content using BeautifulSoup in Python 3.6? PrateekG 5 5,418 Apr-27-2018, 01:14 PM
Last Post: snippsat
  [Python / BS4] How to Scrape digitalmatic7 9 5,492 Oct-21-2017, 12:55 AM
Last Post: snippsat
  Scrape Facebook page user posts text stockholm 6 5,091 May-08-2017, 12:24 PM
Last Post: Joseph_f2

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020