Python Forum
Beautifulsoup doesn't scrape page (python 2.7)
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Beautifulsoup doesn't scrape page (python 2.7)
#1
Hi,

Python newb here so go easy on me, using 2.7 because I am following a video course that involves scraping web pages for data and the instructor is demonstrating with that version, but when he does it with Spyder, the console shows the output data. Mine doesn't.

from os import chdir 
import pandas as pd
import csv
#webscraping functions
from bs4 import BeautifulSoup
import urllib2

chdir("C:\\Users\\Admin\\Documents\\test")

istst = (r'http://financials.morningstar.com/finan/ajax/exportKR2CSV.html?&callback=?&t=XNAS:AAPL&region=usa&culture=en-US&cur=&order=asc')
header = {'User-Agent': 'Mozilla/5.0'}
req = urllib2.Request(istst, headers = header)
page = urllib2.urlopen(req)
soup = BeautifulSoup(page, "lxml")
soup
What I wanna do is save the data to a .csv but right now I'm trying to find out why beautifulsoup is failing.

Thanks.

EDIT: I am on Windows 7 x64.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  how to scrape page that works dynamicaly? samuelbachorik 0 683 Sep-23-2023, 10:38 AM
Last Post: samuelbachorik
  Strange ModuleNotFound Error on BeautifulSoup for Python 3.11 Gaberson19 1 919 Jul-13-2023, 10:38 AM
Last Post: Gaurav_Kumar
  How can I web scrape the "alt" attribute from a "img" tag with Python? cisky 1 3,763 Aug-19-2022, 04:59 AM
Last Post: snippsat
Lightbulb Python Obstacles | Kung-Fu | Full File HTML Document Scrape and Store it in MariaDB BrandonKastning 5 2,819 Dec-29-2021, 02:26 AM
Last Post: BrandonKastning
  Python Obstacles | American Kenpo | Wiki Scrape URL/Table and Store it in MariaDB BrandonKastning 6 2,783 Dec-29-2021, 12:38 AM
Last Post: BrandonKastning
  Python Obstacles | Karate | HTML/Scrape Specific Tag and Store it in MariaDB BrandonKastning 8 3,091 Nov-22-2021, 01:38 AM
Last Post: BrandonKastning
  Python BeautifulSoup gives unusable text? dggo666 0 1,405 Oct-29-2021, 05:12 AM
Last Post: dggo666
  Python BeautifulSoup IndexError: list index out of range rhat398 1 6,163 May-28-2021, 09:09 PM
Last Post: Daring_T
  Python 3.9 : BeautifulSoup: 'NoneType' object has no attribute 'text' fudgemasterultra 1 8,816 Mar-03-2021, 09:40 AM
Last Post: Larz60+
  to scrape wiki-page: getting back the results - can i use pandas also apollo 2 2,602 Feb-09-2021, 03:57 PM
Last Post: apollo

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020