Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
beautifulsoup error
#1
I'm using python 3.6 on Windows 7 and I'm following the directions on this page -> https://medium.freecodecamp.org/how-to-s...46935d93fe but I continue getting errors. I'm currently getting this error: AttributeError: 'NoneType' object has no attribute 'text'

Here's my code:

#import libraries
import urllib
import urllib.request
from bs4 import BeautifulSoup

#specify the url
quote_page = 'https://www.bloomberg.com/quote/SPX:IND'

#query the website and return the html to the variable 'page'
page = urllib.request.urlopen(quote_page)

#parse the html using beautiful soup and store in variable 'soup'
soup = BeautifulSoup(page, 'html.parse')

#Take out the <div> of name and get its value
name_box = soup.find('h1', attr={'class': 'name'})

name = name_box.text.strip() # strip() is used to remove starting & trailing
print(name)
Reply
#2
They might be putting up a recaptcha. Do you get this if you just do
print(soup.title)
Output:
<title>Bloomberg - Are you a robot?</title>
I tried using their site with selenium and it brings me straight to a recaptcha for human verification.
Recommended Tutorials:
Reply
#3
(Feb-21-2019, 07:05 PM)metulburr Wrote: They might be putting up a recaptcha. Do you get this if you just do
print(soup.title)
Output:
<title>Bloomberg - Are you a robot?</title>
I tried using their site with selenium and it brings me straight to a recaptcha for human verification.

I changed the code to the same as yours and I got the same thing. Maybe I should just find another tutorial? What I'm trying to accomplish can be found at -> Python Forum -> Web Development -> browser table value counting

Thanks.
Reply
#4
(Feb-21-2019, 07:47 PM)rudolphyaber Wrote: Maybe I should just find another tutorial?
That website looks like it might be harder to parse. Try a different website, especially if you are new at it.
Recommended Tutorials:
Reply
#5
(Feb-21-2019, 07:47 PM)rudolphyaber Wrote: Maybe I should just find another tutorial?
Have a couple of here,that's up to date.
Web-Scraping part-1
Web-Scraping part-2

That Bloomberg site is not easy to parse(have changes since tutorial you look at).
They have API with Python support,but need C++ SDK.
Quote:> python -m pip install --index-url=https://bloomberg.bintray.com/pip/simple blpapi
Prebuilt binaries are provided for Python 2.7, 3.5, 3.6 and 3.7 for Windows, in both 32 and 64 bits.
A source package is also provided for other platforms/Python versions.
A local installation of the C++ API is required both for importing the blpapi module in Python and for building the module from sources, if needed.
For stocks there are better API free one like ALPHA VANTAGE.
Site over is okay for training to use an API to get data,then use Requests(not urllib),to get data in eg JSON.
Reply
#6
Thanks. Wasn't actually needing to get stocks, that was just the website that that tutorial was using to teach with.
Reply
#7
There are some sites that have specific methods to throw off bots. Its usually better to start on a simple one and work your way up. One program i wrote in selenium; half of the code was just to handle their anti-bot measures.
Recommended Tutorials:
Reply
#8
If you need quotes, look here:
https://python-forum.io/Thread-Exchange-Quotes
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Strange ModuleNotFound Error on BeautifulSoup for Python 3.11 Gaberson19 1 921 Jul-13-2023, 10:38 AM
Last Post: Gaurav_Kumar
  [Solved]Help with BeautifulSoup.getText() Error Extra 5 3,643 Jan-19-2023, 02:03 PM
Last Post: prvncpa
  Error with NumPy, BeautifulSoup when using pip tsurubaso 7 5,164 Oct-20-2020, 04:34 PM
Last Post: tsurubaso
  Python beautifulsoup pagination error The61 5 3,402 Apr-09-2020, 09:17 PM
Last Post: Larz60+
  BeautifulSoup: Error while extracting a value from an HTML table kawasso 3 3,156 Aug-25-2019, 01:13 AM
Last Post: kawasso
  BeautifulSoup Parsing Error slinkplink 6 9,459 Feb-12-2018, 02:55 PM
Last Post: seco

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020