Python Forum
Traceback error in PyCharm but not in other IDEs?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Traceback error in PyCharm but not in other IDEs?
#1
I'm wanting to switch to PyCharm but I'm getting a Traceback error with the following code that I haven't gotten before in other IDEs. I'm running v3.8.2 on a Mac. Any help would be greatly appreciated!

# import libraries
from urllib.request import urlopen
from bs4 import BeautifulSoup

# specify the url
url = "https://www.bbc.com/sport/football/46897172"

# Connect to the website and return the html to the variable ‘page’
try:
    page = urlopen(url)
except:
    print("Error opening the URL")

# parse the html using beautiful soup and store in variable `soup`
soup = BeautifulSoup(page, 'html.parser')

# Take out the <div> of name and get its value
content = soup.find('div', {"class": "story-body sp-story-body gel-body-copy"})

article = ''
for i in content.findAll('p'):
    article = article + ' ' + i.text
print(article)

# Saving the scraped text
with open('scraped_text.txt', 'w') as file:
    file.write(article)
Error:
Traceback (most recent call last): File "<input>", line 1, in <module> File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile pydev_imports.execfile(filename, global_vars, local_vars) # execute the script File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile exec(compile(contents+"\n", file, 'exec'), glob, loc) File "/Users/dv/Desktop/Test/test.py", line 15, in <module> soup = BeautifulSoup(your_mom, 'html.parser') NameError: name 'page' is not defined
Reply
#2
remove the general except on line 11, so that you can get what exactly the error is. At the moment you hide the actual error with that all-catching except.

Also note that the error does not match the code provided.
on line #15 you have
soup = BeautifulSoup(page, 'html.parser')
while the error refer to
soup = BeautifulSoup(your_mom, 'html.parser')
note page vs your_nom
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#3
Thanks for the reply. I accidentally had two variable names in there because I was swapping them out thinking "page" might be a constant in Python. Anyway, I ran the code with "page" consistent throughout and removed "except:" from line 11 as well:

from urllib.request import urlopen
from bs4 import BeautifulSoup

# specify the url
url = "https://www.bbc.com/sport/football/46897172"

# Connect to the website and return the html to the variable ‘page’
try:
    page = urlopen(url)

    print("Error opening the URL")

# parse the html using beautiful soup and store in variable `soup`
soup = BeautifulSoup(page, 'html.parser')

# Take out the <div> of name and get its value
content = soup.find('div', {"class": "story-body sp-story-body gel-body-copy"})

article = ''
for i in content.findAll('p'):
    article = article + ' ' + i.text
print(article)

# Saving the scraped text
with open('scraped_text.txt', 'w') as file:
    file.write(article)



Here's the new error:

Error:
Traceback (most recent call last): File "<input>", line 1, in <module> File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile pydev_imports.execfile(filename, global_vars, local_vars) # execute the script File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile exec(compile(contents+"\n", file, 'exec'), glob, loc) File "/Users/dv/Desktop/Test/test.py", line 15 soup = BeautifulSoup(page, 'html.parser') ^ SyntaxError: invalid syntax
Reply
#4
you cannot remove just the except. remove also the try and fix the indentation
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#5
Doing that threw a crazy long error – in a nutshell:
Error:
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED]
Thanks to that new Traceback error though, I was able to find a fix on Stack Overflow. The fix was enabled by double clicking on the 'Install Certificates.command' in my Python install folder. Not really sure what the issue was, but after doing that the error was gone.

Thanks again for your help – it's greatly appreciated.
Reply
#6
Some advice drop urllib and use Requests.
Here how that code would look with some advice,more here in tutorial part-1.
import requests
from bs4 import BeautifulSoup

url = "https://www.bbc.com/sport/football/46897172"
response = requests.get(url)
# lxml is faster parser
soup = BeautifulSoup(response.content, 'lxml')
# Now do not use dict call,easier to copy from page and just add one _ after class
content = soup.find('div', class_="story-body sp-story-body gel-body-copy")

with open('scraped_text.txt', 'w') as f_out:
    for p_tag in content.find_all('p'):
        print(p_tag.text)
        f_out.write(f'{p_tag.text}\n')
Reply
#7
Thanks for the Requests recommendation and tutorial -- this is a much better solution!
Reply
#8
@0ba22b85af, you understand that OP problem has been resolved like 1 week ago, right? And posting spam is NOT tolerated - the link in the post has been removed.
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Pycharm error zuri 1 437 Nov-01-2023, 03:38 PM
Last Post: deanhystad
  Python Traceback Error mg24 2 903 Nov-21-2022, 02:35 PM
Last Post: snippsat
  Keeping up with IDEs and Virtual Environments... bytecrunch 7 2,368 Sep-05-2022, 08:04 PM
Last Post: snippsat
  Pandas - error when running Pycharm, but works on cmd line zxcv101 1 1,321 Jun-18-2022, 01:09 PM
Last Post: snippsat
  PyCharm Error? coder_sw99 4 1,946 Sep-24-2021, 06:16 PM
Last Post: Yoriz
  Keep getting Session management error when running imshow in pycharm pace 0 2,061 Mar-25-2021, 10:06 AM
Last Post: pace
  REGEX Install Error in PyCharm charlesauspicks 1 2,725 May-14-2020, 08:10 PM
Last Post: buran
  Nameparser - Traceback error tjnichols 2 2,327 Feb-28-2020, 05:11 PM
Last Post: GodMaster
  Error in importing package in pycharm fullstop 0 2,314 Dec-12-2019, 04:03 PM
Last Post: fullstop
  can't assign to literal error in pycharm ryder5227 2 2,926 Oct-07-2019, 08:57 PM
Last Post: Bmart6969

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020