Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Python Scraping Error
#1
I wanted to get .text of <a>, but it simply doesn't work.

Here is my code:
from bs4 import BeautifulSoup
import requests

generatedLink = "MyLink"

page = requests.get(generatedLink)
contents = page.text
soup = BeautifulSoup(contents, "html.parser")

name = soup.find('a',class_=["yt-simple-endpoint", "style-scope", "ytd-video-renderer"])

print(name)
And it returns "None"
Output:
<a id="video-title" class="yt-simple-endpoint style-scope ytd-video-renderer" aria-label="TURNIR 1 VS 1 U LOLU FINALEE!! od korisnika KaLuu Vrijeme streaminga: prije 3 dana 3 sata i 49 minuta 644 pregleda" href="/watch?v=5N4X4hjkzOw" title="TURNIR 1 VS 1 U LOLU FINALEE!!"> TURNIR 1 VS 1 U LOLU FINALEE!! </a>
Ik that something is rly wrong here, but if you can, help me please. Thank you!
Reply
#2
Look at what soup source return.
If you see <a> tag in browser dos not always mean that you get that in source Requests get.
This is because of JavaScript,what you see in browser inspect is what has been rendered in DOM.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  I am scraping a web page but got an Error Sarmad54 3 1,418 Mar-02-2023, 08:20 PM
Last Post: Sarmad54
  error in code web scraping alexisbrunaux 5 3,730 Aug-19-2020, 02:31 AM
Last Post: alexisbrunaux
  error zomato scraping data syxzetenz 3 3,319 Jun-23-2020, 08:53 PM
Last Post: Gribouillis
  Web scraping error jithin123 0 2,387 Mar-22-2020, 08:13 PM
Last Post: jithin123
  Web Scraping Error : Not getting expected result adminravi 4 2,332 Oct-08-2019, 09:53 AM
Last Post: snippsat
  Scraping data saving to DB error with Cursor cubangt 3 2,739 May-20-2019, 08:30 PM
Last Post: Yoriz
  Error while scraping item price from online store mgtheboss 2 5,192 Jan-12-2018, 06:42 PM
Last Post: snippsat
  Error while scraping links with beautiful soup mgtheboss 4 8,294 Dec-22-2017, 12:41 PM
Last Post: mgtheboss

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020