Python Forum
Beautiful Soup (suddenly) doesn't get full webpage html
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Beautiful Soup (suddenly) doesn't get full webpage html
#2
Hey bro,

So i would recommend using the code that i used below:-
It returns full request. Its hard for me to know what it is missing compared to an original you ran months ago tho...

The below query works nicely as far as i am aware.

import requests
from bs4 import BeautifulSoup

url = requests.get("https://www.youtube.com/results?search_query=python").content
soup = BeautifulSoup(url, 'lxml') # You can use html.parser here alternatively - Depends on what you are wanting to achieve
print(soup)
Reply


Messages In This Thread
RE: Beautiful Soup (suddenly) doesn't get full webpage html - by HarleyQuin - Jul-11-2020, 09:37 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Selenium suddenly fails to find element Pavel_47 3 6,339 Sep-04-2022, 11:06 AM
Last Post: Pavel_47
Lightbulb Python Obstacles | Kung-Fu | Full File HTML Document Scrape and Store it in MariaDB BrandonKastning 5 2,924 Dec-29-2021, 02:26 AM
Last Post: BrandonKastning
  Beautiful Soup - access a rating value in a class KatMac 1 3,479 Apr-16-2021, 01:27 PM
Last Post: snippsat
  HTML multi select HTML listbox with Flask/Python rfeyer 0 4,654 Mar-14-2021, 12:23 PM
Last Post: rfeyer
  *Beginner* web scraping/Beautiful Soup help 7ken8 2 2,627 Jan-28-2021, 04:26 PM
Last Post: 7ken8
  Help: Beautiful Soup - Parsing HTML table ironfelix717 2 2,703 Oct-01-2020, 02:19 PM
Last Post: snippsat
  Requests-HTML vs Beautiful Soup - How to Choose? robin73 0 3,833 Jun-23-2020, 02:53 PM
Last Post: robin73
  Python3 + BeautifulSoup4 + lxml (HTML -> CSV) - How to loop to next HTML/new CSV Row BrandonKastning 0 2,379 Mar-22-2020, 06:10 AM
Last Post: BrandonKastning
  looking for direction - scrappy, crawler, beautiful soup Sly_Corn 2 2,470 Mar-17-2020, 03:17 PM
Last Post: Sly_Corn
  Beautiful soup truncates results jonesjoz 4 3,897 Mar-09-2020, 06:04 PM
Last Post: jonesjoz

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020