Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Scraping Whole Page Source
#1
Is there a way to input the whole page source in exactly same form as you would be seeing it when you click rmb 'View page source' on a browser, just a raw page with thousands of lines of text?. I've tried requests.get(), but I'm only getting a fraction of it.
Reply
#2
Posting threads when too tired for an adequate research maybe not the best idea.

from selenium import webdriver

browser = webdriver.Chrome()
browser.get('https://whateverpage.com')
x = browser.page_source
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  I am scraping a web page but got an Error Sarmad54 3 1,418 Mar-02-2023, 08:20 PM
Last Post: Sarmad54
  Scraping the page without distorting content oleglpts 5 2,443 Dec-16-2021, 05:08 PM
Last Post: oleglpts
  Scraping a page with log in data (security, proxies) iamaghost 0 2,103 Mar-27-2021, 02:56 PM
Last Post: iamaghost
  Scraping .aspx page Larz60+ 21 50,854 Mar-18-2021, 10:16 AM
Last Post: Larz60+
  scraping from a website that hides source code PIWI_Protein 1 1,938 Mar-27-2020, 05:08 PM
Last Post: Larz60+
  use Xpath in Python :: libxml2 for a page-to-page skip-setting apollo 2 3,580 Mar-19-2020, 06:13 PM
Last Post: apollo
  Scraping next page of LinkedIn jobs RiteshMahto 6 6,287 Dec-09-2019, 09:43 PM
Last Post: Larz60+
  Scraping data from ebay seller page yuvalta 3 5,948 Sep-25-2019, 04:22 AM
Last Post: sandramoraes
  Web Page not opening while web scraping through python selenium sumandas89 4 10,000 Nov-19-2018, 02:47 PM
Last Post: snippsat
  Python requests.get() returns broken source code instead of expected source code? FatalPythonError 3 3,677 Sep-21-2018, 02:46 PM
Last Post: nilamo

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020