Python Forum
Selenium Parsing (unable to Parse page after loading)
Thread Rating:
  • 3 Vote(s) - 2.67 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Selenium Parsing (unable to Parse page after loading)
#8
(Jan-07-2019, 03:08 AM)oneclick Wrote: Im trying to scrape a torrentz url, I get the Page loading html instead of search result html, tried to put sleep time didnt worked out.

anyone knows how to do
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup
import requests

browser = webdriver.Firefox()
browser.get("https://torrentz.eu/")
time.sleep(10)

#class selenium.webdriver.support.expected_conditions.title_contains(Torrent)

search = browser.find_element_by_id('thesearchbox')
search.send_keys('xxxxx')
search.send_keys(Keys.RETURN) # hit return after you enter search text
time.sleep(10)
    
tempurl = browser.current_url
print(tempurl)
tempcont = requests.get(tempurl, timeout=10)
soup = BeautifulSoup(tempcont.content, "html.parser")

print(soup.prettify())

Amazing one sir I have used it before but seems like this website is not working any more could you please create for this one? torrentzeu.org

Thank You.
Reply


Messages In This Thread
RE: Selenium Parsing (unable to Parse page after loading) - by tomalex - Oct-30-2020, 08:13 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Unable to convert browser generated xml to parse in BeautifulSoup Nik1811 0 288 Mar-22-2024, 01:37 PM
Last Post: Nik1811
  Click on a button on web page using Selenium Pavel_47 7 4,728 Jan-05-2023, 04:20 AM
Last Post: ellapurnellrt
  Selenium/Helium loads up a blank web page firaki12345 0 2,062 Mar-23-2021, 11:51 AM
Last Post: firaki12345
  Parsing html page and working with checkbox (on a captcha) straannick 17 11,375 Feb-04-2021, 02:54 PM
Last Post: snippsat
  Saving html page and reloading into selenium while developing all xpaths Larz60+ 4 4,210 Feb-04-2021, 07:01 AM
Last Post: jonathanwhite1
  Selenium Page Object Model with Python Cryptus 5 3,993 Aug-19-2020, 06:30 AM
Last Post: mlieqo
  Selenium on Angular page Martinelli 3 5,748 Jul-28-2020, 12:40 PM
Last Post: Martinelli
  Unable to click element in web page Kalpana 0 1,877 Jun-25-2020, 05:20 AM
Last Post: Kalpana
  use Xpath in Python :: libxml2 for a page-to-page skip-setting apollo 2 3,650 Mar-19-2020, 06:13 PM
Last Post: apollo
  Selenium get data from newly accessed page hoff1022 2 2,959 Oct-09-2019, 06:52 PM
Last Post: hoff1022

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020