Python Forum
Selenium Parsing (unable to Parse page after loading)
Thread Rating:
  • 3 Vote(s) - 2.67 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Selenium Parsing (unable to Parse page after loading)
#6
your code does work with that site for me. it printed out the html. However you dont need requests or beautifulsoup, if you use selenium. Selenium can make requests and parse html. You can also do it in the background so it doesnt bring up a browser. You should also use wait instead of time sleep. It will be faster.

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

 
browser = webdriver.Firefox()
browser.get("https://torrentz2.eu/")
WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.ID, 'search')))
 
#class selenium.webdriver.support.expected_conditions.title_contains(Torrent)
 
search = browser.find_element_by_id('thesearchbox')
search.send_keys('xxxxx')
search.send_keys(Keys.RETURN) # hit return after you enter search text
WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.CLASS_NAME, 'results')))
     
Recommended Tutorials:
Reply


Messages In This Thread
RE: Selenium Parsing (unable to Parse page after loading) - by metulburr - Jan-11-2019, 03:39 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Unable to convert browser generated xml to parse in BeautifulSoup Nik1811 0 350 Mar-22-2024, 01:37 PM
Last Post: Nik1811
  Click on a button on web page using Selenium Pavel_47 7 4,788 Jan-05-2023, 04:20 AM
Last Post: ellapurnellrt
  Selenium/Helium loads up a blank web page firaki12345 0 2,080 Mar-23-2021, 11:51 AM
Last Post: firaki12345
  Parsing html page and working with checkbox (on a captcha) straannick 17 11,503 Feb-04-2021, 02:54 PM
Last Post: snippsat
  Saving html page and reloading into selenium while developing all xpaths Larz60+ 4 4,249 Feb-04-2021, 07:01 AM
Last Post: jonathanwhite1
  Selenium Page Object Model with Python Cryptus 5 4,056 Aug-19-2020, 06:30 AM
Last Post: mlieqo
  Selenium on Angular page Martinelli 3 5,802 Jul-28-2020, 12:40 PM
Last Post: Martinelli
  Unable to click element in web page Kalpana 0 1,901 Jun-25-2020, 05:20 AM
Last Post: Kalpana
  use Xpath in Python :: libxml2 for a page-to-page skip-setting apollo 2 3,670 Mar-19-2020, 06:13 PM
Last Post: apollo
  Selenium get data from newly accessed page hoff1022 2 2,987 Oct-09-2019, 06:52 PM
Last Post: hoff1022

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020