Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 Project: Opening all links within a web page
#1
I'm trying to write a code that opens all links that are on a web page in one tab for each.
import webbrowser
import bs4
import requests

def my_funct(search_url):

    while True:
	    print(f'Downloading page {url}...')
	    res = requests.get(search_url)
	    res.status_code
	    soup = bs4.BeautifulSoup(res.text, "html.parser")
	    urls = soup.select('#href')
	    if urls == []:
		    print('Page doesn\'t have links!')
		    break
	    else:
		    res = requests.get(urls)
		    numLin = len(res)
		    for i in range(numLin):
		    	webbrowser.open(res, new=2)
	
def test_it(args):
	my_funct(args)
	
if __name__ == '__main__':
	url = input("Please add full url address of desired web site: ")
	test_it(url)
Whatever website that I add within url it prints a message that page doesn't have links. I'm thinking that there is something wrong with the selector that I chose.
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  Unable to switch out of nested frames into main page abi17124 0 42 Jul-17-2019, 06:06 PM
Last Post: abi17124
  Django Two blocks of dynamic content on one page iFunKtion 5 449 Jul-04-2019, 02:31 AM
Last Post: noisefloor
  Can't get method to scroll down page. caarsonr 5 280 Jun-20-2019, 09:14 PM
Last Post: caarsonr
  webscrapping links from pandas dataframe Wolverin 1 130 Jun-19-2019, 11:22 PM
Last Post: Larz60+
  webscrapping links and then enter those links to scrape data kirito85 2 175 Jun-13-2019, 02:23 AM
Last Post: kirito85
  page navigation & form filling rudolphyaber 0 205 Mar-13-2019, 06:31 PM
Last Post: rudolphyaber
  Sorting getting off, when I switch page Django 1.11 m0ntecr1st0 0 206 Feb-12-2019, 06:40 PM
Last Post: m0ntecr1st0
  Selenium Parsing (unable to Parse page after loading) oneclick 6 595 Jan-13-2019, 03:10 AM
Last Post: oneclick
  Difficult web page -- Selenium Larz60+ 2 353 Dec-31-2018, 06:51 PM
Last Post: Larz60+
  Copy & Paste Web Page ian 2 470 Dec-24-2018, 04:56 PM
Last Post: ian

Forum Jump:


Users browsing this thread: 1 Guest(s)