Python Forum
Project: Opening all links within a web page
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Project: Opening all links within a web page
#1
I'm trying to write a code that opens all links that are on a web page in one tab for each.
import webbrowser
import bs4
import requests

def my_funct(search_url):

    while True:
	    print(f'Downloading page {url}...')
	    res = requests.get(search_url)
	    res.status_code
	    soup = bs4.BeautifulSoup(res.text, "html.parser")
	    urls = soup.select('#href')
	    if urls == []:
		    print('Page doesn\'t have links!')
		    break
	    else:
		    res = requests.get(urls)
		    numLin = len(res)
		    for i in range(numLin):
		    	webbrowser.open(res, new=2)
	
def test_it(args):
	my_funct(args)
	
if __name__ == '__main__':
	url = input("Please add full url address of desired web site: ")
	test_it(url)
Whatever website that I add within url it prints a message that page doesn't have links. I'm thinking that there is something wrong with the selector that I chose.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  use Xpath in Python :: libxml2 for a page-to-page skip-setting apollo 2 3,578 Mar-19-2020, 06:13 PM
Last Post: apollo
  webscrapping links and then enter those links to scrape data kirito85 2 3,143 Jun-13-2019, 02:23 AM
Last Post: kirito85
  Web Page not opening while web scraping through python selenium sumandas89 4 9,994 Nov-19-2018, 02:47 PM
Last Post: snippsat
  Flask - Opening second page via href is failing - This site can’t be reached rafiPython1 2 5,433 Apr-11-2018, 08:41 AM
Last Post: rafiPython1

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020