Python Forum
I wan't to Download all .zip Files From A Website (Project AI)
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
I wan't to Download all .zip Files From A Website (Project AI)
#8
The following will extract the page links from the web page in your last post, and print the url's that reference pages
(indexes for remaining pages)

it will also print download links and zipfile names
the actual download links appear to be like: https://www.flightsim.com/vbfs/fslib.php...&fid=64358
import requests
from bs4 import BeautifulSoup


class MyAttempt:
    def __init__(self):
        self.build_catalog()

    def build_catalog(self):
        page1_url = 'https://www.flightsim.com/vbfs/fslib.php?searchid=65842563'
        page = self.get_page(page1_url)
        soup = BeautifulSoup(page, 'lxml')
        for link in soup.findAll('a', href=True):
            url = link['href']
            text = link.text
            if 'page=' in url:
                print(f'page in url: {url}\ntext: {text}\n')
            if 'copyright' in url:
                print(f'actual download link: {url}\ntext: {text}\n')
                

    def get_page(self, url):
        ok_status = 200
        page = None
        response = requests.get(url, allow_redirects=False)
        if response.status_code == ok_status:
            page = response.content
        else:
            print(f'Could not load url: {url}')
        return page


if __name__ == '__main__':
    MyAttempt()
Please note copyright!
Reply


Messages In This Thread
RE: I wan't to Download all .zip Files From A Website (Project AI) - by Larz60+ - Aug-26-2018, 02:31 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Website scrapping and download santoshrane 3 4,509 Apr-14-2021, 07:22 AM
Last Post: kashcode
  Login and download an exported csv file within a ribbon/button in a website Alekhya 0 2,775 Feb-26-2021, 04:15 PM
Last Post: Alekhya
  Cant Download Images from Unsplash Website firaki12345 1 2,391 Feb-08-2021, 04:15 PM
Last Post: buran
  Download some JPG files and make it a single PDF & share it rompdeck 5 5,864 Jul-31-2020, 01:15 AM
Last Post: Larz60+
  download pdf file from website m_annur2001 1 3,099 Jun-21-2019, 05:03 AM
Last Post: j.crater
  Access my webpage and download files from Python Pedroski55 7 5,875 May-26-2019, 12:08 PM
Last Post: snippsat
  Download all secret links from a map design website fyec 0 2,940 Jul-24-2018, 09:08 PM
Last Post: fyec
  I Want To Download Many Files Of Same File Extension With Either Wget Or Python, eddywinch82 15 14,973 May-20-2018, 06:05 PM
Last Post: eddywinch82

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020