I wan't to Download all .zip Files From A Website (Project AI)

[eddywinch82]

Hi there,

I downloaded .zip Files, a while back, using a Python Code,which I was very kindly helped with by snippsat, and others on here. I would now like to download all the available Project AI .zip Files, from the www.flightsim.com Website.

I tried to adapt the original code, so it would download all the .zip Files from the www.flightsim.com Website. My Adapted code, won't download the Files unsurprisingly, but no errors either, the code when run does nothing. the Plane .zip Files are not in Plane Categories this time, there are 253 pages, with .zip Files on all 253 Pages about 2500 .zip Files altogether.

The search Id is not the same each time you do a search, the number changes, you simply choose the Category in the File Library, i.e. Project AI Files, and leave the search box blank, if you want to search for all the .zip Files :- Here is my adapted code :-

from bs4 import BeautifulSoup
import requests, zipfile, io, concurrent.futures

def download(number_id):
    a_zip = 'http://www.flightsim.com/vbfs/fslib.php?do=copyright&fid={}'.format(number_id)
    with open('{}.zip'.format(number_id), 'wb') as f:
        f.write(requests.get(a_zip).content)

if __name__ == '__main__':
    file_id = list(range(1,50))
    with concurrent.futures.ProcessPoolExecutor(max_workers=10) as executor:
        for number_id in file_id:
            executor.submit(download, number_id)

def get_zips(zips_page):
    # print(zips_page)
    zips_source = requests.get(zips_page).text
    zip_soup = BeautifulSoup(zips_source, "html.parser")
    for zip_file in zip_soup.select("a[href*=fslib.php?searchid=65822324&page=]"):
        zip_url = link_root + zip_file['href']
        print('downloading', zip_file.text, '...',)
        r = requests.get(zip_url)
        with open(zip_file.text, 'wb') as zipFile:
            zipFile.write(r.content)


def download_links(root, page):        
    url = ''.join([root, page])      
    source_code = requests.get(url)
    plain_text = source_code.text
    soup = BeautifulSoup(plain_text, "html.parser")

    for zips_suffix in soup.select("a[href*=fslib.php?do=copyright&fid=]"):
        # get_zips(root, zips_suffix['href'])
        next_page = ''.join([root, zips_suffix['href']])
        get_zips(next_page)


link_root = 'http://www.flightsim.com/vbfs/fslib.php?'

page = 'do=copyright&fid='
download_links(link_root, page)

Can Someone help me make corrections to my Code ? Or point me in the right direction ?

Any help would be much appreciated

Eddie