I wan't to Download all .zip Files From A Website (Project AI)

[eddywinch82]

Hi Larz60+,

Could you look at this code again for me ? When you have some spare time ?
I am keen to download the .zip Files, I want from the Website in question.

Eddie

[Larz60+]

Could you look at this code again for me

I'll bet that 'this code' has changed since the last post.
re-post the complete code again, and don't touch it until you get a reply.
It's impossible to work on a moving target.

[eddywinch82]

Hi Larz60+,

Here is the current Python Code I have, it hasn't been updated, since you last messaged me in early
September :-

import sys
import getpass
import hashlib
import requests
import os
from bs4 import BeautifulSoup
 
 
BASE_URL = 'https://www.flightsim.com/'
 
LOGIN_PAGE = 'https://www.flightsim.com/vbfs/login.php?do=login'
 
 
def do_login(credentials):
    session = requests.Session()
    session.get(BASE_URL)
    req = session.post(LOGIN_PAGE, params={'do': 'login'}, data=credentials)
    if req.status_code != 200:
        print('Login not successful')
        sys.exit(1)
    # session is now logged in
    return session
 
 
def get_credentials():
    username = input('Username: ')
    password = getpass.getpass()
    password_md5 = hashlib.md5(password.encode()).hexdigest()
    return {
        'cookieuser': 1,
        'do': 'login',
        's': '',
        'securitytoken': 'guest',
        'vb_login_md5_password': password_md5,
        'vb_login_md5_password_utf': password_md5,
        'vb_login_password': '',
        'vb_login_password_hint': 'Password',
        'vb_login_username': username,
    }
 
 
credentials = get_credentials()
session = do_login(credentials)
 
 
class ScrapeUrlList:
    def __init__(self):
        self.fpath = os.fspath
        self.ziplinks = []
 
    def get_url(self, url):
        page = None
        response = requests.get(url)
        if response.status_code == 200:
            page = response.content
        else:
            print(f'Cannot load URL: {url}')
            # pass
        return page
 
    def get_catalog(self):
        base_url = 'https://www.flightsim.com/vbfs'
        with self.fpath.links.open('w') as fp: 'C:\Python37\Lib'
            baseurl = self.fpath.base_catalog_url
            for pageno in range(1, 254):
                url = f'https://www.flightsim.com/vbfs/fslib.php?searchid=65893537&page={pageno}'
                print(f'url: {url}')
                page = self.get_url(self.fpath.base_catalog_url)
            if page:
                soup = BeautifulSoup(page, 'lxml')
                zip_links = soup.find_all('div', class_="fsc_details")
                for link in zip_links:
                    fp.write(f"{link.find('a').text}, {base_url}/{link.find('a').get('href')}")
                input()
            else:
                print(f'No page: {url}')
 
 
def main():
    sul = ScrapeUrlList()
    sul.get_catalog()
 
 
if __name__ == '__main__':
    main()

Regards

Eddie Winch

[Larz60+]

I'll look at this after dinner this evening.
May not respond until my A.M. (EDT)

[eddywinch82]

Many thanks Larz60+, I appreciate you doing that for me, very much.

Best Regards

Eddie Winch

---

---

Hi Larz60+,

Could the .zip Files be saved, as the outer-most folder name in the zip (so the name would be LNI-Lion Mentari Airlines.zip) for example ? Is it fairly easy, to write a simple pipeline, to rename the files in this way ? The Zip file names start pai, i.e. paicm027.zip, so can't be indentified easily, like that.

I am on holiday from tomorrow, and back on Sunday. I can access the Python Forums, on the internet on my phone and type reply's to posts and messages, but I won't have access to my computer therefore, until Sunday evening.

[Larz60+]

Haven't got to this yet. Forgot I have an event this evening, and been spending the day preparing for that.
No promise on resolution, but I will get to it.

[eddywinch82]

Thankyou so much Larz60+, I very much appreciate all your time and effort, on this coding problem. Eddie

[Larz60+]

I looked at this for a bit. Sorry, but I can't continue, too frustrating. I'v already spent too much time on it.
Perhaps another mod will be willing to pick up.

[eddywinch82]

Thanks Larz60+ for all your efforts for me. I understand, your help is much appreciated. Yes can anyone else help me, with this coding problem ?