Python Forum
I Want To Download Many Files Of Same File Extension With Either Wget Or Python,
Thread Rating:
  • 1 Vote(s) - 2 Average
  • 1
  • 2
  • 3
  • 4
  • 5
I Want To Download Many Files Of Same File Extension With Either Wget Or Python,
#10
(May-19-2018, 10:55 AM)eddywinch82 Wrote: I used the following written code in Python 2.79, found on a video on Youtube, inserting my info to achieve my aim, but it unsurprisingly didn't work when I ran it timeouts and errors etc, probably due to it's simplicity :-
Yes,i guess there are some lacking in understating this of topic,and maybe Python in general Wink

The wget get all files method,may work or not.
Not work then back to look at site source for an other method.
As i took a look,it's not so difficult to get all .UTU files.
from bs4 import BeautifulSoup
import requests

url = 'http://web.archive.org/web/20070611232047/http://ultimatetraffic.flight1.net:80/utfiles.asp?mode=1&index=0'
url_get = requests.get(url)
soup = BeautifulSoup(url_get.content, 'lxml')
b_tag = soup.find_all('b')
for a in b_tag:
    link = a.find('a')['href']
    #print(link)
    f_name = link.split('id=')[-1]
    with open(f_name, 'wb')as f:
        f.write(requests.get(link).content)
Reply


Messages In This Thread
RE: I Want To Download Many Files Of Same File Extension With Either Wget Or Python, - by snippsat - May-19-2018, 03:21 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Creating a Browser Extension using Python j49857 3 1,035 Feb-13-2024, 10:49 PM
Last Post: j49857
  Login and download an exported csv file within a ribbon/button in a website Alekhya 0 2,670 Feb-26-2021, 04:15 PM
Last Post: Alekhya
  Download some JPG files and make it a single PDF & share it rompdeck 5 5,706 Jul-31-2020, 01:15 AM
Last Post: Larz60+
  Cannot download latest version of a file TheTechRobo 3 2,316 May-20-2020, 08:33 PM
Last Post: TheTechRobo
  Expose chrome extension buttons to Python robertjaxe 2 2,361 May-12-2020, 07:52 PM
Last Post: robertjaxe
  download pdf file from website m_annur2001 1 3,015 Jun-21-2019, 05:03 AM
Last Post: j.crater
  Access my webpage and download files from Python Pedroski55 7 5,663 May-26-2019, 12:08 PM
Last Post: snippsat
  XML file to multiple txt files in PYTHON ayjay516 1 2,204 Jan-31-2019, 10:21 PM
Last Post: Larz60+
  Flask generating a file for download darktitan 0 3,359 Dec-30-2018, 02:02 PM
Last Post: darktitan
  I wan't to Download all .zip Files From A Website (Project AI) eddywinch82 68 38,262 Oct-28-2018, 02:13 PM
Last Post: eddywinch82

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020