+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Multi-threading (/thread-28344.html)
Multi-threading - Evil_Patrick - Jul-15-2020
Can I use Multi-threading to scrape images from internet fast
If yes can anyone Explain me how it works
Can't find anything related this on Internet or Youtube
for link in tqdm(images):
filename = link.split("/")[-1]
urllib.request.urlretrieve(link, filename)
RE: Multi-threading - HarleyQuin - Jul-15-2020
Good Morning,
I followed a really good tutorial on Youtube.
https://www.youtube.com/watch?v=IEEhzQoKtQU
RE: Multi-threading - snippsat - Jul-15-2020
(Jul-15-2020, 05:26 AM)Evil_Patrick Wrote: Can I use Multi-threading to scrape images from internet fastThere is one example in Web-scraping part-2
So take time from 1,10 minute no Multi-threading,down to 10-sec for 200-images downloads.
So there use ProcessPoolExecutor loading 20 processes so quite heavy for this task,but it's blazing fast
