Feb-10-2019, 01:55 AM
This script is supposed to download images in multithreading processes.
Also, I would appreciate if anyone could explain me the last chunk of code. Why calling a Thread object’s join() method will block until that thread has finished? The assumption is that there’s some code you don’t want to run in the main thread until all the threads have completed.
#! python3 # multidx.py - Downloads XKCD comics using multiple threads. import requests, os, bs4, threading os.makedirs('xkcd', exist_ok=True) # store comics in ./xkcd def downloadXkcd(startComic, endComic): for urlNumber in range(startComic, endComic + 1): print(f'Downloading page http://xkcd.com/{urlNumber}...') res = requests.get(f'http://xkcd.com/{urlNumber}') res.raise_for_status() soup = bs4.BeautifulSoup(res.text) comicElem = soup.select('#comic img') if comicElem == []: print('Could not find comic image.') else: comicUrl = comicElem[0].get('src') print(f'Downloading image {comicUrl}') res = requests.get(comicUrl) res.raise_for_status() imageFile = open(os.path.join('xkcd', os.path.basename(comicUrl)), 'wb') for chunk in res.iter_content(100000): imageFile.write(chunk) imageFile.close() downloadThreads = [] # a list of all Thread objects for i in range(0, 1400, 100): # loops 14 times, creates 14 threads downloadThread = threading.Thread(target=downloadXkcd, args(i, i + 99)) downloadThreads.append(downloadThread) downloadThread.start() # Wait for all threads to end for downloadThread in downloadThreads: downloadThread.join() print('Done.')but I receive an error
Error:C:\Python36\kodovi>multidx.py
File "C:\Python36\kodovi\multidx.py", line 29
downloadThread = threading.Thread(target=downloadXkcd, args(i, i + 99))
^
SyntaxError: positional argument follows keyword argument
I thought that the keyword argument goes first and then positional.Also, I would appreciate if anyone could explain me the last chunk of code. Why calling a Thread object’s join() method will block until that thread has finished? The assumption is that there’s some code you don’t want to run in the main thread until all the threads have completed.