Python Forum

(Dec-09-2019, 05:29 PM)Evil_Patrick Wrote: [ -> ]tqdm tutorials for CLI Progressbar

So can write a little about tqdm,with some useful scenarios to use it in.
Example on site it's easy to see that can plug it into any exciting loop you have.

# start_xkcd.py
import requests
from bs4 import BeautifulSoup
import os
from tqdm import tqdm

def xkcd(start, stop):
    for numb in range(start, stop):
        url = 'http://xkcd.com/{}/'.format(numb)
        url_get = requests.get(url)
        soup = BeautifulSoup(url_get.content, 'lxml')
        link = soup.find('div', id='comic').find('img').get('src')
        link = link.replace('//', 'http://')
        img_name = os.path.basename(link)
        try:
            img = requests.get(link)
            with open(img_name, 'wb') as f_out:
                f_out.write(img.content)
        except:
            # Just want images don't care about errors
            pass

if __name__ == '__main__':
    start_img = 1
    stop_img = 30
    xkcd(start_img, stop_img)

Now is downloading 30 images,but don't see any progress,just that it finish.
Plug in tqdm is only this change:

for numb in tqdm(range(start, stop)):

---

As this is for command can also make so argument is given from command line and not in script.
Here i use Click.
Click has a simple progressbar build in,but it can be eaiser to just use tqdm.

import requests
from bs4 import BeautifulSoup
import os
import click
from tqdm import tqdm

@click.command()
@click.argument('start', type=int)
@click.argument('stop', type=int)
def xkcd(start, stop):
    ''' xkcd_arg.py start_image stop_image'''
    for numb in tqdm(range(start, stop)):
        url = 'http://xkcd.com/{}/'.format(numb)
        url_get = requests.get(url)
        soup = BeautifulSoup(url_get.content, 'lxml')
        link = soup.find('div', id='comic').find('img').get('src')
        link = link.replace('//', 'http://')
        img_name = os.path.basename(link)
        try:
            img = requests.get(link)
            with open(img_name, 'wb') as f_out:
                f_out.write(img.content)
        except:
            # Just want images don't care about errors
            pass

if __name__ == '__main__':
    xkcd()

usage:

---

Last look at solution for a single big file for web.
Here need to get info about size of file before can make a loop eg content-length.
When have this info can use tq_size.update in the loop.

import requests
from tqdm import tqdm

url = "https://file-examples.com/wp-content/uploads/2017/04/file_example_MP4_1920_18MG.mp4"
r = requests.get(url, stream=True)
total_size = int(r.headers.get('content-length', 0))
block_size = 1024
tq_size = tqdm(total=total_size, unit='iB', unit_scale=True)
with open('foo.mp4', 'wb') as f:
    for chunk in r.iter_content(block_size):
        tq_size.update(len(chunk))
        f.write(chunk)
tq_size.close()