Matrix
Everything that Has a Beginning Has an End
Browse: Home / 2020 / 1月 / use tqdm and requests download file

use tqdm and requests download file

By matrix on 2020/01/19

如果需要在console中显示progress bar,tqdm是一个好选择:

def download_file(url,refer,local_filename):
    headers = {
        'User-Agent':USER_AGENT,
        'referer': refer,
    }
    # NOTE the stream=True parameter below
    with requests.get(url, headers=headers,stream=True) as r:
        total_size = int(r.headers.get('content-length', 0))
        block_size = 8192 #8 Kibibyte
        t=tqdm(total=total_size, unit='iB', unit_scale=True)
        r.raise_for_status()
        with open(local_filename, 'wb') as f:
            for chunk in r.iter_content(chunk_size=block_size): 
                if chunk: # filter out keep-alive new chunks
                    t.update(len(chunk))
                    f.write(chunk)
                    # f.flush()
        t.close()
        if total_size != 0 and t.n != total_size:
            logger.error("ERROR, something went wrong")
    return local_filename

github: https://github.com/tqdm/

Posted in Code, Dev | Tagged python

« Previous

Copyright © 2023 Matrix.

Powered by WordPress and Hybrid.