使用tqdm的不同行为

问题描述

我当时正在为网站制作图像下载项目，但是使用tqdm遇到了一些奇怪的行为。在下面的代码中，我包括了两个用于制作tqdm进度条的选项。在选项一中，我没有将可迭代的内容从响应直接传递到tqdm，而在第二个选项中，我做了。尽管代码看起来相似，但结果却截然不同。

This is what the progress bar's result looks like using Option 1

This is what the progress bar's result looks like using Option 2

选项1是我想要的结果，但我只是找不到使用选项2的行为的解释。有人可以帮我解释这种行为吗？

import requests
from tqdm import tqdm
import os

# Folder to store in
default_path = "D:\\Downloads"


def download_image(url):
    """

    This function will download the given url's image with proper filename labeling
    If a path is not provided the image will be downloaded to the Downloads folder
    """

    # Establish a Session with cookies
    s = requests.Session()

    # Fix for pixiv's request you have to add referer in order to download images
    response = s.get(url,headers={'User-Agent': 'Mozilla/5.0','referer': 'https://www.pixiv.net/'},stream=True)

    file_name = url.split("/")[-1]  # Retrieve the file name of the link
    together = os.path.join(default_path,file_name)  # Join together path with the file_name. Where to store the file
    file_size = int(response.headers["Content-Length"])  # Get the total byte size of the file
    chunk_size = 1024  # Consuming in 1024 byte per chunk

    # Option 1
    progress = tqdm(total=file_size,unit='B',unit_scale=True,desc="Downloading {file}".format(file=file_name))

    # Open the file destination and write in binary mode
    with open(together,"wb") as f:
        # Loop through each of the chunks in response in chunk_size and update the progres by calling update using
        # len(chunk) not chunk_size
        for chunk in response.iter_content(chunk_size):
            f.write(chunk)

            progress.update(len(chunk))

    # Option 2
    """progress = tqdm(response.iter_content(chunk_size),total=file_size,desc="Downloading {file}".format(file = file_name))

    with open(together,"wb") as f:

        for chunk in progress:

            progress.update(len(chunk))
            f.write(chunk)

    # Close the tqdm object and file object as good practice
    """

    progress.close()
    f.close()


if __name__ == "__main__":
    download_image("Image Link")

解决方法

看起来像是tqdm的现有错误。 https://github.com/tqdm/tqdm/issues/766

选项1：

提供tqdm的总大小
在每次迭代中，更新进度。期望进度条继续前进。
工作正常。

选项2：

提供tqdm的总大小以及跟踪进度的 generator 函数。
在每次迭代中，它应自动从生成器获取更新并按下进度条。
但是，您也可以手动调用progress.update，情况并非如此。
相反，让发电机来做。
但这也不起作用，并且已经报告了该问题。

关于选项1的建议 为了避免手动关闭流，您可以将它们包含在with语句中。同样适用于tqdm。

# Open the file destination and write in binary mode
    with tqdm(total=file_size,unit='B',unit_scale=True,desc="Downloading {file}".format(file=file_name)
    ) as progress,open(file_name,"wb") as f:
        # Loop through each of the chunks in response in chunk_size and update the progres by calling update using
        # len(chunk) not chunk_size
        for chunk in response.iter_content(chunk_size):
            progress.update(len(chunk))
            f.write(chunk)

python tqdm