django-storages(or boto)和S3bucket不断重复headbucket操作并挂起6-7秒

问题描述

我使用 DRF 进行 API 开发。 我已经将这个包用于其他 2 个项目并且没有问题。但是这次我在 s3 存储桶 head_object 操作过程中遇到了麻烦。 DEBUG 消息如下,我不知道出了什么问题。

...
django      | DEBUG 2021-02-09 22:06:43,065 factory 15 140241970775872 Loading s3:Object
django      | DEBUG 2021-02-09 22:06:43,068 action 15 140241970775872 Calling s3:head_object with {'Bucket': 'staging-2021020303212639920000000b','Key': 'media/images/users/profiles/profile_rVvDZa9Mk28f58ch2AXuNt.jpeg'}
django      | DEBUG 2021-02-09 22:06:43,084 retryhandler 15 140241970775872 No retry needed.
django      | DEBUG 2021-02-09 22:06:43,084 action 15 140241970775872 Response: {'ResponseMetadata': {'RequestId': '23A9B0689BB283FF','HostId': '1uaejTch2UdsR2NKw7uilynovbr0QPnefeg4HwMFmJdhBf1SK2SdwApkmEy1BteoVGR4I56ige0=','HTTPStatusCode': 200,'HTTPHeaders': {'x-amz-id-2': '1uaejTch2UdsR2NKw7uilynovbr0QPnefeg4HwMFmJdhBf1SK2SdwApkmEy1BteoVGR4I56ige0=','x-amz-request-id': '23A9B0689BB283FF','date': 'Tue,09 Feb 2021 13:06:44 GMT','last-modified': 'Tue,09 Feb 2021 12:28:36 GMT','etag': '"12aa400f79e5172a4ca388bce267e895"','x-amz-version-id': 'axFjXeb12wKkL8M.18OsAiyRx_1MneIn','accept-ranges': 'bytes','content-type': 'image/jpeg','content-length': '174111','server': 'AmazonS3'},'RetryAttempts': 0},'AcceptRanges': 'bytes','LastModified': datetime.datetime(2021,2,9,12,28,36,tzinfo=tzutc()),'ContentLength': 174111,'ETag': '"12aa400f79e5172a4ca388bce267e895"','VersionId': 'axFjXeb12wKkL8M.18OsAiyRx_1MneIn','ContentType': 'image/jpeg','Metadata': {}}
django      | DEBUG 2021-02-09 22:06:43,086 utils 15 140241970775872 Acquiring 0
django      | DEBUG 2021-02-09 22:06:43,087 tasks 15 140241836529408 DownloadSubmissionTask(transfer_id=0,{'transfer_future': <s3transfer.futures.TransferFuture object at 0x7f8c9341a490>}) about to wait for the following futures []
django      | DEBUG 2021-02-09 22:06:43,{'transfer_future': <s3transfer.futures.TransferFuture object at 0x7f8c9341a490>}) done waiting for dependent futures
django      | DEBUG 2021-02-09 22:06:43,087 tasks 15 140241836529408 Executing task DownloadSubmissionTask(transfer_id=0,{'transfer_future': <s3transfer.futures.TransferFuture object at 0x7f8c9341a490>}) with kwargs {'client': <botocore.client.S3 object at 0x7f8c98ed4550>,'config': <boto3.s3.transfer.TransferConfig object at 0x7f8c93404e20>,'osutil': <s3transfer.utils.OSUtils object at 0x7f8c934be100>,'request_executor': <s3transfer.futures.BoundedExecutor object at 0x7f8c93405dc0>,'transfer_future': <s3transfer.futures.TransferFuture object at 0x7f8c9341a490>,'io_executor': <s3transfer.futures.BoundedExecutor object at 0x7f8c93466490>}
django      | DEBUG 2021-02-09 22:06:43,141 retryhandler 15 140241836529408 No retry needed.
django      | DEBUG 2021-02-09 22:06:43,141 futures 15 140241836529408 Submitting task ImmediatelyWriteIOGetobjectTask(transfer_id=0,{'bucket': 'staging-2021020303212639920000000b','key': 'media/images/users/profiles/profile_rVvDZa9Mk28f58ch2AXuNt.jpeg','extra_args': {}}) to executor <s3transfer.futures.BoundedExecutor object at 0x7f8c93405dc0> for transfer request: 0.
django      | DEBUG 2021-02-09 22:06:43,141 utils 15 140241836529408 Acquiring 0
django      | DEBUG 2021-02-09 22:06:43,143 tasks 15 140241755174656 ImmediatelyWriteIOGetobjectTask(transfer_id=0,'extra_args': {}}) about to wait for the following futures []
django      | DEBUG 2021-02-09 22:06:43,'extra_args': {}}) done waiting for dependent futures
django      | DEBUG 2021-02-09 22:06:43,143 utils 15 140241836529408 Releasing acquire 0/None
django      | DEBUG 2021-02-09 22:06:43,144 tasks 15 140241755174656 Executing task ImmediatelyWriteIOGetobjectTask(transfer_id=0,'extra_args': {}}) with kwargs {'client': <botocore.client.S3 object at 0x7f8c98ed4550>,'bucket': 'staging-2021020303212639920000000b','fileobj': <tempfile.SpooledTemporaryFile object at 0x7f8c93404d30>,'extra_args': {},'callbacks': [],'max_attempts': 5,'download_output_manager': <s3transfer.download.DownloadSeekableOutputManager object at 0x7f8c934a5c40>,'io_chunksize': 262144,'bandwidth_limiter': None}
django      | DEBUG 2021-02-09 22:06:43,166 retryhandler 15 140241755174656 No retry needed.
django      | DEBUG 2021-02-09 22:06:43,200 tasks 15 140241755174656 IOWriteTask(transfer_id=0,{'offset': 0}) about to wait for the following futures []
django      | DEBUG 2021-02-09 22:06:43,{'offset': 0}) done waiting for dependent futures
django      | DEBUG 2021-02-09 22:06:43,200 tasks 15 140241755174656 Executing task IOWriteTask(transfer_id=0,{'offset': 0}) with kwargs {'fileobj': <tempfile.SpooledTemporaryFile object at 0x7f8c93404d30>,'offset': 0}
django      | DEBUG 2021-02-09 22:06:43,201 tasks 15 140241755174656 CompleteDownloadNOOPTask(transfer_id=0,{}) about to wait for the following futures []
django      | DEBUG 2021-02-09 22:06:43,{}) done waiting for dependent futures
django      | DEBUG 2021-02-09 22:06:43,201 tasks 15 140241755174656 Executing task CompleteDownloadNOOPTask(transfer_id=0,{}) with kwargs {}
django      | DEBUG 2021-02-09 22:06:43,201 utils 15 140241755174656 Releasing acquire 0/None
django      | DEBUG 2021-02-09 22:06:43,203 PngImagePlugin 15 140241970775872 STREAM b'IHDR' 16 13
django      | DEBUG 2021-02-09 22:06:43,203 PngImagePlugin 15 140241970775872 STREAM b'IDAT' 41 8192
...

此操作至少重复 20 多次,但它按预期返回 json 响应。任何想法或帮助将不胜感激。 谢谢。

解决方法

暂无找到可以解决该程序问题的有效方法,小编努力寻找整理中!

如果你已经找到好的解决方法,欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@)