问题描述
在我的 Django 项目中，我使用 Celery 异步运行我的任务。
我面临的问题是“task_args”中的“django_celery_results_taskresult”列被截断（即文本后跟省略号）。
例如，我期望“process_4_items”中的整个“task_args”值。然而，事实并非如此。我可以在执行中的 Celery 工作线程的命令提示符中看到相同的截断文本（请参见下面的“[CELERY WORKER COMMAND PROMPT FOR ONE IteraTION]”）。
我真的不确定我需要做什么才能在“process_4_items”中获得整个“task_args”值。
我在下面详细说明了我认为我需要复制此项目的所有内容。
最重要的是，当我调试项目并逐行执行时，“process_4_items”值的全部内容都出现在“task_args”中。
[celeryconfig.py]
broker_url = 'amqp://localhost:5672//'
imports = ('operation_hawkeye.tasks',)
result_backend = 'django-db'
accept_content = ['json']
task_always_eager = False
[views.py]
from celery import group
process_4_items = [
{
"service_id": "7f7962b082cb8da55e01f4f013d8fefbc97948ec03400c93423b050d",
"service_id_unique": "c30ed891987f4d8d3debcaef7edb0f582e6b634f1ebed00fb88f0ec7",
"service_type": "buzz_collect_compare",
"service_file_pf_folder": "",
"service_file_nanonsportsproductivity_folder": "",
"service_file_tba_report": "",
"service_file_buzz_collect_compare": "services/buzzcollect_xml_compare/2021/06/01/file_2_name_Update_COOKINGHD_May_2021_Listin_yWoWXo1.xml",
"service_file_type": "XML 1.0 document,ASCII text",
"service_file_name": "file_2_name_Update_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml",
"service_file_size": 396557,
"service_status": "In Queue for Upload",
"service_status_remarks": {
"source_information": {
"source_id": "5191",
"source_name": "10NEWS2",
"file_1_name": "Original_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml",
"file_2_name": "Update_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml"
}
},
"service_creation_date": "2021-06-01T18:18:40.968Z",
"service_owner": "p.premlal@spi-global.com"
},
{
"service_id": "7f7962b082cb8da55e01f4f013d8fefbc97948ec03400c93423b050d",
"service_id_unique": "2edaf2535945992d13b9ee85e908b21cbd520a02642bd7faf7f229bc",
"service_file_buzz_collect_compare": "services/buzzcollect_xml_compare/2021/06/01/file_1_name_Original_COOKINGHD_May_2021_List_P2PlYY7.xml",
"service_file_name": "file_1_name_Original_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml",
"service_file_size": 396409,
"service_creation_date": "2021-06-01T18:18:40.935Z",
"service_owner": "p.premlal@spi-global.com"
}
]
jobs_4 = group(tasks.buzz_collect_compare_main.s(process_4_items))
result_4 = jobs_4.apply_async()
[tasks.py]
@app.task(bind=True)
def buzz_collect_compare_main(self,items):
process_items = items
...
[CELERY WORKER COMMAND PROMPT FOR ONE IteraTION]
[2021-06-01 09:37:29,972: DEBUG/ForkPoolWorker-32] (0.010)
UPDATE
"django_celery_results_taskresult"
SET
"task_id" = 'daca83c1-3297-4e64-8a5e-f4b61aef84e2',
"task_name" = 'operation_hawkeye.tasks.buzz_collect_compare_main',
"task_args" = '([{''service_id'': ''dc30fc45948975044cb00c39a4d5a0db8e28bc34f0b4d52fd4dcddef'',''service_id_unique'': ''2004e35ae5dcc2586ae5fca16ab8a5b2708ffeca233d3bb234feb347'',''service_type'': ''buzz_collect_compare'',''service_file_pf_folder'': '''',''service_file_nanonsportsproductivity_folder'': '''',''service_file_tba_report'': '''',''service_file_buzz_collect_compare'': ''services/buzzcollect_xml_compare/2021/06/01/file_2_name_Update_COOKINGHD_May_2021_Listin_726wZgI.xml'',''service_file_type'': ''XML 1.0 document,ASCII text'',''service_file_name'': ''file_2_name_Update_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml'',''service_file_size'': 396557,''service_status'': ''In Queue for Upload'',''service_status_remarks'': ''{"source_information": "{\"source_id\": \"27227\",\"source_name\": \"COOKINGHD\",\"file_1_name\": \"Original_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml\",\"file_2_name\": \"Update_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml\"}"}'',''service_creation_date'': ''2021-06-01T09:36:56.825Z'',''service_owner'': ''p...'',...],)',
"task_kwargs" = '{}',
"status" = 'PROGRESS',
"worker" = 'celery@AMD-RYZEN-UBUNTU',
"content_type" = 'application/json',
"content_encoding" = 'utf-8',
"result" = '{"pending": false,"current": 1254,"total": 1254,"percent": 100.0,"description": "Building row 1254 of 1254"}',
"date_created" = '2021-06-01T09:36:58.465884+00:00'::timestamptz,
"date_done" = '2021-06-01T09:37:29.962267+00:00'::timestamptz,
"traceback" = NULL,
"Meta" = '{"children": []}'
WHERE
"django_celery_results_taskresult"."id" = 808;
args=('daca83c1-3297-4e64-8a5e-f4b61aef84e2','operation_hawkeye.tasks.buzz_collect_compare_main','([{\'service_id\': \'dc30fc45948975044cb00c39a4d5a0db8e28bc34f0b4d52fd4dcddef\',\'service_id_unique\': >\'2004e35ae5dcc2586ae5fca16ab8a5b2708ffeca233d3bb234feb347\',\'service_type\': \'buzz_collect_compare\',\'service_file_pf_folder\': \'\',\'service_file_nanonsportsproductivity_folder\': \'\',\'service_file_tba_report\': \'\',\'service_file_buzz_collect_compare\': \'services/buzzcollect_xml_compare/2021/06/01/file_2_name_Update_COOKINGHD_May_2021_Listin_726wZgI.xml\',\'service_file_type\': \'XML 1.0 document,ASCII text\',\'service_file_name\': \'file_2_name_Update_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml\',\'service_file_size\': 396557,\'service_status\': \'In Queue for Upload\',\'service_status_remarks\': \'{"source_information": "{\\"source_id\\": \\"27227\\",\\"source_name\\": \\"COOKINGHD\\",\\"file_1_name\\": \\"Original_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml\\",\\"file_2_name\\": \\"Update_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml\\"}"}\',\'service_creation_date\': \'2021-06-01T09:36:56.825Z\',\'service_owner\': \'p...\','{}','PROGRESS','celery@AMD-RYZEN-UBUNTU','application/json','utf-8','{"pending": false,datetime.datetime(2021,6,1,9,36,58,465884,tzinfo=<UTC>),37,29,962267,'{"children": []}',808)
[$ celery -A hawkeye report]
software -> celery:4.4.7 (cliffs) kombu:4.6.11 py:3.8.6
billiard:3.6.3.0 py-amqp:2.6.1
platform -> system:Linux arch:64bit,ELF
kernel version:5.8.0-53-generic imp:cpython
loader -> celery.loaders.app.AppLoader
settings -> transport:amqp results:django-db
accept_content: ['json']
broker_url: 'amqp://guest:********@localhost:5672//'
imports: ('operation_hawkeye.tasks',)
result_backend: 'django-db'
task_always_eager: False
[更新]
通过在 argsrepr_maxsize 中定义 celery.py 值解决了这个问题。虽然我之前知道这个解决方案，但我没有意识到 argsrepr_maxsize 在 celery.ampq 内。
[celery.py]
import os
from celery import Celery
os.environ.setdefault('DJANGO_SETTINGS_MODULE','hawkeye.settings')
os.environ.setdefault('FORKED_BY_MULTIPROCESSING','1')
app = Celery('hawkeye')
default_config = 'hawkeye.celeryconfig'
app.config_from_object(default_config)
app.amqp.argsrepr_maxsize = 10485760
解决方法
暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！
如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。
小编邮箱:dio#foxmail.com (将#修改为@）