从 docker 运行 celery 时失败

问题描述

我正在尝试让 celery 与 Django 和 docker 一起工作，并且该建筑运行良好，但我的 celery 无法运行。有什么想法吗？

这是 docker-compose logs -f 错误

Starting django-celery_redis_1 ... done
Starting django-celery_db_1    ... done
Starting django-celery_flower_1        ... done
Starting django-celery_celery_beat_1   ... done
Starting django-celery_celery_worker_1 ... done
Starting django-celery_web_1           ... done
Attaching to django-celery_db_1,django-celery_redis_1,django-celery_celery_worker_1,django-celery_flower_1,django-celery_celery_beat_1,django-celery_web_1
celery_beat_1    | standard_init_linux.go:219: exec user process caused: exec format error
db_1             | 2021-03-28 18:18:15.611 UTC [1] LOG:  starting Postgresql 12.0 on x86_64-pc-linux-musl,compiled by gcc (Alpine 8.3.0) 8.3.0,64-bit
db_1             | 2021-03-28 18:18:15.613 UTC [1] LOG:  listening on IPv4 address "0.0.0.0",port 5432
db_1             | 2021-03-28 18:18:15.616 UTC [1] LOG:  listening on IPv6 address "::",port 5432
celery_worker_1  | standard_init_linux.go:219: exec user process caused: exec format error
db_1             | 2021-03-28 18:18:15.648 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGsql.5432"
redis_1          | 1:C 28 Mar 2021 18:18:15.425 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1          | 1:C 28 Mar 2021 18:18:15.425 # Redis version=5.0.12,bits=64,commit=00000000,modified=0,pid=1,just started
redis_1          | 1:C 28 Mar 2021 18:18:15.425 # Warning: no config file specified,using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1          | 1:M 28 Mar 2021 18:18:15.427 * Running mode=standalone,port=6379.
redis_1          | 1:M 28 Mar 2021 18:18:15.427 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1          | 1:M 28 Mar 2021 18:18:15.427 # Server initialized
redis_1          | 1:M 28 Mar 2021 18:18:15.427 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root,and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1          | 1:M 28 Mar 2021 18:18:15.428 * DB loaded from disk: 0.000 seconds
redis_1          | 1:M 28 Mar 2021 18:18:15.428 * Ready to accept connections
flower_1         | standard_init_linux.go:219: exec user process caused: exec format error
web_1            | standard_init_linux.go:219: exec user process caused: exec format error
db_1             | 2021-03-28 18:18:15.777 UTC [19] LOG:  database system was shut down at 2021-03-28 18:16:52 UTC
db_1             | 2021-03-28 18:18:15.791 UTC [1] LOG:  database system is ready to accept connections
django-celery_celery_worker_1 exited with code 1
django-celery_flower_1 exited with code 1
django-celery_celery_beat_1 exited with code 1
django-celery_web_1 exited with code 1

更新：添加了 docker-compose.yml 文件，以便更好地参考堆栈问题。构建成功，但在运行 docker-compose up 时没有成功并抛出 celery 错误

docker-compose.yml

version: '3.8'

services:
  web:
    build:
      context: .
      dockerfile: ./compose/local/django/Dockerfile
    image: django_celery_example_web
    # '/start' is the shell script used to run the service
    command: /start
    # this volume is used to map the files and folders on the host to the container
    # so if we change code on the host,code in the docker container will also be changed
    volumes:
      - .:/app
    ports:
      - 8010:8000
    # env_file is used to manage the env variables of our project
    env_file:
      - ./.env/.dev-sample
    depends_on:
      - redis
      - db

  db:
    image: postgres:12.0-alpine
    volumes:
      - postgres_data:/var/lib/postgresql/data/
    environment:
      - POSTGRES_DB=hello_django_dev
      - POSTGRES_USER=hello_django
      - POSTGRES_PASSWORD=hello_django

  redis:
    image: redis:5-alpine

  celery_worker:
    build:
      context: .
      dockerfile: ./compose/local/django/Dockerfile
    image: django_celery_example_celery_worker
    command: /start-celeryworker
    volumes:
      - .:/app
    env_file:
      - ./.env/.dev-sample
    depends_on:
      - redis
      - db

  celery_beat:
    build:
      context: .
      dockerfile: ./compose/local/django/Dockerfile
    image: django_celery_example_celery_beat
    command: /start-celerybeat
    volumes:
      - .:/app
    env_file:
      - ./.env/.dev-sample
    depends_on:
      - redis
      - db

  flower:
    build:
      context: .
      dockerfile: ./compose/local/django/Dockerfile
    image: django_celery_example_celey_flower
    command: /start-flower
    volumes:
      - .:/app
    env_file:
      - ./.env/.dev-sample
    ports:
      - 5557:5555
    depends_on:
      - redis
      - db

volumes:
  postgres_data:

解决方法

我不确定它是否适用于您的情况，但在 docker 中，我使用以下命令启动 celery：

command: celery -A my_proj worker -l DEBUG

因为它给你的错误是“exec format error”，所以可能就是这个。