问题描述
EDIT1 31/12/2020
它不起作用，因为 django 服务仍然不健康 需要帮助
编辑 31/12/2020
错误 使用 django/postgresql/celery/celery-beat 和 DOCKER 的 django_migrations 的 UniqueViolation（celery-beat）和 DuplicateTable（celery）错误
我正在学习 docker-compose healthcheck，因为 depends_on 是不够的，因为容器只等待它依赖的容器启动
如果 celery 和 celery-beat “wait” for django web 应用程序在 http://0.0.0.0:8000/ 上运行，它应该防止下面的错误吗？
...
django:
restart: always
container_name: django
...
healthcheck:
test: ["CMD","curl","-f","http://0.0.0.0:8000"]
interval: 30s
timeout: 10s
retries: 5
celery:
container_name: celery
build:
context: ./app
dockerfile: Dockerfile.dev
command: celery -A core worker -l info
volumes:
- ./app:/usr/src/app
env_file:
- ./.env.dev
depends_on:
web:
condition: service_healthy
redis:
condition: service_started
编辑
错误不一样，现在处理auth_permission表
我开发了一个 Django 应用并尝试为 dbbackup 实现异步任务
stack: django/postgresql/celery/celery-beat 和 DOCKER（只有 postgresql 不在容器中）
我是 celery 和 celery-beat 的新手
运行我的项目时，celery 和 celery-beat 有 2 个错误，我不明白
痕迹：
D:\Users\jl3\DOCKER\Apps\cafe_tropical>docker-compose -f docker-compose.dev.yml up
Creating network "cafe_tropical_default" with the default driver
Creating redis ... done Creating web ... done Creating celery ... done Creating celery-beat ... done Attaching to redis,celery,web,celery-beat
celery | Waiting for postgres...
celery | Postgresql started
celery-beat | Waiting for postgres...
celery-beat | Postgresql started
redis | 1:C 30 Dec 2020 15:21:18.345 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis | 1:C 30 Dec 2020 15:21:18.345 # Redis version=6.0.9,bits=64,commit=00000000,modified=0,pid=1,just started
redis | 1:C 30 Dec 2020 15:21:18.345 # Warning: no config file specified,using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis | 1:M 30 Dec 2020 15:21:18.346 * Running mode=standalone,port=6379.
redis | 1:M 30 Dec 2020 15:21:18.346 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis | 1:M 30 Dec 2020 15:21:18.346 # Server initialized
redis | 1:M 30 Dec 2020 15:21:18.346 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis | 1:M 30 Dec 2020 15:21:18.346 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo madvise > /sys/kernel/mm/transparent_hugepage/enabled' as root,and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled (set to 'madvise' or 'never').
redis | 1:M 30 Dec 2020 15:21:18.348 * Ready to accept connections
web | Waiting for postgres...
web | Postgresql started
celery | No changes detected
celery-beat | No changes detected
web | No changes detected
celery | Operations to perform:
celery | Apply all migrations: admin,api,auth,authtoken,cafe,contenttypes,django_celery_beat,parameters,sessions,sites
celery | Running migrations:
celery | Applying contenttypes.0001_initial... OK
web | Operations to perform:
web | Apply all migrations: admin,sites
web | Running migrations:
celery-beat | Operations to perform:
celery-beat | Apply all migrations: admin,sites
celery-beat | Running migrations:
celery | Applying auth.0001_initial... OK
celery-beat | Applying auth.0001_initial...Traceback (most recent call last):
celery-beat | File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py",line 82,in _execute
celery-beat | return self.cursor.execute(sql)
celery-beat | psycopg2.errors.UniqueViolation: ERREUR: la valeur d'une clé dupliquée rompt la contrainte unique « pg_type_typname_nsp_index »
celery-beat | DETAIL: La clé « (typname,typnamespace)=(auth_permission_id_seq,2200) » existe déjà.
celery-beat |
celery-beat |
celery-beat | The above exception was the direct cause of the following exception:
celery-beat |
celery-beat | Traceback (most recent call last):
celery-beat | File "manage.py",line 22,in <module>
web | Applying auth.0001_initial...Traceback (most recent call last):
web | File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py",in _execute
web | return self.cursor.execute(sql)
web | psycopg2.errors.UniqueViolation: ERREUR: la valeur d'une clé dupliquée rompt la contrainte unique « pg_type_typname_nsp_index »
web | DETAIL: La clé « (typname,2200) » existe déjà.
web |
web |
web | The above exception was the direct cause of the following exception:
web |
web | Traceback (most recent call last):
web | File "manage.py",in <module>
web | main()
web | File "manage.py",line 18,in main
celery-beat | main()
celery-beat | File "manage.py",in main
web | execute_from_command_line(sys.argv)
web | File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py",line 381,in execute_from_command_line
web | utility.execute()
web | File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py",line 375,in execute
web | self.fetch_command(subcommand).run_from_argv(self.argv)
web | File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py",line 323,in run_from_argv
web | self.execute(*args,**cmd_options)
web | File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py",line 364,in execute
web | output = self.handle(*args,**options)
web | File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py",line 83,in wrapped
web | res = handle_func(*args,**kwargs)
web | File "/usr/local/lib/python3.8/site-packages/django/core/management/commands/migrate.py",line 232,in handle
web | post_migrate_state = executor.migrate(
web | File "/usr/local/lib/python3.8/site-packages/django/db/migrations/executor.py",line 117,in migrate
web | state = self._migrate_all_forwards(state,plan,full_plan,fake=fake,fake_initial=fake_initial)
web | File "/usr/local/lib/python3.8/site-packages/django/db/migrations/executor.py",line 147,in _migrate_all_forwards
web | state = self.apply_migration(state,migration,line 245,in apply_migration
web | state = migration.apply(state,schema_editor)
web | File "/usr/local/lib/python3.8/site-packages/django/db/migrations/migration.py",line 124,in apply
web | operation.database_forwards(self.app_label,schema_editor,old_state,project_state)
web | File "/usr/local/lib/python3.8/site-packages/django/db/migrations/operations/models.py",line 92,in database_forwards
web | schema_editor.create_model(model)
web | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/schema.py",line 307,in create_model
web | self.execute(sql,params or None)
web | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/schema.py",line 137,in execute
web | cursor.execute(sql,params)
web | File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py",line 99,in execute
web | return super().execute(sql,line 67,in execute
web | return self._execute_with_wrappers(sql,params,many=False,executor=self._execute)
web | File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py",line 76,in _execute_with_wrappers
web | return executor(sql,many,context)
web | File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py",line 84,in _execute
web | return self.cursor.execute(sql,params)
web | File "/usr/local/lib/python3.8/site-packages/django/db/utils.py",line 89,in __exit__
web | raise dj_exc_value.with_traceback(traceback) from exc_value
web | File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py",in _execute
web | return self.cursor.execute(sql)
web | django.db.utils.IntegrityError: ERREUR: la valeur d'une clé dupliquée rompt la contrainte unique « pg_type_typname_nsp_index »
web | DETAIL: La clé « (typname,2200) » existe déjà.
web |
celery-beat | execute_from_command_line(sys.argv)
celery-beat | File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py",in execute_from_command_line
celery-beat | utility.execute()
celery-beat | File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py",in execute
celery-beat | self.fetch_command(subcommand).run_from_argv(self.argv)
celery-beat | File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py",in run_from_argv
celery-beat | self.execute(*args,**cmd_options)
celery-beat | File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py",in execute
celery-beat | output = self.handle(*args,**options)
celery-beat | File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py",in wrapped
celery-beat | res = handle_func(*args,**kwargs)
celery-beat | File "/usr/local/lib/python3.8/site-packages/django/core/management/commands/migrate.py",in handle
celery-beat | post_migrate_state = executor.migrate(
celery-beat | File "/usr/local/lib/python3.8/site-packages/django/db/migrations/executor.py",in migrate
celery-beat | state = self._migrate_all_forwards(state,fake_initial=fake_initial)
celery-beat | File "/usr/local/lib/python3.8/site-packages/django/db/migrations/executor.py",in _migrate_all_forwards
celery-beat | state = self.apply_migration(state,in apply_migration
celery-beat | state = migration.apply(state,schema_editor)
celery-beat | File "/usr/local/lib/python3.8/site-packages/django/db/migrations/migration.py",in apply
celery-beat | operation.database_forwards(self.app_label,project_state)
celery-beat | File "/usr/local/lib/python3.8/site-packages/django/db/migrations/operations/models.py",in database_forwards
celery-beat | schema_editor.create_model(model)
celery-beat | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/schema.py",in create_model
celery-beat | self.execute(sql,params or None)
celery-beat | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/schema.py",in execute
celery-beat | cursor.execute(sql,params)
celery-beat | File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py",in execute
celery-beat | return super().execute(sql,in execute
celery-beat | return self._execute_with_wrappers(sql,executor=self._execute)
celery-beat | File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py",in _execute_with_wrappers
celery-beat | return executor(sql,context)
celery-beat | File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py",in _execute
celery-beat | return self.cursor.execute(sql,params)
celery-beat | File "/usr/local/lib/python3.8/site-packages/django/db/utils.py",in __exit__
celery-beat | raise dj_exc_value.with_traceback(traceback) from exc_value
celery-beat | File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py",in _execute
celery-beat | return self.cursor.execute(sql)
celery-beat | django.db.utils.IntegrityError: ERREUR: la valeur d'une clé dupliquée rompt la contrainte unique « pg_type_typname_nsp_index »
celery-beat | DETAIL: La clé « (typname,2200) » existe déjà.
celery-beat |
celery | Applying admin.0001_initial... OK
celery | Applying admin.0002_logentry_remove_auto_add... OK
celery | Applying admin.0003_logentry_add_action_flag_choices... OK
celery | Applying api.0001_initial... OK
celery | Applying contenttypes.0002_remove_content_type_name... OK
celery | Applying auth.0002_alter_permission_name_max_length... OK
celery | Applying auth.0003_alter_user_email_max_length... OK
celery | Applying auth.0004_alter_user_username_opts... OK
celery | Applying auth.0005_alter_user_last_login_null... OK
celery | Applying auth.0006_require_contenttypes_0002... OK
celery | Applying auth.0007_alter_validators_add_error_messages... OK
celery | Applying auth.0008_alter_user_username_max_length... OK
celery | Applying auth.0009_alter_user_last_name_max_length... OK
celery | Applying auth.0010_alter_group_name_max_length... OK
celery | Applying auth.0011_update_proxy_permissions... OK
celery | Applying authtoken.0001_initial... OK
celery | Applying authtoken.0002_auto_20160226_1747... OK
celery | Applying cafe.0001_initial... OK
celery | Applying django_celery_beat.0001_initial... OK
celery | Applying django_celery_beat.0002_auto_20161118_0346... OK
celery | Applying django_celery_beat.0003_auto_20161209_0049... OK
celery | Applying django_celery_beat.0004_auto_20170221_0000... OK
celery | Applying django_celery_beat.0005_add_solarschedule_events_choices... OK
celery | Applying django_celery_beat.0006_auto_20180322_0932... OK
celery | Applying django_celery_beat.0007_auto_20180521_0826... OK
celery | Applying django_celery_beat.0008_auto_20180914_1922... OK
celery | Applying django_celery_beat.0006_auto_20180210_1226... OK
celery | Applying django_celery_beat.0006_periodictask_priority... OK
celery | Applying django_celery_beat.0009_periodictask_headers... OK
celery | Applying django_celery_beat.0010_auto_20190429_0326... OK
celery | Applying django_celery_beat.0011_auto_20190508_0153... OK
celery | Applying django_celery_beat.0012_periodictask_expire_seconds... OK
celery | Applying django_celery_beat.0013_auto_20200609_0727... OK
celery | Applying django_celery_beat.0014_remove_clockedschedule_enabled... OK
celery | Applying parameters.0001_initial... OK
celery | Applying sessions.0001_initial... OK
celery | Applying sites.0001_initial... OK
celery | Applying sites.0002_alter_domain_unique... OK
celery-beat | celery beat v5.0.5 (singularity) is starting.
celery-beat | __ - ... __ - _
celery-beat | LocalTime -> 2020-12-30 15:21:35
celery-beat | Configuration ->
celery-beat | . broker -> redis://redis:6379//
celery-beat | . loader -> celery.loaders.app.AppLoader
celery-beat | . scheduler -> celery.beat.PersistentScheduler
celery-beat | . db -> celerybeat-schedule
celery-beat | . logfile -> [stderr]@%INFO
celery-beat | . maxinterval -> 5.00 minutes (300s)
celery-beat | [2020-12-30 15:21:35,216: INFO/MainProcess] beat: Starting...
celery-beat | [2020-12-30 15:21:35,678: INFO/MainProcess] Scheduler: Sending due task hello (cafe.tasks.hello)
celery-beat | [2020-12-30 15:21:35,724: INFO/MainProcess] Scheduler: Sending due task backup (cafe.tasks.backup)
web | Watching for file changes with StatReloader
web | Performing system checks...
web |
web | System check identified no issues (0 silenced).
web | December 30,2020 - 15:21:37
web | Django version 2.2.5,using settings 'core.settings.dev'
web | Starting development server at http://0.0.0.0:8000/
web | Quit the server with CONTROL-C.
celery | /usr/local/lib/python3.8/site-packages/celery/platforms.py:796: RuntimeWarning: You're running the worker with superuser privileges: this is
celery | absolutely not recommended!
celery |
celery | Please specify a different user using the --uid option.
celery |
celery | User information: uid=0 euid=0 gid=0 egid=0
celery |
celery | warnings.warn(RuntimeWarning(ROOT_disCOURAGED.format(
celery |
celery | -------------- celery@e53033835ccb v5.0.5 (singularity)
celery | --- ***** -----
celery | -- ******* ---- Linux-4.19.128-microsoft-standard-x86_64-with 2020-12-30 15:21:37
celery | - *** --- * ---
celery | - ** ---------- [config]
celery | - ** ---------- .> app: core:0x7fe23b6ba220
celery | - ** ---------- .> transport: redis://redis:6379//
celery | - ** ---------- .> results: redis://redis:6379/
celery | - *** --- * --- .> concurrency: 4 (prefork)
celery | -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
celery | --- ***** -----
celery | -------------- [queues]
celery | .> celery exchange=celery(direct) key=celery
celery |
celery |
celery | [tasks]
celery | . cafe.tasks.backup
celery | . cafe.tasks.hello
celery | . core.celery.debug_task
celery |
celery | [2020-12-30 15:21:37,940: INFO/MainProcess] Connected to redis://redis:6379//
celery | [2020-12-30 15:21:37,951: INFO/MainProcess] mingle: searching for neighbors
celery | [2020-12-30 15:21:38,979: INFO/MainProcess] mingle: all alone
celery | [2020-12-30 15:21:39,009: WARNING/MainProcess] /usr/local/lib/python3.8/site-packages/celery/fixups/django.py:203: UserWarning: Using settings.DEBUG leads to a memory
但对于 django Web 应用程序容器，所有迁移似乎都在正确运行
我尝试在构建和运行容器之前删除 postgresql 数据库，所以我不明白 django_migrations 表如何已经存在
可能是容器之间的依赖
我根据redis设置了所有容器
version: '3.7'
services:
web:
restart: always
container_name: web
build:
context: ./app
dockerfile: Dockerfile.dev
restart: always
command: python manage.py runserver 0.0.0.0:8000
volumes:
- ./app:/usr/src/app
ports:
- 8000:8000
env_file:
- ./.env.dev
depends_on:
- redis
redis:
image: "redis:alpine"
celery:
build:
context: ./app
dockerfile: Dockerfile.dev
command: celery -A core worker -l info
volumes:
- ./app:/usr/src/app
env_file:
- ./.env.dev
depends_on:
- redis
celery-beat:
build:
context: ./app
dockerfile: Dockerfile.dev
command: celery -A core beat -l info
volumes:
- ./app:/usr/src/app
env_file:
- ./.env.dev
depends_on:
- redis
解决方法
尝试使用 http://localhost:8000/（不要忘记末尾的“/”），它应该可以工作