在CentOS 7上使用systemd使Celery进程守护化

3
我正在尝试使用systemd将celery 4.3.0作为守护进程运行,但是出现了以下错误:
它启动了工作进程,但很快停止了它们。然而,我可以通过键入命令来手动运行worker:
celery -A bin.proj.celery_app worker1
celery.service - Celery Service
   Loaded: loaded (/etc/systemd/system/celery.service; enabled; vendor preset: disabled)
   Active: inactive (dead) since Fri 2020-04-10 08:39:59 CEST; 35s ago
  Process: 19276 ExecStop=/bin/sh -c ${CELERY_BIN} multi stopwait ${CELERYD_NODES} -- 
pidfile=${CELERYD_PID_FILE} (code=exited, status=0/SUCCESS)
  Process: 19245 ExecStart=/bin/sh -c ${CELERY_BIN} multi start ${CELERYD_NODES} -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} ${CELERYD_OPTS} (code=exited, status=0/SUCCESS)
 Main PID: 19245 (code=exited, status=0/SUCCESS)

Apr 10 08:39:58 prod1 sh[19276]: > Stopping nodes...
Apr 10 08:39:58 prod1 sh[19276]: > worker1@prod1: TERM -> 19254
Apr 10 08:39:58 prod1 sh[19276]: > worker2@prod1: TERM -> 19264
Apr 10 08:39:58 prod1 sh[19276]: > Waiting for 2 nodes -> 19254, 19264......
Apr 10 08:39:58 prod1 sh[19276]: > worker1@prod1: OK
Apr 10 08:39:58 prod1 sh[19276]: > worker1@prod1: DOWN
Apr 10 08:39:58 prod1 sh[19276]: > Waiting for 2 nodes -> None, None....
Apr 10 08:39:58 prod1 sh[19276]: > worker2@prod1: OK
Apr 10 08:39:58 prod1 sh[19276]: > worker2@prod1: DOWN
Apr 10 08:39:58 prod1 sh[19276]: > Waiting for 1 node -> None...

这里是 celery.service 的内容:

[Unit]
Description=Celery Service
After=network.target

[Service]
type=forking
User=ssc
Group=ssc
EnvironmentFile=/etc/conf/celery
WorkingDirectory=/home/ssc/Mr.Robot/
ExecStart=/bin/sh -c '${CELERY_BIN} multi start ${CELERYD_NODES} -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} ${CELERYD_OPTS}'
ExecStop=/bin/sh -c '${CELERY_BIN} multi stopwait ${CELERYD_NODES} --pidfile=${CELERYD_PID_FILE}'
ExecReload=/bin/sh -c '${CELERY_BIN} restart ${CELERYD_NODES} -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FILE} ${CELERYD_OPTS}'

[Install]
WantedBy=multi-user.target

这是相关的配置文件:

# Name of nodes to start
# here we have a single node
#CELERYD_NODES="w1"
# or we could have three nodes:
CELERYD_NODES="worker1 worker2 worker3"

# Absolute or relative path to the 'celery' command:
CELERY_BIN="/usr/local/bin/celery"
#CELERY_BIN="/virtualenvs/def/bin/celery"

# App instance to use
# comment out this line if you don't use an app
CELERY_APP="bin.proj.celery_app"
# or fully qualified:
#CELERY_APP="proj.tasks:app"

# How to call manage.py
CELERYD_MULTI="multi"

# Extra command-line arguments to the worker
CELERYD_OPTS="--loglevel=INFO --time-limit=300 --concurrency=3"

# - %n will be replaced with the first part of the nodename.
# - %I will be replaced with the current child process index
#   and is important when using the prefork pool to avoid race conditions.
CELERYD_PID_FILE="/var/run/celery/%n.pid"
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_LOG_LEVEL="INFO"

# you may wish to add these options for Celery Beat
CELERYBEAT_PID_FILE="/var/run/celery/beat.pid"
CELERYBEAT_LOG_FILE="/var/log/celery/beat.log"

还有应用程序文件:

import sys,os
sys.path.append(os.getcwd())

from celery import Celery
from bin.proj.utils import create_rotating_log


# initialize logger
global logger
logger = create_rotating_log('./log/activity.log')

# initialize celery
try:
    app =   Celery('celery_app', 
                    broker="".join(['redis://:pass@localhost:6379/10']), 
                    backend="".join(['redis://:pass@localhost:6379/11']), 
                    include=['bin.proj.tasks'])
    #logger.info('Celery session OK')
except Exception as e:
    logger.error('Celery session KO')
    logger.error(e)

if __name__ == '__main__':
    app.start() 

用户是“ssc”,该用户不属于root组,我已经为“ssc”用户在项目上赋予了所需的所有权限。 任何帮助都将不胜感激,我已经花费了太多时间,谢谢。

嘿,你修好了吗?每次设置Celery守护进程都让我头疼不已。如果你能分享答案就太好了。 - Ken
1个回答

0

我曾经遇到过类似的问题,即守护进程化celery时与pidfile和logfile权限相关的问题。

目录/var/run/celery/var/log/celery应该属于运行服务的用户,这里是用户scc。问题在于当作为守护进程运行时,输出被关闭,从而隐藏错误消息。

如果您想调试服务并找到问题,只需使用'scc'帐户运行完整命令。它应该告诉您任何访问权限错误或其他任何问题。

/bin/sh -c '/usr/local/bin/celery multi start worker1 worker2 worker3 -A bin.proj.celery_app --pidfile=/var/run/celery/%n.pid --logfile=/var/log/celery/%n%I.log --loglevel=INFO --time-limit=300 --concurrency=3'

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接