logo标签列表

使用VSCode调试Celery

pythonvisual-studio-codeceleryvscode-debugger
23

我正在使用VSCode开发基于Django框架的网站。对于Django调试没有问题，但是当我尝试使用Celery时，调试器不会在断点处停止。以下是我用于运行Celery和Celery Beat的配置：

{
    "name": "Beat",
    "type": "python",
    "request": "launch",
    "pythonPath": "/home/MyName/job/MyProject/venv/bin/python",
    "program": "/home/MyName/job/MyProject/venv/bin/celery",
    "console": "integratedTerminal",
    "args": [
        "-A",
        "bgp",
        "beat",
        "-l",
        "info"
    ]
},
{
    "name": "Celery",
    "type": "python",
    "request": "launch",
    "pythonPath": "/home/MyName/job/MyProject/venv/bin/python",
    "program": "/home/MyName/job/MyProject/venv/bin/celery",
    "console": "integratedTerminal",
    "args": [
        "-A",
        "bgp",
        "worker",
        "-l",
        "info",
        "-Q",
        "ssh",
        "--concurrency=1",
    ]
},

当我运行Celery时，我遇到了以下回溯信息：

[2018-11-29 13:18:34,112: CRITICAL/MainProcess] Unrecoverable error: RuntimeError('already started',)
Traceback (most recent call last):
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/celery/worker/worker.py", line 205, in start
    self.blueprint.start(self)
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/celery/bootsteps.py", line 119, in start
    step.start(parent)
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/celery/bootsteps.py", line 369, in start
    return self.obj.start()
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/celery/concurrency/base.py", line 131, in start
    self.on_start()
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/celery/concurrency/prefork.py", line 112, in on_start
    **self.options)
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/celery/concurrency/asynpool.py", line 432, in __init__
    super(AsynPool, self).__init__(processes, *args, **kwargs)
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/billiard/pool.py", line 1007, in __init__
    self._create_worker_process(i)
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/celery/concurrency/asynpool.py", line 449, in _create_worker_process
    return super(AsynPool, self)._create_worker_process(i)
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/billiard/pool.py", line 1116, in _create_worker_process
    w.start()
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/billiard/process.py", line 124, in start
    self._popen = self._Popen(self)
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/billiard/context.py", line 333, in _Popen
    return Popen(process_obj)
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/billiard/popen_fork.py", line 24, in __init__
    self._launch(process_obj)
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/billiard/popen_fork.py", line 72, in _launch
    self.pid = os.fork()
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/ptvsd/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 488, in new_fork
    _on_forked_process()
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/ptvsd/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 56, in _on_forked_process
    pydevd.settrace_forked()
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/ptvsd/_vendored/pydevd/pydevd.py", line 1723, in settrace_forked
    patch_multiprocessing=True,
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/ptvsd/_vendored/pydevd/pydevd.py", line 1488, in settrace
    stop_at_frame,
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/ptvsd/_vendored/pydevd/pydevd.py", line 1536, in _locked_settrace
    debugger.connect(host, port)  # Note: connect can raise error.
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/ptvsd/_vendored/pydevd/pydevd.py", line 484, in connect
    s = start_client(host, port)
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/ptvsd/pydevd_hooks.py", line 125, in <lambda>
    _start_client = (lambda h, p: start_client(daemon, h, p))
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/ptvsd/pydevd_hooks.py", line 71, in start_client
    sock, start_session = daemon.start_client((host, port))
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/ptvsd/daemon.py", line 208, in start_client
    with self.started():
  File "/usr/local/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/ptvsd/daemon.py", line 109, in started
    self.start()
  File "/home/MyName/job/MyProject/venv/lib/python2.7/site-packages/ptvsd/daemon.py", line 144, in start
    raise RuntimeError('already started')
RuntimeError: already started
[2018-11-29 13:18:34,158: INFO/MainProcess] Connected to amqp://project:**@127.0.0.1:5672/project
[2018-11-29 13:18:34,210: INFO/MainProcess] mingle: searching for neighbors
[2018-11-29 13:18:35,292: INFO/MainProcess] mingle: all alone
[2018-11-29 13:18:35,353: WARNING/MainProcess] /home/MyName/job/MyProject/venv/lib/python2.7/site-packages/celery/fixups/django.py:200: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
  warnings.warn('Using settings.DEBUG leads to a memory leak, never '
[2018-11-29 13:18:35,354: INFO/MainProcess] celery@MyName-vm ready..

Celery运行良好。 但是，当我在任何任务中设置断点时，Celery线程不会停在它上面。 我该如何解决？
我的Celery版本：celery [redis] == 4.2.0

我不是孤单一人。 - anderson
我也需要解决这个问题。 - Perfect
4个回答
49

您可以尝试将“-P solo”添加到Celery配置的参数中。 请参考https://github.com/Microsoft/ptvsd/issues/1046

这是我的Celery配置。 对我来说它可以正常工作。

    {
        "name": "Python: Celery",
        "type": "python",
        "request": "launch",
        "module": "celery",
        "console": "integratedTerminal",
        "args": [
            "-A",
            "tsbc",
            "worker",
            "-l",
            "info",
            "-P",
            "solo",
        ]
    }
1
这对我很有用，它节省了我许多手动调试的时间。非常感谢@Kevin Z. Li。 - Divyarajsinh Jadeja
1
也许很明显，但如果你正在使用 python setup.py install 安装你的代码，请确保在 site-packages 版本中设置断点，而不是在你的项目工作目录中，否则断点将无法触发。 - mckendricks
6

我发现使用VS Code中的debugpy可以比Celery pdb有更好的调试体验。即使你不使用docker-compose，也可以使用这个方法。

首先，使用debugpy启动celery。

  celery:
    command:
      [
        "sh",
        "-c",
        "pip install debugpy -t /tmp && python /tmp/debugpy --listen 0.0.0.0:6900 -m celery -A backend.celery worker -l info",
      ]
    ports:
      - 6900:6900

接下来是launch.json文件：

{
    "name": "Celery: Remote Attach",
    "type": "python",
    "request": "attach",
    "connect": {
        "host": "localhost",
        "port": 6900
    },
    "pathMappings": [{
        "localRoot": "${workspaceFolder}",
        "remoteRoot": "/app"
    }],
    // "preLaunchTask": "docker-compose up",
    "django": true,
},

就是这样，享受调试吧！ 您可以像往常一样设置断点和评估变量。 输入图像描述

我有一篇关于此的文章，欢迎阅读。

我尝试按照这种方法进行操作，调试器在断点处完美地暂停了。但是当我断开调试器时，即使调试子进程已经终止，它仍然没有断开连接。 - Pravin
断开连接（分离）不会停止Celery，您可以只需docker-compose stop/restart celery（如果我正确理解了您的问题）。 - miaoz2001
这是 debugpy / vscode 的限制吗？此外，热重载似乎无法与 Celery 一起使用。对于没有 —noreload —nothreading 的应用程序，它可以正常工作。 - Pravin
热重载是 Django 的一个功能，而不是 Celery 的。如果您的代码发生更改，您必须手动重新启动 Celery。 - miaoz2001
6

这是我的配置，运行得很好。

{
  "name": "Python: Django Shell",
  "type": "python",
  "request": "launch",
  "program": "${workspaceFolder}/manage.py",
  "args": [
    "shell"
  ],
  "django": true
},
{
  "name": "Python: Celery Workers",
  "type": "python",
  "request": "launch",
  "module": "celery",
  "console": "integratedTerminal",
  "envFile": "${workspaceFolder}/.env",
  "args": ["-A", "yourproject", "worker", "-l", "debug", "-Q", "queueName"]

}

1

对我来说，没有任何答案适用，因为我不想安装额外的调试器。同时，在参数中放置“-P solo”并不是必需的，正如其中一个答案所指出的那样。以下是对我有效的配置：

需要注意的仅有两件事：

  1. cwd 或当前工作目录应该与您在终端中运行 celery 的文件夹相同。在我的情况下，clickserver 是包含 celery.py 的 django 文件夹，因此我将在该文件夹上方运行它。
  2. PYTHONPATH 应该与 cwd 相同。

这应该可以无缝地工作。

 "configurations": [
            {
                "name": "Python: Celery",
                "type": "python",
                "cwd": "${workspaceFolder}/clickstream-server/django/clickserver",
                "request": "launch",
                "module": "celery",
                "console": "integratedTerminal",
                "env": {
                    "PYTHONPATH": "${workspaceFolder}/clickstream-server/django/clickserver"
                  },
                "args": [
                    "-A",
                    "clickserver",
                    "worker",
                    "-l",
                    "info",
                ]
            },
网页内容由stack overflow 提供, 点击上面的
可以查看英文原文，
原文链接