Gitlab CI:构建opencv-python轮失败

3

我正在为我的Python/Django项目在GitLab上进行CI/CD。

我遇到了一个错误--GitLab CI:构建OpenCV-Python的wheel失败。

完整的GitLab CI日志--https://pastebin.com/pZdZ6ws2

我在build_pip阶段遇到了一个错误:gitlab-ci.yaml

stages:
  - linter
  - build_pip
  - build
  - meta
  - code_quality
  - deploy

.except-tags:
  except:
    - tags

build_pip:build_dist:
  stage: build_pip
  # image: $CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX/python:3.9-alpine
  image: python:3.9-alpine
  before_script:
    - apk update && apk add postgresql-dev gcc python3-dev musl-dev g++ jpeg-dev zlib-dev
    - pip install pip --upgrade
    - pip install -r requirements/production.txt --no-cache
  script:
    - python setup.py bdist_wheel
    - echo PIP_CI_JOB_ID=$CI_JOB_ID > PIP_CI_JOB_ID.env
  dependencies: []
  artifacts:
    expire_in: 1 hour
    paths:
      - dist/
      - version
    reports:
      dotenv: PIP_CI_JOB_ID.env
  extends:
    - .except-tags

meta:version:
  stage: meta
  needs:
    - job: build_pip:build_dist
      artifacts: true
  script:
    - cat version
  artifacts:
    expire_in: never
    paths:
      - version
  extends: .except-tags


build:build_api:
  stage: build
  image: registry.ml.bastion-tech.ru:8843/ansible/infrastructure/ansible_tools:2.9
  needs:
    - job: build_pip:build_dist
      artifacts: true
  before_script:
    - ansible-vault decrypt /ansible/infrastructure/secrets/ansible@infrastructure/id_rsa --vault-password-file=${ANSIBLE_VAULT_PASSWORD}
  script:
    - |
      ansible-playbook -i /ansible/infrastructure/inventories/ml.inventory \
      --vault-password-file=${ANSIBLE_VAULT_PASSWORD} \
      --private-key /ansible/infrastructure/secrets/ansible@infrastructure/id_rsa \
      -e ansible_ssh_user=deploy \
      -e smartconstructions_pip_ci_job_id=${PIP_CI_JOB_ID} \
      -e build=true -e smartconstructions_build_ref=${CI_COMMIT_BRANCH} \
      /ansible/infrastructure/ml_smartconstructions.yml
  tags:
    - linux-docker

deploy:deploy_api:
  stage: deploy
  image: registry.ml.bastion-tech.ru:8843/ansible/infrastructure/ansible_tools:2.9
  needs:
    - job: build_pip:build_dist
      artifacts: true
  when: manual
  only:
    - master
    - dev
  before_script:
    - ansible-vault decrypt /ansible/infrastructure/secrets/ansible@infrastructure/id_rsa --vault-password-file=${ANSIBLE_VAULT_PASSWORD}
  script:
    - |
      ansible-playbook -i /ansible/infrastructure/inventories/ml.inventory \
      --vault-password-file=${ANSIBLE_VAULT_PASSWORD} \
      --private-key /ansible/infrastructure/secrets/ansible@infrastructure/id_rsa \
      -e ansible_ssh_user=deploy \
      -e smartconstructions_pip_ci_job_id=${PIP_CI_JOB_ID} \
      -e run=true -e frontend_restart=true \
      /ansible/infrastructure/ml_smartconstructions.yml
  tags:
    - linux-docker

include:
  - local: .gitlab/ci/code-quality.yml

requirements/production.txt

djangorestframework==3.12.4
drf-extra-fields==3.1.1
djangorestframework-camel-case==1.2.0  # https://pypi.org/project/djangorestframework-camel-case/
Pillow==8.3.2
python-dateutil==2.8.2  # datetime formatting
psycopg2==2.9.1
opencv-python==4.5.3.56
drf-yasg==1.20.0
sentry-sdk==1.4.3
gunicorn==20.1.0
requests==2.26.0
yarl==1.7.0
googlemaps==4.5.3
django_redis==5.0.0
celery==5.2.0
channels==3.0.4
channels_redis==3.3.1
2个回答

4

在你的日志中,我们可以看到以下错误:

gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -DTHREAD_STACK_SIZE=0x100000 -fPIC -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/usr/local/include/python3.9 -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.9/c/_cffi_backend.o
c/_cffi_backend.c:15:10: fatal error: ffi.h: No such file or directory
  15 | #include <ffi.h>
    |          ^~~~~~~
compilation terminated.
error: command '/usr/bin/gcc' failed with exit code 1

这样的错误提示表明您缺少头文件。

在Alpine中,ffi.h文件应该是libffi-dev的一部分。尝试执行以下操作:

apk add libffi-dev 

谢谢,我已添加这个库,但在opencv 中仍然出现相同的错误。完整日志: https://pastebin.com/XELTV8aj - Alex
1
错误提示为 CMake Error: CMake was unable to find a build program corresponding to Ninja。也许需要执行 apk add ninja 命令安装Ninja? - SYN

1
也许你的用例可以从使用opencv-python-headless受益?它基本上是相同的库,由相同的人维护,但依赖于Xorg和GUI软件包较少。这意味着由于缺少软件包而导致的错误较少(即安装过程中需要安装的内容较少)。
你可以在development.txt文件中使用opencv-python,并将生产环境更改为无头版本。
从文档中可以得知:

这些软件包比上面介绍的两个软件包要小,因为它们不包含任何GUI功能(未编译Qt /其他GUI组件)。这意味着软件包避免了对X11库的重度依赖链,结果您会得到例如更小的Docker映像。如果您不使用cv2.imshow等功能或者使用OpenCV之外的其他软件包(例如PyQt)来创建GUI,则应始终使用这些软件包。

选项3-无头主模块软件包:pip install opencv-python-headless

选项4-无头完整软件包(包含主模块和contrib / extra模块):pip install opencv-contrib-python-headless(从OpenCV文档中检查contrib / extra模块列表)


除此之外,使用基于Alpine的镜像+ Python +科学工具真的是个好主意吗?考虑到一些Python工具与GCC等工具有多么纠缠不清,即使这个镜像成功构建,你也可能会在运行时遇到更多问题。
尝试使用3.9-slim-bullseye,它是一个基于Debian的镜像。

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接