在Dockerfile中安装PyTorch CPU版本

8

我对 Docker 和容器化还比较陌生。我想要减小在生产环境中 my_proj Docker 容器的大小。

我更喜欢通过 Poetry 安装软件包和管理依赖。

如何在 Dockerfile 中指定使用只支持 CPUPyTorch

如果要在 bash 终端上执行此操作,可以输入以下命令:

poetry add pytorch-cpu torchvision-cpu -c pytorch

(或者conda install...)


我现有的Dockerfile

FROM python:3.7-slim as base
RUN apt-get update -y \
    && apt-get -y --no-install-recommends install curl wget\
    && rm -rf /var/lib/apt/lists/* 
ENV ROOT /home/worker/python/my_proj
WORKDIR $ROOT

ARG ATLASSIAN_TOKEN
ARG POETRY_HTTP_BASIC_AZURE_PASSWORD
ARG ACCESS_KEY
ENV AWS_ACCESS_KEY_ID=$ACCESS_KEY
ARG SECRET_KEY
ENV AWS_SECRET_ACCESS_KEY=$SECRET_KEY
ARG REPO
ENV REPO_URL=$REPO
ENV PYPIRC_PATH=$ROOT/.pypirc

ENV \
    PYTHONFAULTHANDLER=1 \
    POETRY_VERSION=1.1.4 \
    POETRY_HOME=/etc/poetry \
    XDG_CACHE_HOME=/home/worker/.cache \
    POETRY_VIRTUALENVS_IN_PROJECT=true \
    MPLCONFIGDIR=/home/worker/matplotlib \
    PATH=/home/worker/python/my_proj/.venv/bin:/usr/local/bin:/etc/poetry/bin:$PATH

ADD https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py ./
RUN python get-poetry.py && chmod +x /etc/poetry/bin/poetry
RUN --mount=type=cache,target=/root/.cache pip install twine keyring artifacts-keyring
RUN --mount=type=cache,target=/root/.cache apt update && apt install gcc -y

FROM base as ws
ARG WS_APIKEY
ARG WS_PROJECTVERSION=
ARG WS_PROJECTNAME=workers-python-my_proj
ARG WS_PRODUCTNAME=HALO
COPY --chown=worker:worker . .
RUN --mount=type=cache,uid=1000,target=/home/worker/.cache poetry install --no-dev
COPY --from=openjdk:15-slim-buster /usr/local/openjdk-15 /usr/local/openjdk-15
ENV JAVA_HOME /usr/local/openjdk-15
ENV PATH $JAVA_HOME/bin:$PATH
RUN --mount=type=cache,uid=1000,target=/home/worker/.cache ./wss_agent.sh

FROM base as test
COPY . .
RUN poetry config experimental.new-installer false
RUN poetry install
RUN cd my_proj && poetry run invoke deployconfluence_server_pass=$ATLASSIAN_TOKEN

FROM base as package
COPY . .
RUN poetry build
RUN python -m pip install --upgrade pip && \
pip install twine keyring artifacts-keyring && \
twine upload -r $REPO_URL --config-file $PYPIRC_PATH dist/* --skip-existing

FROM base as build
COPY . .
RUN poetry config experimental.new-installer false
RUN poetry install --no-dev
RUN pip3 --no-cache-dir install --upgrade awscli
RUN aws s3 cp s3://....tar.gz $ROOT/my_proj # censored url
RUN mkdir $ROOT/my_proj/bert-base-cased && cd $ROOT/my_proj/bert-base-cased && \
wget https://huggingface.co/bert-base-cased/resolve/main/config.json && \
wget https://huggingface.co/bert-base-cased/resolve/main/tokenizer.json && \
wget https://huggingface.co/bert-base-cased/resolve/main/tokenizer_config.json 

FROM python:3.7-slim as production
ENV  ROOT=/home/worker/python/my_proj \
     VIRTUAL_ENV=/home/worker/python/my_proj/.venv\
     PATH=/home/worker/python/my_proj/.venv/bin:/home/worker/python/my_proj:$PATH
COPY --from=build /home/worker/python/my_proj/pyproject.toml /home/worker/python/
COPY --from=build /home/worker/python/my_proj/.venv /home/worker/python/my_proj/.venv
COPY --from=build /home/worker/python/my_proj/my_proj /home/worker/python/my_proj
WORKDIR $ROOT
ENV PYTHONPATH=$ROOT:/home/worker/python/
ENTRYPOINT [ "primary_worker", "--mongo" ]
2个回答

9

使用pip安装应该可以解决:

RUN pip3 install torch==1.9.0+cpu torchvision==0.10.0+cpu torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html

有没有一种方法可以“测试运行”Dockerfile?在隔离的环境中,比如在我的本地机器上或者在一个网站上? - user16341274
我应该在哪一行插入/替换这个代码? - user16341274

0

如果你遇到了 ERROR: Could not find a version that satisfies the requirement / ERROR: No matching distribution found 的问题,以下是通过 pip 安装的另一种方法:

RUN pip install torch==2.0.0 --index-url https://download.pytorch.org/whl/cpu

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接