大家好,
我在一个专门用于开发的Docker容器中使用Julia。各个开发人员在不同架构的主机上使用相同的Dockerfile。Julia已经安装在Docker镜像中,但是源代码存储库在运行时作为挂载卷加载。虽然它正常工作,但每次启动容器时都需要重新实例化Julia环境。因此,我正在尝试在构建时将实例化的环境嵌入到Docker映像中。
在将Julia安装到Docker镜像中之后,我会将项目存储库暂时复制到镜像内与运行时相同的目录中。以下是Dockerfile的一部分:
# syntax = edrevo/dockerfile-plus
ARG PROGRAM_NAME=wrong_program
ARG DOCKER_UBUNTU_VERSION=wrong_version
FROM ubuntu:${DOCKER_UBUNTU_VERSION}
SHELL ["/bin/bash", "-c"]
RUN apt-get update \
&& apt-get install -y -qq --no-install-recommends \
libglvnd0 \
libgl1 \
libglx0 \
libegl1 \
libxext6 \
libx11-6 \
glmark2 \
mesa-utils \
&& rm -rf /var/lib/apt/lists/*
# Env vars for the nvidia-container-runtime.
ENV NVIDIA_VISIBLE_DEVICES all
ENV NVIDIA_DRIVER_CAPABILITIES graphics,utility,compute
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
apt-get install -qqy --no-install-recommends \
sudo \
apt-utils \
autotools-dev \
build-essential \
ca-certificates \
g++ \
git \
iputils-ping \
libssl-dev \
nano \
openssl \
python-dev \
unzip \
vim \
sed \
x11-apps
RUN update-ca-certificates
# Change to non-root privilege
# https://dev.to/emmanuelnk/using-sudo-without-password-prompt-as-non-root-docker-user-52bg
ARG USERNAME=wrongUser
ARG USERID=1000
ARG GID=101
ARG GROUPNAME=dev
RUN addgroup --gid ${GID} ${GROUPNAME} \
&& adduser --uid ${USERID} --disabled-password --home /home/${USERNAME} --shell /bin/bash --gecos '' ${USERNAME} \
&& adduser ${USERNAME} sudo \
&& adduser ${USERNAME} dev \
&& grep -qxF '%sudo ALL=(ALL) NOPASSWD:ALL' /etc/sudoers || echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
USER ${USERNAME}
RUN grep -qxF 'export LS_COLORS="$LS_COLORS:ow=1;34:tw=1;34:"' ~/.bashrc || echo 'export LS_COLORS="$LS_COLORS:ow=1;34:tw=1;34:"' >> ~/.bashrc
ARG TEMPDIR=${THIRDPARTYDIR}/temp
RUN echo "THIRDPARTYDIR = ${THIRDPARTYDIR}" && \
echo "TEMPDIR = ${TEMPDIR}"
#--------------------------------------
# Install julia
#--------------------------------------
ARG JULIA_VERSION
ARG JULIA_DOWNLOAD_FILE
RUN echo "JULIA_VERSION = ${JULIA_VERSION}" && \
echo "JULIA_DOWNLOAD_FILE = ${JULIA_DOWNLOAD_FILE}"
COPY --chown=${USERNAME}:${USERNAME} downloads/${JULIA_DOWNLOAD_FILE} ${THIRDPARTYDIR}/
RUN chown ${USERNAME}:${USERNAME} ${THIRDPARTYDIR}/${JULIA_DOWNLOAD_FILE} && \
chmod 744 ${THIRDPARTYDIR}/${JULIA_DOWNLOAD_FILE}
WORKDIR ${THIRDPARTYDIR}
RUN tar xf ${JULIA_DOWNLOAD_FILE} && \
rm -rf ${JULIA_DOWNLOAD_FILE}
WORKDIR ${THIRDPARTYDIR}
RUN sudo ln -s ${THIRDPARTYDIR}/julia-${JULIA_VERSION}/bin/julia /usr/bin/julia && \
sudo apt-get install -y wget
ENV CMAKE_PREFIX_PATH=${THIRDPARTYDIR}/julia-${JULIA_VERSION}:${CMAKE_PREFIX_PATH}
# instantiate all Julia Packages
RUN echo "intiantiating Julia Packages"
ARG WORKSPACEDIR
RUN mkdir -p ${WORKSPACEDIR}/src/
COPY --chown=${USERNAME}:${USERNAME} JuliaProjectManifests/ ${WORKSPACEDIR}/src/
RUN echo "WORKSPACEDIR = ${WORKSPACEDIR}"
#RUN sudo chown -R ${USERNAME}:${GROUPNAME} /tmp
WORKDIR ${WORKSPACEDIR}/src/
COPY --chown=${USERNAME}:${USERNAME} ./julia_instantiate.sh ${WORKSPACEDIR}/src/
RUN chmod +x ./julia_instantiate.sh
RUN source ./julia_instantiate.sh
#RUN sudo -E -s \
# && source ./julia_instantiate.sh \
# && exit
WORKDIR ${THIRDPARTYDIR}
RUN rm -rf temp
RUN sudo ldconfig
WORKDIR ${THIRDPARTYDIR}/..
CMD ["bash"]
为了更好的解释,ARG环境变量是在启动构建时在命令行定义的。该脚本如下所示:
DOCKER_BUILDKIT=1 docker build \
--no-cache=false \
--network=host \
--rm \
--build-arg PROGRAM_NAME=${PROGRAM_NAME} \
--build-arg WORKSPACEDIR=${WORKSPACEDIR} \
--build-arg USERNAME=${USER} \
--build-arg USERID=${UID} \
--build-arg THIRDPARTYDIR=${THIRDPARTYDIR} \
--build-arg DOCKER_UBUNTU_VERSION=${DOCKER_UBUNTU_VERSION} \
--build-arg JULIA_VERSION=${JULIA_VERSION} \
--build-arg JULIA_DOWNLOAD_FILE=${JULIA_DOWNLOAD_FILE} \
-f ${CDDS}/${DOCKER_BUILD_FOLDER}/${DOCKER_FILE_NAME} \
-t ${DOCKER_IMAGE} \
${DOCKER_BUILD_CONTEXT} \
2>&1 | tee ${CDDS}/${DOCKER_BUILD_FOLDER}/${DOCKER_BUILD_LOG}
当我尝试在Docker构建期间实例化项目时,我会收到以下错误信息:
#115 sha256:0c589043bbee81f6e25ae6da0177acfea129c478ba518ca117a39838dc68d6f1
#115 0.148 iKinQP/
#115 0.924 Installing known registries into `~/.julia`
#115 1.032 ERROR: SystemError: mktemp: No such file or directory
#115 1.703 Stacktrace:
#115 1.935 [1] systemerror(::Symbol, ::Int32; extrainfo::Nothing) at ./error.jl:168
#115 2.272 [2] #systemerror#48 at ./error.jl:167 [inlined]
#115 2.279 [3] systemerror at ./error.jl:167 [inlined]
#115 2.279 [4] #mktemp#18 at ./file.jl:589 [inlined]
#115 2.279 [5] mktemp at ./file.jl:587 [inlined] (repeats 2 times)
#115 2.298 [6] probe_platform_engines!(; verbose::Bool) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/PlatformEngines.jl:280
#115 2.299 [7] probe_platform_engines! at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/PlatformEngines.jl:181 [inlined]
#115 2.299 [8] pkg_server_registry_urls() at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/Types.jl:944
#115 2.349 [9] clone_default_registries(::Pkg.Types.Context; only_if_empty::Bool) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/Types.jl:868
#115 2.350 [10] clone_default_registries at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/Types.jl:862 [inlined]
#115 2.350 [11] find_registered!(::Pkg.Types.Context, ::Array{String,1}, ::Array{Base.UUID,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/Types.jl:1239
#115 2.355 [12] find_registered! at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/Types.jl:1200 [inlined]
#115 2.355 [13] check_registered(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/Operations.jl:1054
#115 2.356 [14] instantiate(::Pkg.Types.Context; manifest::Nothing, update_registry::Bool, verbose::Bool, platform::Pkg.BinaryPlatforms.Linux, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/API.jl:833
#115 2.403 [15] instantiate(::Pkg.Types.Context) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/API.jl:795
#115 2.404 [16] #instantiate#169 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/API.jl:791 [inlined]
#115 2.404 [17] instantiate() at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/API.jl:791
#115 2.404 [18] top-level scope at none:1
#115 DONE 2.5s
这里是 Dockerfile 运行 julia_instantiate.sh 的过程:
#!/bin/bash
cd ${WORKSPACEDIR}/src/;
folders=$(ls -d */)
echo ${folders}
for m in ${folders[@]}; do
cd "${WORKSPACEDIR}/src/${m}"
echo "instantiating: ${WORKSPACEDIR}/src/${m}"
julia --project=. -e "using Pkg; Pkg.instantiate(); using $(echo "${m}" | tr -d /); exit();"
done
- Docker镜像操作系统:Ubuntu 18.04
- 主机操作系统:WSL2 Ubuntu 20.04
- Docker版本:20.10.6,build 370c289
- Julia版本:1.5.3
Julia问题#33593似乎相关,但它没有给我提供解决方法。
如何解决这个问题?
编辑: 另一点是,如果我将 julia_instantiate.sh复制到Docker镜像中,但在构建过程中不执行它,我会得到一个包含该脚本和Julia包源文件的镜像。 如果我以交互方式手动运行 julia_instantiate.sh,它就可以正常运行。 我以与嵌入到docker镜像中的相同用户身份运行容器并附加。 因此,只有当由docker build运行脚本时,该过程才会失败。
注意:我已经将*julia_instantiate.sh更新为我一直在测试的最小值。 我还添加了有关Dockerfile的其他信息。
sudo
,实例化是否能够工作? 如果使用 root 用户,是否可以工作?感谢您发布整个 Dockerfile,如果到下周还没有解决,我会尝试查看它。 - Matěj RačinskýRUN source ./julia_instantiate.sh
之前添加了RUN sudo chown -R ${USERNAME}:${GROUPNAME} /tmp
,但没有任何变化。然后我删除了该命令,并改用以下命令:RUN sudo -E -s && source ./julia_instantiate.sh && exit
,结果还是出现了同样的错误...所以,即使以所有ENV可访问的root身份运行脚本也没有改变事情。 顺便说一句:这不是整个Dockerfile,而是最相关的部分... - Southern.Cross