共享相同步骤的GitHub Actions作业

Question

共享相同步骤的GitHub Actions作业

yamlgithub-actions

12

我有一个跨平台项目，需要在两个平台上构建：mac和linux(ubuntu)。

我的流水线包含3个作业：

准备带有构建项目所需工具的docker镜像。
在准备好的docker容器中使用ubuntu进行构建，依赖于步骤1。
在MacOS上构建，不需要任何内容。

Linux和MacOS的步骤肯定是相同的。但矩阵差异很大，并且Linux构建在容器内运行。

是否有一种方法可以在两个不同的作业之间共享步骤？

我尝试过YAML锚点，但GitHub不支持它们。

完整的工作流程

on:
  push:
    branches: [ main, support/1.2.x ]
  pull_request:
    branches: [ main, support/1.2.x ]

jobs:
  Docker-iroha-builder:
    runs-on: ubuntu-latest
    steps:
      -
        name: Checkout
        uses: actions/checkout@v2
      -
        name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v1
      -
        name: Cache Docker layers
        uses: actions/cache@v2
        with:
          path: /tmp/.buildx-cache
          key: ${{ runner.os }}-buildx-${{ github.sha }}
          restore-keys: |
            ${{ runner.os }}-buildx-
      -
        name: Login to DockerHub
        uses: docker/login-action@v1 
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}
      -
        name: Build and push
        uses: docker/build-push-action@v2
        with:
          file: docker/develop/Dockerfile.builder
          # context: .
          push: true
          tags: ${{ secrets.DOCKERHUB_ORG }}/iroha:builder
          cache-from: type=local,src=/tmp/.buildx-cache
          cache-to: type=local,dest=/tmp/.buildx-cache-new
      -
        # Temp fix
        # https://github.com/docker/build-push-action/issues/252
        # https://github.com/moby/buildkit/issues/1896
        name: Move cache
        run: |
          rm -rf /tmp/.buildx-cache
          mv /tmp/.buildx-cache-new /tmp/.buildx-cache


  build-iroha-ubuntu:
    needs: Docker-iroha-builder
    runs-on: ubuntu-latest
    container: ikyb/iroha:builder
    strategy:
      fail-fast: false
      matrix:
        cc: [ gcc-9, gcc-10, clang ]  ##todo g++-10
        USE_BURROW: [ -DUSE_BURROW=OFF ]
        debrel: [ Debug ] #,Release, RelWithDebInfo
    steps:
      - ## Takes 22 seconds with default github runner
        name: Homebrew
        run: brew install cmake ninja coreutils
        if: ${{ runner.os == 'MacOS' }}
      -
        name: Checkout
        uses: actions/checkout@v2
      -
        name: Cache vcpkg
        uses: actions/cache@v2
        with:
          path: |
            build-vcpkg
            build/vcpkg_installed
            $HOME/.cache/vcpkg
          key:          ${{ runner.os }}-vcpkg-${{ github.sha }}
          restore-keys: ${{ runner.os }}-vcpkg-
      - 
        name: Build Iroha vcpkg dependancies
        run: ./vcpkg/build_iroha_deps.sh $PWD/build-vcpkg
      - 
        name: CMake configure
        run: |
          export CC=${{ matrix.cc }} CXX=$(echo ${{ matrix.cc }} | sed -es,gcc,g++, -es,clang,clang++,)
          cmake -B build -DCMAKE_TOOLCHAIN_FILE=$PWD/build-vcpkg/scripts/buildsystems/vcpkg.cmake \
             ${{ matrix.USE_BURROW }} -GNinja #-DCMAKE_VERBOSE_MAKEFILE=ON
      -
        name: CMake build
        run: cmake --build build --config ${{ matrix.debrel }}

  build-iroha-macos:
    runs-on: macos-latest
    strategy:
      fail-fast: false
      matrix:
        USE_BURROW: [ -DUSE_BURROW=OFF ]
        debrel: [ Debug,Release ]
    steps:
      - ## Takes 22 seconds with default github runner
        name: Homebrew
        run: brew install cmake ninja coreutils
        if: ${{ runner.os == 'MacOS' }}
      -
        name: Checkout
        uses: actions/checkout@v2
      -
        name: Cache vcpkg
        uses: actions/cache@v2
        with:
          path: |
            build-vcpkg
            build/vcpkg_installed
            $HOME/.cache/vcpkg
          key:          ${{ runner.os }}-vcpkg-${{ github.sha }}
          restore-keys: ${{ runner.os }}-vcpkg-
      - 
        name: Build Iroha vcpkg dependancies
        run: ./vcpkg/build_iroha_deps.sh $PWD/build-vcpkg
      - 
        name: CMake configure
        run: |
          export CC=${{ matrix.cc }} CXX=$(echo ${{ matrix.cc }} | sed -es,gcc,g++, -es,clang,clang++,)
          cmake -B build -DCMAKE_TOOLCHAIN_FILE=$PWD/build-vcpkg/scripts/buildsystems/vcpkg.cmake \
             ${{ matrix.USE_BURROW }} -GNinja #-DCMAKE_VERBOSE_MAKEFILE=ON
      -
        name: CMake build
        run: cmake --build build --config ${{ matrix.debrel }}

- kyb

2

这是许多人问过的问题，你可以在这里找到一些相关参考资料(https://github.community/t/reusing-sharing-inheriting-steps-between-jobs-declarations/16851/16)和这里(https://github.community/t/sharing-steps-between-jobs-and-jobs-between-workflows/17751/4)。目前的解决方案似乎是使用`composite run steps actions

(https://docs.github.com/en/actions/creating-actions/about-actions#composite-run-steps-actions)，另一个解决方案是使用

YAML anchors`(https://github.community/t/reusing-sharing-inheriting-steps-between-jobs-declarations/16851/21)。 - GuiFalourd

我发现了下面这些材料，试图解决这个问题：1. https://github.community/t/support-for-yaml-anchors/16128/60 2. https://github.com/mithro/actions-includes 3. https://github.com/allejo/gha-workflows - kyb

2个回答

0

虽然 github actions 不直接支持 YAML 锚点，但可以通过将 YAML 转换为 JSON，然后再转换回 YAML 来扩展这些锚点。我在这里执行了这个操作（.github/workflows/Makefile）: https://github.com/agda/agda/blob/557681d04aae2100ccde2e045a8afcf30528c3a5/.github/workflows/Makefile

srcpath=../../src/github/workflows
sources=$(wildcard $(srcpath)/*.yml $(srcpath)/*.yaml)
targets=$(sort $(notdir $(sources)))

all : $(targets)

# Normalize YAML files by going via JSON.
# This expands anchors which are not understood by github workflows.

% : $(srcpath)/% 
    yaml2json $< | json2yaml - > $@

一个带有锚点的工作流文件示例在这里：https://github.com/agda/agda/blob/557681d04aae2100ccde2e045a8afcf30528c3a5/src/github/workflows/test.yml

jobs:
  build:
    runs-on: &runs_on ubuntu-22.04
    steps:
    - &checkout
      uses: actions/checkout@v3

    - &haskell_setup
      uses: haskell/actions/setup@v2
      with:
        ghc-version: ${{ env.GHC_VER }}
...
  test:
    needs: build
    runs-on: *runs_on

    steps:
    - *checkout
    - *haskell_setup
...

- Andreas Abel

网页内容由stack overflow 提供, 点击上面的

可以查看英文原文，
原文链接

- kyb · Accepted Answer

简述

我使用shell工具yq解决了我的问题。

yq eval 'explode(.)' file.yml

这个仓库包含示例用法和详细描述，可能对于轻松入门有帮助。它是从这个答案中制作的。https://github.com/kuvaldini/make-workflows.sh请注意操作选项卡。

长回答

GitHub Workflow描述在YAML中不支持锚点。有几个解决方法 => 无论如何都需要从源代码构建 - 编辑工作流yaml。因此，我建议另一个基于YAML工具yq的make-workflows.sh。

用法

将您的工作流程移动到.github / * .src.yml
将make-workflows.sh放置在目录.github / 中
（可选）将pre-commit.sh复制或链接到.git / hooks / pre-commit 像ln -s ../../.github/pre-commit.sh .git/hooks/pre-commit这样

文件make-workflows.sh

#!/usr/bin/env bash
set -euo pipefail

## The script expands '*.src.yml' from $1(default: script's directory) 
## to $2 (default:subdirectory 'workflows') with corresponding name '*.yml'
## Main goal is to dereference YAML anchors.
## Deals only with Git cached/indexed files
## Set -x to debug

script_dir=$(dirname $(realpath "$0"))
dir_from=${1:-${script_dir}}
dir_to=${2:-workflows}
cd $dir_from

edited=
for f in $(git status -s -- \*.src.yml | sed 's,^.. ,,') ;do
    readonly out=$(echo $f | sed s,.src.yml\$,.yml,)
    readonly wout=$dir_to/$out
    readonly tempout=$(mktemp)
    trap "rm -f $tempout" EXIT
    echo >>$tempout "## DO NOT EDIT"
    echo >>$tempout "## Generated from $f with $(basename $0)"
    echo >>$tempout ""
    yq eval 'explode(.)' $f >>$tempout
    if ! diff -q $wout $tempout &>/dev/null ;then
        mv $tempout $wout
        edited+="'$out' "
    fi
done

if [[ -n "$edited" ]] 
then echo >&2 "make-workflows: these files were edited: $edited"
else echo >&2 "make-workflows: everything is up to date"
fi

文件 pre-commit.sh

#!/usr/bin/env bash
set -euo pipefail

gitroot=$(git rev-parse --show-toplevel)

cd $gitroot
./.github/make-workflows.sh
git add .github/workflows

链接

https://github.com/kuvaldini/make-workflows.sh：具有详细说明的可直接使用的解决方案。
共享相同步骤的不同GitHub Actions作业。
https://github.community/t/support-for-yaml-anchors/16128/60：YAML锚点支持。
https://github.com/mithro/actions-includes：包含操作的GitHub Action。
https://github.com/allejo/gha-workflows：gha-workflows库，其中包括了各种类型的GitHub Actions工作流示例。