如何使github动作矩阵元素有条件

编程问答 2022-06-02

问题描述

我有一个使用 'strategy' = 'matrix' 和要构建的特定配置列表的工作流。

Link to Workflow YAML (also provided below)

#
# build-N-test-v2.1-Dev and build-N-test-v2.1-Release are neary
# identical,but a few tests are commented out (to not needlessly stress CI system)
# for v2.1-Dev builds
#
# NOTE: I've tried many tricks - none which seem to work - to get this working on one file with one
# workflow and tests
#     https://github.community/t/what-is-the-correct-if-condition-syntax-for-checking-matrix-os-version/16221
#     https://github.community/t/how-to-conditionally-include-exclude-items-in-matrix-eg-based-on-branch/16853
#
# but none seem to work
#

name: build-N-test-v2.1-Dev

on:
  push:
    branches:
      - v2.1-Dev
      #- v2.1-Release
  workflow_dispatch:
    inputs:
      ignored:
        description: "ignored"
        required: false
        default: ""

## NB: JOBS section IDENTICAL between v2.1-Dev and 2.1-Release files EXCEPT that on v2.1-Dev file
## comment out all entries marked with includeInDevBranchBuilds: false
jobs:
  build-n-test-Linux:
    runs-on: ${{ matrix.runs_on }}
    strategy:
      #
      # Configuration notes
      #   o --debug-symbols false to reduce build disk size (and we aren't debugging anyhow) in many debug configurations
      #
      matrix:
        include:
          # ## SADLY: Container operations are only supported on Linux runners
          # - displayTargetName: windows-DBG
          #   os: windows
          #   compiler: g++-8
          #   runs_on: windows-latest
          #   container_image: sophistsolutionsinc/stroika-buildvm-windows-cygwin-vs2k19
          #   cpp_version: c++17
          #   config_name: Debug
          #   extra_config_args: --apply-default-debug-flags --trace2file enable

          ## centos 8
          - displayTargetName: centos-8
            os: unix
            compiler: g++
            runs_on: ubuntu-latest
            container_image: sophistsolutionsinc/stroika-buildvm-centos-8-small
            cpp_version: c++17
            config_name: Release
            extra_config_args: --apply-default-release-flags --trace2file enable
            includeInDevBranchBuilds: true

          ## ubuntu 18.04
          - displayTargetName: ubuntu-18.04-g++-8 (Debug)
            os: unix
            compiler: g++-8
            runs_on: ubuntu-latest
            container_image: sophistsolutionsinc/stroika-buildvm-ubuntu1804-regression-tests
            cpp_version: c++17
            config_name: Debug
            extra_config_args: --apply-default-debug-flags --trace2file enable --debug-symbols false
            includeInDevBranchBuilds: true

          - displayTargetName: ubuntu-18.04-cross-compile-raspberrypi (Debug)
            os: unix
            compiler: g++-8
            runs_on: ubuntu-latest
            container_image: sophistsolutionsinc/stroika-buildvm-ubuntu1804-regression-tests
            cpp_version: c++17
            config_name: Debug
            extra_config_args: --apply-default-release-flags --trace2file enable --compiler-driver arm-linux-gnueabihf-g++-8 --cross-compiling true
            includeInDevBranchBuilds: true

          # ubuntu 20.04
          # - displayTargetName: ubuntu-20.04-g++-9 (Debug)
          #   os: unix
          #   compiler: g++-9
          #   runs_on: ubuntu-latest
          #   container_image: sophistsolutionsinc/stroika-buildvm-ubuntu2004-regression-tests
          #   cpp_version: c++17
          #   config_name: Debug
          #   extra_config_args: --apply-default-debug-flags --trace2file enable --debug-symbols false
          #   includeInDevBranchBuilds: false

          # - displayTargetName: ubuntu-20.04-g++-10 (Debug)
          #   os: unix
          #   compiler: g++-10
          #   runs_on: ubuntu-latest
          #   container_image: sophistsolutionsinc/stroika-buildvm-ubuntu2004-regression-tests
          #   cpp_version: c++17
          #   config_name: Debug
          #   extra_config_args: --apply-default-debug-flags --trace2file enable --debug-symbols false
          #   includeInDevBranchBuilds: false

          - displayTargetName: ubuntu-20.04-g++-10
            os: unix
            compiler: g++-10
            runs_on: ubuntu-latest
            container_image: sophistsolutionsinc/stroika-buildvm-ubuntu2004-regression-tests
            cpp_version: c++17
            config_name: Release
            extra_config_args: --apply-default-release-flags --trace2file enable
            includeInDevBranchBuilds: true

          # - displayTargetName: ubuntu-20.04-g++-10-c++2a
          #   os: unix
          #   compiler: g++-10
          #   runs_on: ubuntu-latest
          #   container_image: sophistsolutionsinc/stroika-buildvm-ubuntu2004-regression-tests
          #   cpp_version: c++2a
          #   config_name: Release
          #   extra_config_args: --apply-default-release-flags --trace2file enable
          #   includeInDevBranchBuilds: false

          # - displayTargetName: ubuntu-20.04-clang++-10
          #   os: unix
          #   compiler: clang++-10
          #   runs_on: ubuntu-latest
          #   container_image: sophistsolutionsinc/stroika-buildvm-ubuntu2004-regression-tests
          #   cpp_version: c++17
          #   config_name: Release
          #   extra_config_args: --apply-default-release-flags --trace2file enable
          #   includeInDevBranchBuilds: false

    ### ATTEMPT TO COMPRESS 2 workflow files into one,but so far not working
    ### SEE
    ### https://stackoverflow.com/questions/65384420/how-to-make-a-github-action-matrix-element-conditional/65385385#65385385
    ###
    #if: github.ref == 'refs/heads/v2.1-Release' || matrix.includeInDevBranchBuilds
    env:
      # vm has 2 virtual CPUs,but 8GB ram,so jobs=5 (empirical),and QUICK_BUILD avoids some internal testing
      MAKEFLAGS: "--jobs=3 QUICK_BUILD=1"
    container: ${{ matrix.container_image }}
    steps:
      - uses: actions/checkout@v2
      - name: Build System Info
        if: ${{ matrix.os=='unix' }}
        run: |
          lsb_release -d 2>/dev/null || true
          echo "CWD=" `pwd`
          echo "nproc=" `nproc`
          grep "model name" /proc/cpuinfo | head -1
          grep processor /proc/cpuinfo | wc -l
          grep MemTotal /proc/meminfo
          df -h
      - name: Build System Info (Windows)
        if: ${{ matrix.os=='windows' }}
        run: |
          echo "CWD=" `pwd`
          df -h
      - name: Configure ${{ matrix.config_name }}
        run: |
          ./configure ${{ matrix.config_name }} --compiler-driver ${{ matrix.compiler }} ${{ matrix.extra_config_args }} --cppstd-version ${{ matrix.cpp_version }}
          cat ConfigurationFiles/${{ matrix.config_name }}.xml
      # Break out third-party-components to do clean so we dont run out of disk space,and break out TPC AND library
      # to show the summary time for each part
      - name: Make third-party-components
        run: |
          make third-party-components
          make clean
      - name: Make libraries
        run: make libraries
      - name: Make all
        run: make all
      - name: Run Tests
        run: make run-tests
      - name: Archive Samples Results
        uses: actions/upload-artifact@v2
        with:
          name: Sample apps (${{ matrix.displayTargetName }})
          path: |
            Builds/${{ matrix.config_name }}/Samples-*
      - name: Archive Log Results
        uses: actions/upload-artifact@v2
        with:
          name: Log Data (${{ matrix.displayTargetName }})
          path: |
            Builds/${{ matrix.config_name }}/PerformanceDump.txt
            /tmp/Trace*.txt

  build-n-test-MacOS:
    runs-on: ${{ matrix.runs_on }}
    strategy:
      matrix:
        # Add to extra_config_args for build speed: --Xerces no --OpenSSL no --lzma no --boost no
        include:
          - displayTargetName: MacOS-Debug
            os: macos-10.15
            runs_on: macos-10.15
            config_name: Debug
            extra_config_args: --apply-default-debug-flags --trace2file enable
            includeInDevBranchBuilds: true
          # - displayTargetName: MacOS
          #   os: macos-10.15
          #   runs_on: macos-10.15
          #   config_name: Release
          #   extra_config_args: --apply-default-release-flags --trace2file enable
          #   includeInDevBranchBuilds: false
    env:
      # vm has 2 virtual CPUs,and QUICK_BUILD avoids some internal testing
      MAKEFLAGS: "--jobs=3 QUICK_BUILD=1"
    steps:
      - uses: actions/checkout@v2
      - name: Build System Info
        run: |
          echo "CWD: `pwd`"
          df -h
          system_profiler SPSoftwareDataType
          sw_vers
      # If we had docker ability,most of these would be built into a docker file
      - name: Install Basic Build requirements
        run: |
          brew install gnu-sed
          brew install p7zip
          brew install automake
          make install-realpath
      - name: Configure
        run: |
          ./configure ${{ matrix.config_name }} ${{ matrix.extra_config_args }}
          cat ConfigurationFiles/${{ matrix.config_name }}.xml
      - name: Build third-party-components
        run: |
          make third-party-components
          make clean
      - name: Build Library
        run: |
          make libraries
      - name: Build All
        run: |
          make all
      - name: Run-Tests
        run: |
          make run-tests
      - name: Workaround GitHub-Actions-MacOS Issue with env.TMPDIR
        run: |
          mkdir /tmp/LOGS-ARCHIVE
          cp $TMPDIR/Trace*.txt /tmp/LOGS-ARCHIVE
      - name: DEBUG Workaround GitHub-Actions-MacOS Issue with env.TMPDIR
        run: |
          echo "TMPDIR=$TMPDIR"
          echo "TMPDIR using ENV.TMPDIR=${{ env.TMPDIR }}"
          # Just the echo line above shows empty,and then the ls line causes exit 1/failure
          #ls -l ${{ env.TMPDIR }}/Trace*.txt
          #if this gets fixed,then lose Workaround GitHub-Actions-MacOS,and directly reference ${{ env.TMPDIR }}/Trace*.txt in Archive Log Results
      - name: Build System Info
        run: |
          df -h
      - name: Archive Log Results
        uses: actions/upload-artifact@v2
        with:
          name: Log Results (${{ matrix.displayTargetName }})
          path: |
            Builds/${{ matrix.config_name }}/PerformanceDump.txt
            /tmp/LOGS-ARCHIVE
            #${{ env.TMPDIR }}/Trace*.txt
      - name: Archive Sample Results
        uses: actions/upload-artifact@v2
        with:
          name: Samples (${{ matrix.displayTargetName }})
          path: |
            Builds/${{ matrix.config_name }}/Samples-*

  build-n-test-Windows:
    runs-on: ${{ matrix.runs_on }}
    strategy:
      matrix:
        # Add to extra_config_args for build speed: --Xerces no --OpenSSL no --lzma no --boost no
        include:
          - displayTargetName: windows-x86-Debug
            os: windows
            runs_on: windows-latest
            container_image: sophistsolutionsinc/stroika-buildvm-windows-cygwin-vs2k19
            config_name: Debug
            extra_config_args: --arch x86 --apply-default-debug-flags --trace2file enable
            includeInDevBranchBuilds: true
          # - displayTargetName: windows-x86-Release
          #   os: windows
          #   runs_on: windows-latest
          #   container_image: sophistsolutionsinc/stroika-buildvm-windows-cygwin-vs2k19
          #   config_name: Release
          #   extra_config_args: --arch x86 --apply-default-release-flags --trace2file enable
          #   includeInDevBranchBuilds: false
          # - displayTargetName: windows-x86_64-Debug
          #   os: windows
          #   runs_on: windows-latest
          #   container_image: sophistsolutionsinc/stroika-buildvm-windows-cygwin-vs2k19
          #   config_name: Debug
          #   extra_config_args: --arch x86_64 --apply-default-debug-flags --trace2file enable
          #   includeInDevBranchBuilds: false
          # - displayTargetName: windows-x86_64-Release
          #   os: windows
          #   runs_on: windows-latest
          #   container_image: sophistsolutionsinc/stroika-buildvm-windows-cygwin-vs2k19
          #   config_name: Release
          #   extra_config_args: --arch x86 --apply-default-release-flags --trace2file enable
          #   includeInDevBranchBuilds: false
    env:
      # vm has 2 virtual CPUs,and QUICK_BUILD avoids some internal testing
      MAKEFLAGS: "--jobs=3 QUICK_BUILD=1"
      ARTIFACTS_DIR: "c:/Artifacts/"
    steps:
      - uses: actions/checkout@v2
      # https://stackoverflow.com/questions/58033366/how-to-get-current-branch-within-github-actions
      - name: Extract branch name
        shell: bash
        run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"
        id: extract_branch
      - name: Build System Info
        shell: "bash"
        run: |
          echo "CWD: `pwd`"
          df -h
          systeminfo
          echo NUMBER_OF_PROCESSORS=$NUMBER_OF_PROCESSORS
      - name: docker pull ${{ matrix.container_image }}
        run: docker pull ${{ matrix.container_image }}
      - name: Start docker build environment
        run: |
          docker run --tty --memory 5G --cpus 2 --storage-opt 'size=50GB' --detach --name buildContainer ${{ matrix.container_image }}
      - name: Print Info about docker system
        shell: "bash"
        run: |
          docker ps -a
          docker exec buildContainer systeminfo
          docker exec buildContainer df -h
      - name: Git Checkout
        shell: "bash"
        run: |
          docker exec buildContainer sh -c "git clone https://github.com/SophistSolutions/Stroika.git && cd Stroika && git checkout ${{ steps.extract_branch.outputs.branch }}"
      - name: Configure
        shell: "bash"
        run: |
          docker exec --workdir c:/Stroika buildContainer sh -c "./configure ${{ matrix.config_name }} ${{ matrix.extra_config_args }}"
          docker exec --workdir c:/Stroika buildContainer cat ConfigurationFiles/${{ matrix.config_name }}.xml
      - name: Build
        shell: "bash"
        run: |
          docker exec --workdir c:/Stroika --env MAKEFLAGS="$MAKEFLAGS" buildContainer make all
      - name: Run-Tests
        shell: "bash"
        run: |
          docker exec --workdir c:/Stroika --env MAKEFLAGS="$MAKEFLAGS" buildContainer make run-tests
      - name: Build System Info
        shell: "bash"
        run: |
          df -h
          docker exec buildContainer df -h
      - name: Copy Build Artifacts
        shell: "bash"
        # due to flaws in docker (windows must stop) - and cp no wildcards
        run: |
          docker exec --workdir c:/Stroika buildContainer bash -c 'mkdir TRACE_LOGS && cp $TEMP/Trace*.txt TRACE_LOGS/'
          docker stop buildContainer
          docker cp buildContainer:Stroika/Builds/${{ matrix.config_name }}/ $ARTIFACTS_DIR 2> /dev/null
          docker cp buildContainer:Stroika/TRACE_LOGS $ARTIFACTS_DIR 2> /dev/null
          rm -rf $ARTIFACTS_DIR/{ThirdPartyComponents,Tests,*.lib}
      - name: Archive Log Results
        uses: actions/upload-artifact@v2
        with:
          name: Log Results (${{ matrix.displayTargetName }})
          path: |
            ${{ env.ARTIFACTS_DIR }}PerformanceDump.txt
            ${{ env.ARTIFACTS_DIR }}TRACE_LOGS
      - name: Archive Sample Results
        uses: actions/upload-artifact@v2
        with:
          name: Samples (${{ matrix.displayTargetName }})
          path: |
            ${{ env.ARTIFACTS_DIR }}Samples-*

但我只想在分支 = v2.1-Release 时构建一些配置。也就是说 - 在大多数情况下 - 只需构建一两个配置，但在发布时构建更多配置。

我通过克隆脚本（工作流程）并重命名一些内容并注释掉这些内容来完成此操作。但如果该机制与矩阵元素一起工作，那就太好了。

我意识到可以将“if”功能添加到每个步骤中，但这会创建大量禁用步骤的作业。我想要的是，根本不为每个具有 if 部分评估为 false 的矩阵元素启动这些工作。

解决方法

添加另一个答案，因为这个答案使用 custom Actions 而不是内联 shell 脚本。免责声明：我是其中一个 Actions 的维护者（感谢您的启发）。

通过使用两个额外的操作，

您可以实现与我的其他答案中的 custom shell script 类似的效果，但可以说是更简洁的方式：

.github/workflows/sample.yml

name: Configurable Build Matrix

on: push
jobs:
  matrix_prep:
    runs-on: ubuntu-latest
    outputs:
      matrix: ${{ steps.set-matrix.outputs.matrix }}
    steps:
      - uses: actions/checkout@v2
      - uses: nelonoel/branch-name@v1.0.1
      - id: set-matrix
        uses: JoshuaTheMiller/conditional-build-matrix        
        with:          
          filter: '[?runOn==`${{ env.BRANCH_NAME }}` || runOn==`always`]'
  build-n-test:
    needs: matrix_prep
    runs-on: ${{ matrix.runs_on }}
    strategy:
      matrix: ${{fromJson(needs.matrix_prep.outputs.matrix)}}
    steps:
    - run: echo "Hello ${{ matrix.someValue }}"

.github/workflows/matrix_includes.json

[
    {
        "runs_on":"ubuntu-16.04","someValue":"Foo","runOn":"always"
    },{
        "runs_on":"ubuntu-18.04","someValue":"Bar","runOn":"v2.1-Release"
    },{
        "runs_on":"ubuntu-20.04","someValue":"Hello again","runOn":"v2.1-Release"
    }
]

这两个文件都可以在 repo of the Action

注意事项

  • filter 输入也使用不同的过滤语法：它使用 JMESPATH 代替 jq。我发现 JMESPATH 有很好的文档记录和支持。要详细了解语法的工作原理，您可以访问他们的 interactive examples page。
  • 默认情况下，conditional-build-matrix 操作会在 matrix_includes.json 文件夹下查找名为 .github/workflows/ 的文件（与另一个答案中包含的文件相同）。此位置可以自定义，为 documented on the Action's page。
building-github-actionsgithubgithub-actions

相关问答

导入项目后报错问题
依赖报错 idea导入项目后依赖报错，解决方案：https://blog....
idea不能识别yaml文件
使用mybatis plus常见错误
错误1：代码生成器依赖和mybatis依赖冲突 启动项目时报错如下...
gradle常见问题与错误
错误1：gradle项目控制台输出为乱码 # 解决方案：https://bl...
Mybatis Plus传入参数0不起作用
错误还原：在查询的过程中，传入的workType为0时，该条件不起...
linux中make编译源码包失败
报错如下，gcc版本太低 ^ server.c:5346:31: 错误：‘struct...