如何在Argo Workflow DAG中为每个步骤传递不同的参数值

问题描述

我有如下所述的Argo DAG示例。如下所示，我已经对每个任务的参数值进行了硬编码。每个任务使用不同的参数值。我知道Argo提供了一种以JSON或YAML格式提交参数文件以动态传递参数值的方法。但是，我想知道是否有一种方法可以通过parameter file为以下DAG中的每个任务传递不同的值。

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: dag-diamond-custom-
spec:
  entrypoint: diamond
  templates:
  - name: echo
    retryStrategy:
      limit: 3
    inputs:
      parameters:
      - name: message
      - name: task
    container:
      image: index_ruby:latest
      imagePullPolicy: Never
      command: [ruby,"index.rb","-s","{{inputs.parameters.task}}","-r","{{inputs.parameters.message}}"]
  - name: diamond
    dag:
      tasks:
      - name: A
        template: echo
        arguments:
          parameters: 
            - name: message
              value: |
                {"key": "valueA"}
            - name: task
              value: A
      - name: B
        dependencies: [A]
        template: echo
        arguments:
          parameters: 
            - name: message
              value: |
                {"key": "valueB"}
            - name: task
              value: B
      - name: C
        dependencies: [A]
        template: echo
        arguments:
          parameters: 
            - name: message
              value: |
                {"key": "valueC"}
            - name: task
              value: C
      - name: D
        dependencies: [B,C]
        template: echo
        arguments:
          parameters: 
            - name: message
              value: |
                {"key": "valueD"}
            - name: task
              value: D

解决方法

Argo Workflow规范包括arguments，其中可以定义多个global parameters。

您可以为每个任务定义一个参数，然后通过模板访问它。

（我对echo容器进行了一些修改，因为我在使用Ruby图像时遇到了麻烦。）

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: dag-diamond-custom-
spec:
  entrypoint: diamond
  arguments:
    parameters:
    - name: valueA
    - name: valueB
    - name: valueC
    - name: valueD
  templates:
  - name: echo
    retryStrategy:
      limit: 3
    inputs:
      parameters:
      - name: message
      - name: task
    container:
      image: docker/whalesay:latest
      command: [echo]
      args: ["{{inputs.parameters.task}}","{{inputs.parameters.message}}"]
  - name: diamond
    dag:
      tasks:
      - name: A
        template: echo
        arguments:
          parameters: 
            - name: message
              value: |
                {"key": "{{workflow.parameters.valueA}}"}
            - name: task
              value: A
      - name: B
        dependencies: [A]
        template: echo
        arguments:
          parameters: 
            - name: message
              value: |
                {"key": "{{workflow.parameters.valueB}}"}
            - name: task
              value: B
      - name: C
        dependencies: [A]
        template: echo
        arguments:
          parameters: 
            - name: message
              value: |
                {"key": "{{workflow.parameters.valueC}}"}
            - name: task
              value: C
      - name: D
        dependencies: [B,C]
        template: echo
        arguments:
          parameters: 
            - name: message
              value: |
                {"key": "{{workflow.parameters.valueD}}"}
            - name: task
              value: D

如前所述，您可以创建一个参数的JSON文件...

{
  "valueA": 1,"valueB": 2,"valueC": 3,"valueD": 4
}

...然后通过argo submit从文件中传递参数。

argo submit workflow.yaml -f params.json --watch