有没有办法在gcp中自动构建kubeflow管道

问题描述

这是我的cloudbuild.yaml文件

- name: 'gcr.io/cloud-builders/docker'
  args: ['build','-t','gcr.io/$PROJECT_ID/$_TRAINER_IMAGE_NAME:$TAG_NAME','.']
  dir: $_PIPELINE_FOLDER/trainer_image
  
# Build the base image for lightweight components
- name: 'gcr.io/cloud-builders/docker'
  args: ['build','gcr.io/$PROJECT_ID/$_BASE_IMAGE_NAME:$TAG_NAME','.']
  dir: $_PIPELINE_FOLDER/base_image

# Compile the pipeline
- name: 'gcr.io/$PROJECT_ID/kfp-cli'
  args:
  - '-c'
  - |
    dsl-compile --py $_PIPELINE_DSL --output $_PIPELINE_PACKAGE
  env:
  - 'BASE_IMAGE=gcr.io/$PROJECT_ID/$_BASE_IMAGE_NAME:$TAG_NAME'
  - 'TRAINER_IMAGE=gcr.io/$PROJECT_ID/$_TRAINER_IMAGE_NAME:$TAG_NAME'
  - 'RUNTIME_VERSION=$_RUNTIME_VERSION'
  - 'PYTHON_VERSION=$_PYTHON_VERSION'
  - 'COMPONENT_URL_SEARCH_PREFIX=$_COMPONENT_URL_SEARCH_PREFIX'
  - 'USE_KFP_SA=$_USE_KFP_SA'
  dir: $_PIPELINE_FOLDER/pipeline
  
 # Upload the pipeline
- name: 'gcr.io/$PROJECT_ID/kfp-cli'
  args:
  - '-c'
  - |
    kfp --endpoint $_ENDPOINT pipeline upload -p ${_PIPELINE_NAME}_$TAG_NAME $_PIPELINE_PACKAGE
  dir: $_PIPELINE_FOLDER/pipeline


# Push the images to Container Registry 
images: ['gcr.io/$PROJECT_ID/$_TRAINER_IMAGE_NAME:$TAG_NAME','gcr.io/$PROJECT_ID/$_BASE_IMAGE_NAME:$TAG_NAME']

所以基本上我想做的是写一个包含所有变量的bash脚本，并且每当我进行更改时，云构建应该自动触发。

解决方法

您可以在bash的管道中添加另一个步骤。像下面这样。

steps:
  - name: 'bash'
    args: ['echo','I am bash']

,

您似乎将编译和上传分为两个步骤。我不确定100％是否可以在CloudBuild中使用（我不确定文件是否已共享）。

您可能需要合并这些步骤。