问题描述
我需要为 AWS glue 编写 AWS CloudFormation 模板的代码,我正在为多个 glue 作业创建单个模板。 我为第二份工作添加了谓词条件。不幸的是,虽然谓词接受多个条件,但它只接受前一个作业的一种状态,就像前一个作业状态成功或失败或停止一样,下一个作业仍应运行。但是我需要它在多个状态下运行,因为即使前一个作业具有下一个作业应该添加的任何成功/失败/停止状态。
如果我给出多个条件,它会报错说存在重复的作业,因为我在所有条件下都给出了相同的作业名称
我在模板中写了以下代码片段
'''
Resources:
WF:
Type: AWS::glue::Workflow
Properties:
Description: workflow for Parallel Jobs
Name: Sequence_Job_Triggers
SequenceTrigger:
Type: AWS::glue::Trigger
Properties:
Name: Job_Trigger_1
Type: ON_DEMAND
Description: Description
WorkflowName: !Ref WF
Actions:
- JobName: job1
SequenceTrigger:
Type: AWS::glue::Trigger
Properties:
Name: Job_Trigger_2
Type: CONDITIONAL
Description: Description
WorkflowName: !Ref WF
Predicate:
Conditions:
- LogicalOperator: EQUALS
JobName: job1
State: SUCCEEDED
Actions:
- JobName: job2
'''
解决方法
你不能只创建两个触发器,至少这对我有用:
SucceededSequenceTrigger:
Type: AWS::Glue::Trigger
Properties:
Name: Succeeded_Job_Trigger_2
Type: CONDITIONAL
Description: Description
WorkflowName: !Ref WF
Predicate:
Conditions:
- LogicalOperator: EQUALS
JobName: job1
State: SUCCEEDED
Actions:
- JobName: job2
FailedSequenceTrigger:
Type: AWS::Glue::Trigger
Properties:
Name: Failed_Job_Trigger_2
Type: CONDITIONAL
Description: Description
WorkflowName: !Ref WF
Predicate:
Conditions:
- LogicalOperator: EQUALS
JobName: job1
State: FAILED
Actions:
- JobName: job2