Spring Boot：Spring Batch 作业不会自动启动

编程问答 2022-04-25

问题描述

TL;DR

Spring 启动批处理作业不会在 Windows 或 Spring Cloud Data Flow 中作为 uber jar 启动；但是，相同的代码在 Eclipse 中启动时运行良好。

我使用 Spring Boot 创建了一个简单的 Spring 批处理作业。我想在 Kubernetes 上运行这项工作。我遵循了 this 指南中提到的关于如何在 SCDF (Spring Cloud Data Flow) 中启动 Spring 批处理作业的步骤。当我注册这个应用程序并创建指南中提到的任务时，在 Kubernetes 中创建了一个新的 pod，应用程序运行良好；但是，批处理作业永远不会启动。这可能是什么原因？

我的 Spring Boot 批量应用代码：

主类

@SpringBootApplication
public class BatchApplication {

    public static void main(String[] args) {
        SpringApplication.run(BatchApplication.class,args);
    }
}

作业配置

@Configuration
@EnableTask
@EnableBatchProcessing
@Profile("master")
public class BatchJob {

    @Autowired
    private JobBuilderFactory jobBuilderFactory;

    @Autowired
    private StepBuilderFactory stepBuilderFactory;

    public static final String JOB_NAME = "myJob";
    
    private static final Logger LOGGER = Logger.getLogger(BatchJob .class);

    @Bean(name = JOB_NAME )
    public Job job() {
        LOGGER.info("Creating job bean...");
        return jobBuilderFactory.get(JOB_NAME).incrementer(new RunIdIncrementer())
                .start(helloWorldStep()).build();
    }

    @Bean
    public Step helloWorldStep() {
        LOGGER.info("Creating Step bean...");
        return stepBuilderFactory.get("helloWorldStep").tasklet(helloWorldtasklet()).build();
    }

    @Bean
    public HelloWorldtasklet helloWorldtasklet() {
        LOGGER.info("Creating tasklet...");
        return new HelloWorldtasklet();
    }
}

HelloWorldtasklet

public class HelloWorldtasklet implements tasklet {

    private static final Logger LOGGER = Logger.getLogger(HelloWorldtasklet.class);

    @Override
    public RepeatStatus execute(StepContribution contribution,ChunkContext chunkContext) throws Exception {

        LOGGER.info("Hello World");
        System.out.println("Hello World Again");
        return RepeatStatus.FINISHED;

    }

}

application.properties

spring.application.name=my-batch-app
spring.datasource.url=jdbc:h2:mem:dataflow
spring.datasource.driverClassName=org.h2.Driver
spring.datasource.username=sa
spring.datasource.password=
spring.jpa.database-platform=org.hibernate.dialect.H2Dialect
spring.cloud.task.closecontext_enabled=true

pom.xml 依赖

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <artifactId>my-project-parent</artifactId>
        <groupId>com.myapp</groupId>
        <version>MAR21_1.0</version>
    </parent>

    <groupId>com.myapp</groupId>
    <artifactId>my-batch-app</artifactId>

    <properties>
        <java.version>1.8</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-batch</artifactId>
            <version>2.3.11.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
            <version>2.3.11.RELEASE</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <version>2.3.11.RELEASE</version>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>org.junit.vintage</groupId>
                    <artifactId>junit-vintage-engine</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
         <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>1.2.17-cloudera1</version>
        </dependency>
        <dependency>
            <groupId>com.h2database</groupId>
            <artifactId>h2</artifactId>
            <version>1.4.199</version>
            <!--<scope>runtime</scope>-->
        </dependency>
        
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <version>2.1.0.RELEASE</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>repackage</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

</project>

其中 my-project-parent 是我所有项目的父模块，仅包含 jar 版本的定义。但是，到目前为止，我并没有真正在 my-batch-app 中使用此 pom 中定义的任何这些版本。

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.myapp</groupId>
    <artifactId>my-project-parent</artifactId>
    <version>MAR21_1.0</version>
    <packaging>pom</packaging>
    <name>my-project-parent</name>

    <properties>
        <java.version>1.8</java.version>            
        <spring.kafka.version>2.5.5.RELEASE</spring.kafka.version>          <spring.cloud.starter.task.version>2.2.4.RELEASE</spring.cloud.starter.task.version>            <spring.boot.starter.data.jpa.version>2.2.1.RELEASE</spring.boot.starter.data.jpa.version>
        <database.h2.version>1.4.199</database.h2.version>          <spring.cloud.dataflow.rest.client.version>2.7.1</spring.cloud.dataflow.rest.client.version>
        <jackson-datatype-jsr310.version>2.10.0</jackson-datatype-jsr310.version>
        <spring.batch.version>4.2.4.RELEASE</spring.batch.version>
        <log4jversion>1.2.17-cloudera1</log4jversion>
    </properties>

    <build>
        <plugins>
            <!-- Maven Compiler Plugin -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>${java.version}</source>
                    <target>${java.version}</target>
                </configuration>
            </plugin>
        </plugins>
    </build>


</project>

我为上述应用程序注册了 docker 映像，创建了一个任务并从 SCDF UI 启动了该任务。我在 pod 日志中没有看到 Spring Batch 步骤或作业日志。事实上，我没有看到 System.out.println 或我在日志中的代码周围添加的所有日志语句。

pod 似乎正在运行 BatchApplication 类，但似乎根本没有启动作业。我错过了什么？ 即使我在 Windows 上将 jar 作为 spring boot uber jar 运行，也会观察到同样的问题

Picked up _JAVA_OPTIONS: -Djdk.tls.maxCertificateChainLength=20

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__,| / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.2.5.RELEASE)

2021-06-07 16:23:58.021  INFO 1 --- [           main] .r.c.b.w.b.c.o.BatchApplication : Starting BatchApplication  on batch-controller-test-5wg6eryzpj with PID 1 (/tmp/my-batch-app.jar started by ? in /tmp)
2021-06-07 16:23:58.026  INFO 1 --- [           main] .r.c.b.w.b.c.o.BatchApplication  : The following profiles are active: master
2021-06-07 16:24:03.031  INFO 1 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2021-06-07 16:24:03.316  INFO 1 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 185ms. Found 0 JPA repository interfaces.
2021-06-07 16:24:04.917  INFO 1 --- [           main] faultConfiguringbeanfactoryPostProcessor : No bean named 'errorChannel' has been explicitly defined. Therefore,a default PublishSubscribeChannel will be created.
2021-06-07 16:24:04.935  INFO 1 --- [           main] faultConfiguringbeanfactoryPostProcessor : No bean named 'taskScheduler' has been explicitly defined. Therefore,a default ThreadPoolTaskScheduler will be created.
2021-06-07 16:24:04.949  INFO 1 --- [           main] faultConfiguringbeanfactoryPostProcessor : No bean named 'integrationHeaderChannelRegistry' has been explicitly defined. Therefore,a default DefaultHeaderChannelRegistry will be created.
2021-06-07 16:24:06.814  INFO 1 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.integration.config.IntegrationManagementConfiguration' of type [org.springframework.integration.config.IntegrationManagementConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-06-07 16:24:07.047  INFO 1 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'integrationChannelResolver' of type [org.springframework.integration.support.channel.beanfactoryChannelResolver] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-06-07 16:24:07.113  INFO 1 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'integrationdisposableautocreatedBeans' of type [org.springframework.integration.config.annotation.disposables] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-06-07 16:24:09.125  INFO 1 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2021-06-07 16:24:09.822  INFO 1 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
2021-06-07 16:24:10.512  INFO 1 --- [           main] o.hibernate.jpa.internal.util.LogHelper  : HHH000204: Processing PersistenceUnitInfo [name: default]
2021-06-07 16:24:11.220  INFO 1 --- [           main] org.hibernate.Version                    : HHH000412: Hibernate Core {5.4.8.Final}
2021-06-07 16:24:12.724  INFO 1 --- [           main] o.hibernate.annotations.common.Version   : HCANN000001: Hibernate Commons Annotations {5.1.0.Final}
2021-06-07 16:24:14.019  INFO 1 --- [           main] org.hibernate.dialect.Dialect            : HHH000400: Using dialect: org.hibernate.dialect.H2Dialect
2021-06-07 16:24:16.335  INFO 1 --- [           main] o.h.e.t.j.p.i.JtaPlatformInitiator       : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2021-06-07 16:24:16.429  INFO 1 --- [           main] j.LocalContainerEntityManagerfactorybean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2021-06-07 16:24:19.814  INFO 1 --- [           main] o.s.s.c.ThreadPoolTaskScheduler          : Initializing ExecutorService 'taskScheduler'
2021-06-07 16:24:21.721  INFO 1 --- [           main] o.s.i.endpoint.EventDrivenConsumer       : Adding {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2021-06-07 16:24:21.721  INFO 1 --- [           main] o.s.i.channel.PublishSubscribeChannel    : Channel 'application.errorChannel' has 1 subscriber(s).
2021-06-07 16:24:21.722  INFO 1 --- [           main] o.s.i.endpoint.EventDrivenConsumer       : started bean '_org.springframework.integration.errorLogger'
2021-06-07 16:24:21.818  INFO 1 --- [           main] .r.c.b.w.b.c.o.BatchApplication  : Started BatchApplication  in 27.903 seconds (JVM running for 31.116)
2021-06-07 16:24:21.828  INFO 1 --- [extShutdownHook] o.s.i.endpoint.EventDrivenConsumer       : Removing {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2021-06-07 16:24:21.830  INFO 1 --- [extShutdownHook] o.s.i.channel.PublishSubscribeChannel    : Channel 'application.errorChannel' has 0 subscriber(s).
2021-06-07 16:24:21.830  INFO 1 --- [extShutdownHook] o.s.i.endpoint.EventDrivenConsumer       : stopped bean '_org.springframework.integration.errorLogger'
2021-06-07 16:24:21.832  INFO 1 --- [extShutdownHook] o.s.s.c.ThreadPoolTaskScheduler          : Shutting down ExecutorService 'taskScheduler'
2021-06-07 16:24:21.916  INFO 1 --- [extShutdownHook] j.LocalContainerEntityManagerfactorybean : Closing JPA EntityManagerFactory for persistence unit 'default'
2021-06-07 16:24:21.917  INFO 1 --- [extShutdownHook] .SchemaDropperImpl$DelayedDropActionImpl : HHH000477: Starting delayed evictData of schema as part of SessionFactory shut-down'
2021-06-07 16:24:21.920  INFO 1 --- [extShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown initiated...
2021-06-07 16:24:22.014  INFO 1 --- [extShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown completed.

JAR 内容

为了让事情更清楚，这是我在 JAR 中看到的文件夹列表。看起来是用spring boot maven插件来创建jar的？为什么批处理作业不会在 Eclipse 之外启动：

--my-batch-app.jar
 - BOOT-INF
   - classes
   - lib
 - meta-inf
 - org

注意：如您所见，日志还显示确实选择了主配置文件，因为我在启动任务时传递了 --spring.profiles.active=master 作为输入参数。

解决方法

很少有后续问题-

您是按计划运行还是在启动时运行此应用？

您是否添加了此启动器依赖项？

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-batch</artifactId>
</dependency>

根据您的回答，如果您愿意分享，我可能需要深入研究您的代码。

,

我测试了您的应用程序，您的问题是您的 BatchJob 类未标记为 Spring 配置类。您应该按如下方式更新它：

// Your current imports
import org.springframework.context.annotation.Configuration;

@EnableTask
@EnableBatchProcessing
@Profile("master")
@Configuration // This is missing in your current config
public class BatchJob {
   ...
}

这样，我就能看到您的日志语句，并且作业在应用程序启动时正确启动。

javajavaspring-batchspring-cloud-dataflowspring-cloud-dataflow-ui