Cucumber 报告 - 从失败时多次运行的功能文件创建合并报告

编程问答 2022-05-23

问题描述

我知道有人问过类似的问题，但它们似乎都略有不同，无法找到特定于这个问题的答案。任何帮助/建议将不胜感激。

我想根据重新运行失败的测试创建合并报告。

我正在使用 maven、junit、cucumber、surefire、masterthought。

这是 Jenkins 的假设运行：

Feature: This is the test
    Scenario 1 - Failed
    Scenario 2 - Passed
    Scenario 3 - Failed

有失败，所以再次测试运行：

Feature: This is the test
    Scenario 1 - Passed (this is flaky)
    Scenario 3 - Failed

我目前在 Cucumber 报告中看到的是 RUN 2 的结果。

我想看到的是单个合并报告。类似的东西：

Feature: This is the test
    Scenario 1 - Passed
    Scenario 2 - Passed
    Scenario 3 - Failed

使用surefire插件有：

<rerunFailingTestsCount>1</rerunFailingTestsCount>
<forkCount>2</forkCount>

我也尝试删除 forkCount 并并行运行。

我在 masterthought 插件中尝试了 mergeFeaturesById 和 mergeFeaturesWithRetest，结果没有区别。

如果此配置无法实现，愿意接受替代方案。

这是其中一个跑步者类包含的内容：

package com.postmedia.automation.step_deFinitions.runners.Ads;

import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
import org.junit.runner.RunWith;

@RunWith(Cucumber.class)
@CucumberOptions(features = { "src/main/resources/features/Ads/AdLiteLFPPremium.feature" },glue = {
        "com.postmedia.automation" },plugin = { "pretty","html:target/cucumber-reports","json:target/cucumber-reports/AdLiteLFPPremium.json","junit:target/cucumber-reports/AdLiteLFPPremium.xml" },monochrome = true)
public class Test_AdLiteLFPPremium {
}

这是 pom.xml 文件的一部分

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>
    <groupId>postmedia-automated-testing</groupId>
    <artifactId>postmedia-automated-testing</artifactId>
    <version>1.0-SNAPSHOT</version>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.1</version>
                <configuration>
                    <fork>true</fork>
                    <source>8</source>
                    <target>8</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>com.diffplug.spotless</groupId>
                <artifactId>spotless-maven-plugin</artifactId>
                <version>1.15.0</version>
                <configuration>
                    <java>
                        <eclipse>
                            <file>${basedir}/spotless.eclipse.xml</file>
                            <version>4.7.1</version>
                        </eclipse>
                    </java>
                </configuration>
            </plugin>
            <plugin>
                <groupId>net.masterthought</groupId>
                <artifactId>cucumber-reporting</artifactId>
                <version>5.5.2</version>
                <executions>
                    <execution>
                        <id>execution</id>
                        <phase>verify</phase>
                        <goals>
                            <goal>generate</goal>
                        </goals>
                        <configuration>
                            <projectName>postmedia-automated-testing</projectName>
                            <!-- output directory for the generated report -->
                            <outputDirectory>${project.build.directory}/cucumber-reports</outputDirectory>
                            <!-- optional,defaults to outputDirectory if not specified -->
                            <inputDirectory>${project.build.directory}/cucumber-reports</inputDirectory>
                            <jsonFiles>
                                <!-- supports wildcard or name pattern -->
                                <param>**/*.json</param>
                            </jsonFiles>
                            <!-- optional,defaults to outputDirectory if not specified -->
                            <!--<classificationDirectory>${project.build.directory}/classification</classificationDirectory>-->
                            <classificationFiles>
                                <!-- supports wildcard or name pattern -->
                                <param>sample.properties</param>
                                <param>other.properties</param>
                            </classificationFiles>
                            <parallelTesting>true</parallelTesting>
                            <mergeFeaturesById>true</mergeFeaturesById>
                            <mergeFeaturesWithRetest>true</mergeFeaturesWithRetest>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M5</version>
                <configuration>
                    <rerunFailingTestsCount>1</rerunFailingTestsCount>
                    <forkCount>2</forkCount>
                    <reuseForks>true</reuseForks>
                    <!-- test 1
                    <parallel>classes</parallel>
                    <perCoreThreadCount>true</perCoreThreadCount>
                    -->
                    <!--
                    <debugForkedProcess>true</debugForkedProcess>
                    <parallel>all</parallel>
                    <threadCount>10</threadCount>
                    <threadCountSuites>2</threadCountSuites>
                    <threadCountClasses>2</threadCountClasses>
                    <threadCountMethods>6</threadCountMethods>
                    <perCoreThreadCount>true</perCoreThreadCount>
                    -->
                    <!--
                    <rerunFailingTestsCount>1</rerunFailingTestsCount>
                    -->
                    <!--
                                       <rerunFailingTestsCount>1</rerunFailingTestsCount>
                                       <forkCount>2.5C</forkCount>
                                       <reuseForks>true</reuseForks>
                                       -->
                    <argLine>-Xmx2G</argLine>
                    <systemPropertyVariables>
                        <databaseSchema>MY_TEST_SCHEMA_${surefire.forkNumber}</databaseSchema>
                    </systemPropertyVariables>
                    <!--                    <workingDirectory>FORK_DIRECTORY_${surefire.forkNumber}</workingDirectory>-->
                    <includes>
                        <include>**/Test*.java</include>
                        <include>**/*Test.java</include>
                        <include>**/*Tests.java</include>
                        <include>**/*TestCase.java</include>
                    </includes>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>versions-maven-plugin</artifactId>
                <version>2.7</version>
                <executions>
                    <execution>
                        <phase>validate</phase>
                        <goals>
                            <goal>display-dependency-updates</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
    <dependencies>
        <dependency>
            <groupId>com.github.stephenc.jcip</groupId>
            <artifactId>jcip-annotations</artifactId>
            <version>1.0-1</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.galenframework</groupId>
            <artifactId>galen-java-support</artifactId>
            <version>2.4.4</version>
        </dependency>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit4</artifactId>
            <version>3.0.0-M5</version>
        </dependency>
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>4.7.1</version>
        </dependency>
        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>1.2.17</version>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.9</version>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-server</artifactId>
            <version>3.141.59</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-chrome-driver</artifactId>
            <version>3.141.59</version>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-api</artifactId>
            <version>3.141.59</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/io.cucumber/cucumber-core -->
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-core</artifactId>
            <version>4.7.1</version>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.5.1</version>
        </dependency>
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-junit</artifactId>
            <version>4.7.1</version>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-support</artifactId>
            <version>3.141.59</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-picocontainer</artifactId>
            <version>4.7.1</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>commons-io</groupId>
            <artifactId>commons-io</artifactId>
            <version>2.6</version>
        </dependency>
        <dependency>
            <groupId>com.ibm.icu</groupId>
            <artifactId>icu4j</artifactId>
            <version>64.2</version>
        </dependency>
        <dependency>
            <groupId>com.github.fge</groupId>
            <artifactId>json-schema-validator</artifactId>
            <version>2.2.6</version>
            <scope>system</scope>
            <systemPath>${project.basedir}/jars/java-json-schema.jar</systemPath>
        </dependency>
        <dependency>
            <groupId>org.apache.poi</groupId>
            <artifactId>poi</artifactId>
            <version>4.1.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.poi</groupId>
            <artifactId>poi-ooxml</artifactId>
            <version>4.1.0</version>
        </dependency>
        <dependency>
            <groupId>com.googlecode.json-simple</groupId>
            <artifactId>json-simple</artifactId>
            <version>1.1.1</version>
        </dependency>
        <dependency>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>exec-maven-plugin</artifactId>
            <version>1.6.0</version>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch</groupId>
            <artifactId>elasticsearch</artifactId>
            <version>7.3.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <version>4.5.9</version>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-remote-driver</artifactId>
            <version>3.141.59</version>
        </dependency>
        <dependency>
            <groupId>com.google.code.gson</groupId>
            <artifactId>gson</artifactId>
            <version>2.8.5</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-firefox-driver</artifactId>
            <version>3.141.59</version>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-ie-driver</artifactId>
            <version>3.141.59</version>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-edge-driver</artifactId>
            <version>3.141.5</version>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-safari-driver</artifactId>
            <version>3.141.59</version>
        </dependency>
        <dependency>
            <groupId>ru.yandex.qatools.ashot</groupId>
            <artifactId>ashot</artifactId>
            <version>1.5.2</version>
        </dependency>
        <dependency>
            <groupId>com.assertthat</groupId>
            <artifactId>selenium-shutterbug</artifactId>
            <version>0.9</version>
        </dependency>
        <dependency>
            <groupId>net.lightbody.bmp</groupId>
            <!-- To use the legacy,Jetty-based implementation,change the artifactId to browsermob-core -->
            <artifactId>browsermob-core</artifactId>
            <version>2.1.5</version>
            <scope>compile</scope>
            <exclusions>
                <!-- Exclude Commons Logging in favor of SLF4j -->
                <exclusion>
                    <groupId>org.slf4j</groupId>
                    <artifactId>slf4j-log4j12</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>org.slf4j</groupId>
                    <artifactId>slf4j-api</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
    </dependencies>
</project>

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）

cucumber-javajunitjunitmaven-surefire-plugin