在 DevOps 管道上的 VSTest 任务中跳过所有 Specflow 场景

问题描述

我已经配置了一个 DevOps 管道来执行我的 SpecFlow Selenium 测试。一切似乎都很好，只是我的所有测试场景都在 DevOps 代理上被跳过了。

当我使用 VS2019 测试资源管理器在本地运行测试时，它们运行没有问题。

我尝试启用系统诊断以查看跳过它们的原因，但看不到任何可能是原因的内容。

这是 VSTest 任务的 YAML：

steps:
- task: VSTest@2
  displayName: 'VsTest - RetroSpectro.Tests.UI.Specflow.dll'
  inputs:
    testAssemblyVer2: |
     **\bin\any cpu\release\net5.0\RetroSpectro.Tests.UI.Specflow.dll
     !**\*TestAdapter.dll
     !**\obj\**
     !**\bin\**\ref\**
    uiTests: true
    runSettingsFile: RetroSpectro.Tests.UI.Specflow/Serial.runsettings
    pathtoCustomTestAdapters: packages
    runTestsInIsolation: true
    testRunTitle: 'SpecFlow Tests'
    platform: '$(Buildplatform)'
    configuration: '$(BuildConfiguration)'
    failOnMinTestsNotRun: true
    rerunFailedTests: false

... 以及任务执行的输出：

2021-06-25T09:02:15.0841271Z ##[section]Starting: VsTest - RetroSpectro.Tests.UI.Specflow.dll
2021-06-25T09:02:15.1180446Z ==============================================================================
2021-06-25T09:02:15.1180820Z Task         : Visual Studio Test
2021-06-25T09:02:15.1181621Z Description  : Run unit and functional tests (Selenium,Appium,Coded UI test,etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest,xUnit,NUnit,Chutzpah (for JavaScript tests using QUnit,Mocha and Jasmine),etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
2021-06-25T09:02:15.1182585Z Version      : 2.170.1
2021-06-25T09:02:15.1183892Z Author       : Microsoft Corporation
2021-06-25T09:02:15.1184298Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/test/vstest
2021-06-25T09:02:15.1184712Z ==============================================================================
2021-06-25T09:02:16.6157598Z SystemVssConnection exists true
2021-06-25T09:02:16.7413804Z SystemVssConnection exists true
2021-06-25T09:02:16.9135626Z Running tests using vstest.console.exe runner.
2021-06-25T09:02:16.9136452Z ======================================================
2021-06-25T09:02:16.9138232Z Test selector : Test assemblies
2021-06-25T09:02:16.9139517Z Test filter criteria : null
2021-06-25T09:02:16.9140779Z Search folder : D:\a\1\s
2021-06-25T09:02:16.9937367Z Action when minimum tests threshold not met : fail
2021-06-25T09:02:16.9938318Z Minimum tests expected to be run: 1
2021-06-25T09:02:16.9939834Z VisualStudio version selected for test execution : latest
2021-06-25T09:02:16.9943536Z Attempting to find vstest.console from a visual studio installation with version [16.0,17.0).
2021-06-25T09:02:16.9973962Z Run settings file : D:\a\1\s\RetroSpectro.Tests.UI.Specflow\Serial.runsettings
2021-06-25T09:02:16.9981237Z Run in parallel : false
2021-06-25T09:02:16.9984189Z Run in isolation : true
2021-06-25T09:02:16.9986763Z Path to custom adapters : D:\a\1\s\packages
2021-06-25T09:02:16.9991903Z Other console options : null
2021-06-25T09:02:16.9994735Z Code coverage enabled : false
2021-06-25T09:02:17.0001246Z Diagnostics enabled : false
2021-06-25T09:02:17.0008374Z SystemVssConnection exists true
2021-06-25T09:02:17.0368729Z Run the tests locally using vstest.console.exe
2021-06-25T09:02:17.0369728Z ========================================================
2021-06-25T09:02:17.0464264Z Source filter: **\bin\any cpu\release\net5.0\RetroSpectro.Tests.UI.Specflow.dll,!**\*TestAdapter.dll,!**\obj\**,!**\bin\**\ref\**
2021-06-25T09:02:22.7152869Z SystemVssConnection exists true
2021-06-25T09:02:22.7157229Z [command]D:\a\_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\2.170.1\Modules\DTAExecutionHost.exe --inputFile D:\a\_temp\input_0dad9d50-d594-11eb-bac6-a972b6c3989e.json
2021-06-25T09:02:22.7158487Z ======================================================
2021-06-25T09:02:22.8592004Z ##########################################################################
2021-06-25T09:02:22.8592761Z DtaExecutionHost version 18.170.30112.1.
2021-06-25T09:02:23.7498265Z Starting TestExecution Model...
2021-06-25T09:02:26.9047814Z Result Attachments will be stored in LogStore
2021-06-25T09:02:26.9297128Z Run Attachments will be stored in LogStore
2021-06-25T09:02:26.9433340Z Result Attachments will be stored in LogStore
2021-06-25T09:02:27.1235011Z Result Attachments will be stored in LogStore
2021-06-25T09:02:27.1235834Z Run Attachments will be stored in LogStore
2021-06-25T09:02:27.1470266Z Provided settings file:
2021-06-25T09:02:27.1485829Z <?xml version="1.0" encoding="utf-8"?>
2021-06-25T09:02:27.1486598Z <RunSettings>
2021-06-25T09:02:27.1487139Z    <SpecRun>
2021-06-25T09:02:27.1488252Z        <Profile>Serial.srprofile</Profile>
2021-06-25T09:02:27.1488944Z        <GenerateFeatureTrait>true</GenerateFeatureTrait>
2021-06-25T09:02:27.1490018Z    </SpecRun>
2021-06-25T09:02:27.1490506Z </RunSettings>
2021-06-25T09:02:27.1785620Z Updated Run Settings:
2021-06-25T09:02:27.1788682Z <RunSettings>
2021-06-25T09:02:27.1788992Z   <SpecRun>
2021-06-25T09:02:27.1789483Z     <Profile>Serial.srprofile</Profile>
2021-06-25T09:02:27.1789897Z     <GenerateFeatureTrait>true</GenerateFeatureTrait>
2021-06-25T09:02:27.1790256Z   </SpecRun>
2021-06-25T09:02:27.1790560Z   <runconfiguration>
2021-06-25T09:02:27.1790961Z     <BatchSize>1000</BatchSize>
2021-06-25T09:02:27.1791385Z     <ResultsDirectory>D:\a\_temp\TestResults</ResultsDirectory>
2021-06-25T09:02:27.1791975Z   </runconfiguration>
2021-06-25T09:02:27.1792294Z </RunSettings>
2021-06-25T09:02:27.1979593Z **************** Starting test execution *********************
2021-06-25T09:02:27.2197434Z [command]C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\Extensions\TestPlatform\vstest.console.exe "@D:\a\_temp\y3jxhopis4l.tmp"
2021-06-25T09:02:27.4233968Z Microsoft (R) Test Execution Command Line Tool Version 16.10.0
2021-06-25T09:02:27.4235225Z copyright (c) Microsoft Corporation.  All rights reserved.
2021-06-25T09:02:27.4413296Z vstest.console.exe "D:\a\1\s\RetroSpectro.Tests.UI.Specflow\bin\any cpu\release\net5.0\RetroSpectro.Tests.UI.Specflow.dll"
2021-06-25T09:02:27.4414588Z /Settings:"D:\a\_temp\kpbetfmovlq.tmp.runsettings"
2021-06-25T09:02:27.4415655Z /InIsolation
2021-06-25T09:02:27.4416622Z /Logger:"trx"
2021-06-25T09:02:27.4417670Z /TestAdapterPath:"D:\a\1\s\packages"
2021-06-25T09:02:29.1247592Z Starting test execution,please wait...
2021-06-25T09:02:30.0040388Z A total of 1 test files matched the specified pattern.
2021-06-25T09:02:33.0429462Z SpecFlow+Runner execution started
2021-06-25T09:02:33.0431615Z SpecFlow+Runner 3.9.7 in Framework net50 in x64 mode execution started
2021-06-25T09:02:33.0433131Z TestAdapter Location: D:\a\1\s\RetroSpectro.Tests.UI.Specflow\bin\any cpu\release\net5.0\TechTalk.SpecRun.VisualStudio.TestAdapter.dll
2021-06-25T09:02:33.0433910Z Current UserName: 'VssAdministrator',MachineName: 'WIN-RNFTU0DQEPC'
2021-06-25T09:02:33.3384985Z SpecRun: running tests in D:\a\1\s\RetroSpectro.Tests.UI.Specflow\bin\any cpu\release\net5.0\RetroSpectro.Tests.UI.Specflow.dll
2021-06-25T09:02:33.3386699Z Output folder configured to "D:\a\_temp\TestResults" (default)
2021-06-25T09:02:33.3387477Z Profile: RetroSpectro.Tests.UI.Specflow
2021-06-25T09:02:33.4638846Z Log file path: D:\a\_temp\TestResults\RetroSpectro.Tests.UI.Specflow_RetroSpectro.Tests.UI.Specflow_2021-06-25T090233.log
2021-06-25T09:02:33.4749622Z Checking activation
2021-06-25T09:02:33.4854960Z Activation check completed
2021-06-25T09:02:33.4878329Z Starting test run
2021-06-25T09:02:33.7105584Z discovering target: Chrome
2021-06-25T09:02:33.7106665Z discovering tests from assembly: D:\a\1\s\RetroSpectro.Tests.UI.Specflow\bin\any cpu\release\net5.0\RetroSpectro.Tests.UI.Specflow.dll
2021-06-25T09:02:33.8841939Z discovered tests from assembly: D:\a\1\s\RetroSpectro.Tests.UI.Specflow\bin\any cpu\release\net5.0\RetroSpectro.Tests.UI.Specflow.dll
2021-06-25T09:02:33.8843042Z discovering target: Edge
2021-06-25T09:02:33.8843914Z discovering tests from assembly: D:\a\1\s\RetroSpectro.Tests.UI.Specflow\bin\any cpu\release\net5.0\RetroSpectro.Tests.UI.Specflow.dll
2021-06-25T09:02:33.9067409Z discovered tests from assembly: D:\a\1\s\RetroSpectro.Tests.UI.Specflow\bin\any cpu\release\net5.0\RetroSpectro.Tests.UI.Specflow.dll
2021-06-25T09:02:33.9068593Z discovering target: Firefox
2021-06-25T09:02:33.9069400Z discovering tests from assembly: D:\a\1\s\RetroSpectro.Tests.UI.Specflow\bin\any cpu\release\net5.0\RetroSpectro.Tests.UI.Specflow.dll
2021-06-25T09:02:33.9193883Z discovered tests from assembly: D:\a\1\s\RetroSpectro.Tests.UI.Specflow\bin\any cpu\release\net5.0\RetroSpectro.Tests.UI.Specflow.dll
2021-06-25T09:02:33.9195007Z discovering target: IE
2021-06-25T09:02:33.9196501Z discovering tests from assembly: D:\a\1\s\RetroSpectro.Tests.UI.Specflow\bin\any cpu\release\net5.0\RetroSpectro.Tests.UI.Specflow.dll
2021-06-25T09:02:33.9272484Z discovered tests from assembly: D:\a\1\s\RetroSpectro.Tests.UI.Specflow\bin\any cpu\release\net5.0\RetroSpectro.Tests.UI.Specflow.dll
2021-06-25T09:02:33.9277522Z discovering target: Safari
2021-06-25T09:02:33.9278613Z discovering tests from assembly: D:\a\1\s\RetroSpectro.Tests.UI.Specflow\bin\any cpu\release\net5.0\RetroSpectro.Tests.UI.Specflow.dll
2021-06-25T09:02:33.9352988Z discovered tests from assembly: D:\a\1\s\RetroSpectro.Tests.UI.Specflow\bin\any cpu\release\net5.0\RetroSpectro.Tests.UI.Specflow.dll
2021-06-25T09:02:56.8250888Z Scenario: Follow button is not available in header section on Add Item page in AddEditItemFeature (target: Chrome) -> Skipped on thread #-1
...
<!-- other chrome tests also skipped on thread #-1 -->
...
2021-06-25T09:02:56.8376857Z Scenario: Group description is available in header section on Edit Item page in AddEditItemFeature (target: Edge) -> Skipped on thread #-1
...
<!-- other edge tests also skipped on thread #-1 -->
...
2021-06-25T09:02:56.8518611Z Scenario: Group status is available in header section on Add Item page in AddEditItemFeature (target: Firefox) -> Skipped on thread #-1
...
<!-- other firefox tests also skipped on thread #-1 -->
...
2021-06-25T09:02:56.8587604Z Scenario: Group description is available in header section on Add Item page in AddEditItemFeature (target: IE) -> Skipped on thread #-1
...
<!-- other IE tests also skipped on thread #-1 -->
...
2021-06-25T09:02:56.8648632Z Scenario: Title label is displayed on Add Item page in AddEditItemFeature (target: Safari) -> Skipped on thread #-1
...
<!-- other Safari tests also skipped on thread #-1 -->
...
2021-06-25T09:02:56.8764035Z test run finished
2021-06-25T09:02:56.8765441Z publishing test results
2021-06-25T09:02:56.8766131Z Not publishing results.
2021-06-25T09:02:56.8766729Z test results published
2021-06-25T09:02:56.8767274Z generating reports
2021-06-25T09:02:56.9558142Z Starting external report generation process
2021-06-25T09:02:56.9590180Z Starting external program: "D:\a\1\s\RetroSpectro.Tests.UI.Specflow\bin\any cpu\release\net5.0\SpecFlowPlusRunner\net461\SpecFlow.Plus.Runner.Reporting.exe" "C:\Users\VssAdministrator\AppData\Local\Temp\ca95d93f-da8f-41ce-9966-62cb15b332e9" in 
2021-06-25T09:02:58.4457850Z   Skipped Follow button is not available in header section on Add Item page [1 ms]
...
<!-- all tests were skipped in [< 1 ms] --!>
...
2021-06-25T09:03:00.2990896Z 'D:\a\1\s\RetroSpectro.Tests.UI.Specflow\bin\any cpu\release\net5.0\SpecFlowPlusRunner\net461\SpecFlow.Plus.Runner.Reporting.exe "C:\Users\VssAdministrator\AppData\Local\Temp\ca95d93f-da8f-41ce-9966-62cb15b332e9"' took 3337.3233ms
2021-06-25T09:03:00.2992974Z RAA6AFwAYQBcAF8AdABlAG0AcABcAFQAZQBzAHQAUgBlAHMAdQBsAHQAcwBcAFIAZQB0AHIAbwBTAHAAZQBjAHQAcgBvAC4AVABlAHMAdABzAC4AVQBJAC4AUwBwAGUAYwBmAGwAbwB3AF8AUgBlAHQAcgBvAFMAcABlAGMAdAByAG8ALgBUAGUAcwB0AHMALgBVAEkALgBTAHAAZQBjAGYAbABvAHcAXwAyADAAMgAxAC0AMAA2AC0AMgA1AFQAMAA5ADAAMgAzADMALgBoAHQAbQBsAA==
2021-06-25T09:03:00.2994487Z Successfully generated reports.
2021-06-25T09:03:00.3024068Z Successfully generated reports
2021-06-25T09:03:00.3157341Z Result: test framework error: At least one test thread aborted.
2021-06-25T09:03:00.3159973Z   Total: 185
2021-06-25T09:03:00.3160575Z   Succeeded: 0
2021-06-25T09:03:00.3161111Z   Ignored: 0
2021-06-25T09:03:00.3161627Z   Pending: 0
2021-06-25T09:03:00.3163981Z   Skipped: 185
2021-06-25T09:03:00.3164566Z   Failed: 0
2021-06-25T09:03:00.3165688Z   Execution Time: 00:00:22.9059275
2021-06-25T09:03:00.3166798Z Report file: file:///D:/a/_temp/TestResults/RetroSpectro.Tests.UI.Specflow_RetroSpectro.Tests.UI.Specflow_2021-06-25T090233.html
2021-06-25T09:03:00.3168312Z Adding attachments to VSTest
2021-06-25T09:03:00.3169047Z adding Test Execution Report 'D:\a\_temp\TestResults\RetroSpectro.Tests.UI.Specflow_RetroSpectro.Tests.UI.Specflow_2021-06-25T090233.html'
2021-06-25T09:03:00.3283768Z Log file: file:///D:/a/_temp/TestResults/RetroSpectro.Tests.UI.Specflow_RetroSpectro.Tests.UI.Specflow_2021-06-25T090233.log
2021-06-25T09:03:00.3294371Z SpecFlow+Runner execution finished
2021-06-25T09:03:00.6411255Z Results File: D:\a\_temp\TestResults\VssAdministrator_WIN-RNFTU0DQEPC_2021-06-25_09_02_58.trx
2021-06-25T09:03:00.6488918Z Attachments:
2021-06-25T09:03:00.6490475Z   C:\Users\VssAdministrator\AppData\Local\Temp\0erzmwe3.ry2\RetroSpectro.Tests.UI.Specflow_RetroSpectro.Tests.UI.Specflow_2021-06-25T090233.log
2021-06-25T09:03:00.6494999Z   C:\Users\VssAdministrator\AppData\Local\Temp\0erzmwe3.ry2\RetroSpectro.Tests.UI.Specflow_RetroSpectro.Tests.UI.Specflow_2021-06-25T090233.html
2021-06-25T09:03:00.6813251Z Test Run Successful.
2021-06-25T09:03:00.6814110Z Total tests: 185
2021-06-25T09:03:00.6814725Z     Skipped: 185
2021-06-25T09:03:00.6826934Z  Total time: 30.4837 Seconds
2021-06-25T09:03:00.7194885Z Vstest.console.exe exited with code 0.
2021-06-25T09:03:00.7196364Z **************** Completed test execution *********************
2021-06-25T09:03:00.7398059Z Test results files: D:\a\_temp\TestResults\VssAdministrator_WIN-RNFTU0DQEPC_2021-06-25_09_02_58.trx
2021-06-25T09:03:01.3258001Z Created test run: 1014788
2021-06-25T09:03:01.3261282Z Publishing test results: 185
2021-06-25T09:03:01.6120517Z Publishing test results to test run '1014788'.
2021-06-25T09:03:01.6165436Z TestResults To Publish 185,Test run id:1014788
2021-06-25T09:03:01.6272324Z Test results publishing 185,remaining: 0. Test run id: 1014788
2021-06-25T09:03:02.2543932Z Published test results: 185
2021-06-25T09:03:02.2556259Z Publishing Attachments: 3
2021-06-25T09:03:03.5514586Z ##[error]The specified minimum number of tests 1 were not executed in the test run.
2021-06-25T09:03:04.2147578Z Execution Result Code 1 is non zero,checking for Failed results
2021-06-25T09:03:04.3270302Z Completed TestExecution Model...
2021-06-25T09:03:04.3691332Z ##[warning]Vstest Failed with error. Check logs for failures. There might be Failed tests.
2021-06-25T09:03:04.3697400Z ##[error]Error: The process 'D:\a\_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\2.170.1\Modules\DTAExecutionHost.exe' Failed with exit code 1
2021-06-25T09:03:04.3703872Z ##[error]Vstest Failed with error. Check logs for failures. There might be Failed tests.
2021-06-25T09:03:04.3933532Z ##[section]Finishing: VsTest - RetroSpectro.Tests.UI.Specflow.dll

看起来测试线程被中止了，但我不知道为什么。

.runsettings 相对简单：

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
    <SpecRun>
        <Profile>Serial.srprofile</Profile>
        <GenerateFeatureTrait>true</GenerateFeatureTrait>
    </SpecRun>
</RunSettings>

规范流配置文件是：

<?xml version="1.0" encoding="utf-8"?>
<TestProfile xmlns="http://www.specflow.org/schemas/plus/TestProfile/1.5">
    <Settings projectName="RetroSpectro.Tests.UI.Specflow" />
    <Execution retryFor="Failing" stopAfterFailures="0" testThreadCount="1" testSchedulingMode="Random" retryCount="2" />
    <Environment framework="Net5.0" platform="x86" testThreadisolation="AppDomain" apartmentState="STA"/>
    <TestAssemblyPaths>
        <TestAssemblyPath>RetroSpectro.Tests.UI.Specflow.dll</TestAssemblyPath>
    </TestAssemblyPaths>
    <VSTest testRetryResults="Unified" passRateAbsolute="1"/>
    <Targets>
        <Target name="Chrome">
            <Filter>!@Parallel</Filter>
            <DeploymentTransformationSteps>
                <Environmentvariable variable="RUNNER_TARGET" value="chrome" />
            </DeploymentTransformationSteps>
        </Target>
        <Target name="Edge">
            <Filter>!@Parallel</Filter>
            <DeploymentTransformationSteps>
                <Environmentvariable variable="RUNNER_TARGET" value="edge" />
            </DeploymentTransformationSteps>
        </Target>
        <Target name="Firefox">
            <Filter>!@Parallel</Filter>
            <DeploymentTransformationSteps>
                <Environmentvariable variable="RUNNER_TARGET" value="firefox" />
            </DeploymentTransformationSteps>
        </Target>
        <Target name="IE">
            <Filter>!@Parallel</Filter>
            <DeploymentTransformationSteps>
                <Environmentvariable variable="RUNNER_TARGET" value="ie" />
            </DeploymentTransformationSteps>
        </Target>
        <Target name="Safari">
            <Filter>!@Parallel</Filter>
            <DeploymentTransformationSteps>
                <Environmentvariable variable="RUNNER_TARGET" value="safari" />
            </DeploymentTransformationSteps>
        </Target>
    </Targets>
</TestProfile>

知道为什么跳过测试或我可以尝试找出什么吗？

编辑：

我注意到在之前的（工作）运行中，VSTest 在线程 #-0 中运行场景。崩溃后，它开始记录“在线程 #-1 中跳过”消息。

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）