Specflow by default generates the output results in SpecFlow JSON format. AssertThat plugin can consume it as Cucumber JSON format. The following solution will generate the output result in Cucumber JSON format in parallel and the result could imported to AssertThat plugin using AssertThat Rest API's or running a Command line tool.
Option 1 (.NET 4.5)
Step 1 : Add Dependencies
Code Block |
---|
https://www.nuget.org/packages/SpecNuts/
https://www.nuget.org/packages/SpecNuts.Json/ |
|
...
Step 5 : The generated output result file will be in Cucumber JSON format with the name "specflow_cucumber.json". The generated JSON can be imported into the plugin now.
Option 2
Install SpecFlow and the SpecFlow+ Runner 1.7.2 or newer along with the msbuild helper package; If you're using Visual Studio, just go to NuGet's Console (Tools | NuGet Package Manager | Package Manager Console)
Install-Package SpecRun.SpecFlow
Install-Package SpecRun.SpecRun
Install-Package SpecFlow.Tools.MsBuild.Generation
packages.config Collapse source
Code Block |
---|
<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="Newtonsoft.Json" version="9.0.1" targetFramework="net452" />
<package id="SpecFlow" version="2.3.2" targetFramework="net452" />
<package id="SpecFlow.Tools.MsBuild.Generation" version="2.3.2" targetFramework="net452" />
<package id="SpecRun.Runner" version="1.7.2" targetFramework="net452" />
<package id="SpecRun.SpecFlow" version="1.7.2" targetFramework="net452" />
<package id="SpecRun.SpecFlow.2-3-0" version="1.7.2" targetFramework="net452" />
<package id="System.ValueTuple" version="4.3.0" targetFramework="net452" />
</packages> |
Use the CucumberJson.cshtml
report template provided in this page
Before compiling and running the tests, you have to use a proper SpecFlow report template file in order to generate a valid Cucumber JSON report and you have to configure the test profile to use it.
CucumberJson.cshtml
Code Block |
---|
@inherits TechTalk.SpecRun.Framework.Reporting.CustomTemplateBase<TestRunResult>
@using System
@using System.Collections.Generic
@using System.Linq
@using System.Globalization
@using Newtonsoft.Json
@using Newtonsoft.Json.Converters
@using TechTalk.SpecRun.Framework
@using TechTalk.SpecRun.Framework.Results
@using TechTalk.SpecRun.Framework.TestSuiteStructure
@using TechTalk.SpecRun.Framework.Tracing
@{
var serializationSettings = new JsonSerializerSettings
{
ReferenceLoopHandling = ReferenceLoopHandling.Ignore,
Converters = new List<JsonConverter>() { new StringEnumConverter(false) }
};
var features = GetTextFixtures()
.Select(f => new
{
description = "",
elements = (from scenario in f.SubNodes
let lastExecutionResult = GetTestItemResult(scenario.GetTestSequence().First()).LastExecutionResult()
select new
{
description = "",
id = "",
keyword = "Scenario",
line = scenario.Source.SourceLine + 1,
name = scenario.Title,
tags = scenario.Tags.Select(t => new { name = t, line = 1 }),
steps = from step in lastExecutionResult.Result.TraceEvents
where IsRelevant(step) && (step.ResultType == TestNodeResultType.Succeeded || step.ResultType == TestNodeResultType.Failed || step.ResultType == TestNodeResultType.Pending)
&& (step.Type == TraceEventType.Test || step.Type == TraceEventType.TestAct || step.Type == TraceEventType.TestArrange || step.Type == TraceEventType.TestAssert)
let keyword = step.StepBindingInformation == null ? "" : step.StepBindingInformation.StepInstanceInformation == null ? "" : step.StepBindingInformation.StepInstanceInformation.Keyword
let matchLocation = step.StepBindingInformation == null ? "" : step.StepBindingInformation.MethodName
let name = step.StepBindingInformation == null ? "" : step.StepBindingInformation.Text
let cucumberStatus = step.ResultType == TestNodeResultType.Succeeded ? "Passed" : step.ResultType.ToString()
select new
{
keyword = keyword,
line = 0,
match = new
{
location = matchLocation
},
name = name,
result = new
{
duration = step.Duration.TotalMilliseconds,
error_message = step.StackTrace,
status = cucumberStatus
}
},
type = "scenario"
}).ToList(),
id = "",
keyword = "Feature",
line = f.Source.SourceLine + 1,
tags = f.Tags.Select(t => new { name = t, line = 1 }),
name = f.Title,
uri = f.Source.SourceFile
});
}
@Raw(JsonConvert.SerializeObject(features, Formatting.Indented, serializationSettings)) |
|
Default.srprofile
Code Block |
---|
<?xml version="1.0" encoding="utf-8"?>
<TestProfile xmlns="http://www.specflow.org/schemas/plus/TestProfile/1.5">
<Settings projectName="TestProject" projectId="{xxx-xxx-xxx-xxx}" />
<Execution stopAfterFailures="2" testThreadCount="1" testSchedulingMode="Sequential" />
<TestAssemblyPaths>
<TestAssemblyPath>TestProject.dll</TestAssemblyPath>
</TestAssemblyPaths>
<DeploymentTransformation>
...
</DeploymentTransformation>
<Report>
<Template name="CucumberJson.cshtml" outputName="data.json"/>
</Report>
</TestProfile> |
|
Tests can be run from within the IDE (e.g. Visual Studio) or by the command line; in the later case, make sure to specify the profile name and all the paths properly.
Since there is code-behind file generation, it is required to have the NuGet "SpecFlow.Tools.MsBuild.Generation" package.
Code Block |
---|
msbuild /t:Clean;Rebuild
cd bin\debug
..\..\..\packages\SpecRun.Runner.1.7.2\tools\SpecRun.exe run Default.srprofile /outputFolder:..\..\..\TestResults
cd ..\.. |