SpecFlow integration
- Glib
- Nick
Specflow by default generates the output results in SpecFlow JSON format. AssertThat plugin can consume it as Cucumber JSON format. The following solution will generate the output result in Cucumber JSON format in parallel and the result could imported to the AssertThat plugin using REST API for Jira Cloud V1 or running a Command line tool.
Option 1 (.NET 4.5)
Step 1: Add Dependencies
https://www.nuget.org/packages/SpecNuts/
https://www.nuget.org/packages/SpecNuts.Json/ |
Step 2: Add the below code in .cs of feature file
[BeforeTestRun]
public static void BeforeTestRun() {
SpecNuts.Reporters.Add(new JsonReporter());
SpecNuts.Reporters.FinishedReport += (sender, args) => {
String pathName = "specflow_cucumber.json";
System.IO.File.WriteAllText(pathName, args.Reporter.WriteToString());
Console.WriteLine("Result File: " + System.IO.Directory.GetCurrentDirectory().ToString() + System.IO.Path.DirectorySeparatorChar + pathName);
};
} |
Step 3: Open Text Explorer in Visual Studio by Test > Windows > Test Explorer -> Choose Run All
Step 4: In Visual Studio console, select show output from Test and open Report File as path given in Console
Step 5: The generated output result file will be in Cucumber JSON format with the name "specflow_cucumber.json". The generated JSON can be imported into the plugin now.
Option 2
Install SpecFlow and the SpecFlow+ Runner 1.7.2 or newer along with the msbuild helper package; If you're using Visual Studio, just go to NuGet's Console (Tools | NuGet Package Manager | Package Manager Console)
Install-Package SpecRun.SpecFlow
Install-Package SpecRun.SpecRun
Install-Package SpecFlow.Tools.MsBuild.Generation
packages.config
<?xml version="1.0" encoding="utf-8"?> <packages> <package id="Newtonsoft.Json" version="9.0.1" targetFramework="net452" /> <package id="SpecFlow" version="2.3.2" targetFramework="net452" /> <package id="SpecFlow.Tools.MsBuild.Generation" version="2.3.2" targetFramework="net452" /> <package id="SpecRun.Runner" version="1.7.2" targetFramework="net452" /> <package id="SpecRun.SpecFlow" version="1.7.2" targetFramework="net452" /> <package id="SpecRun.SpecFlow.2-3-0" version="1.7.2" targetFramework="net452" /> <package id="System.ValueTuple" version="4.3.0" targetFramework="net452" /> </packages>
Use the
CucumberJson.cshtmlreport template provided in this page
Before compiling and running the tests, you have to use a proper SpecFlow report template file in order to generate a valid Cucumber JSON report and you have to configure the test profile to use it.
CucumberJson.cshtml
@inherits TechTalk.SpecRun.Framework.Reporting.CustomTemplateBase<TestRunResult>
@using System
@using System.Collections.Generic
@using System.Linq
@using System.Globalization
@using Newtonsoft.Json
@using Newtonsoft.Json.Converters
@using TechTalk.SpecRun.Framework
@using TechTalk.SpecRun.Framework.Results
@using TechTalk.SpecRun.Framework.TestSuiteStructure
@using TechTalk.SpecRun.Framework.Tracing
@{
var serializationSettings = new JsonSerializerSettings
{
ReferenceLoopHandling = ReferenceLoopHandling.Ignore,
Converters = new List<JsonConverter>() { new StringEnumConverter(false) }
};
var features = GetTextFixtures()
.Select(f => new
{
description = "",
elements = (from scenario in f.SubNodes
let lastExecutionResult = GetTestItemResult(scenario.GetTestSequence().First()).LastExecutionResult()
select new
{
description = "",
id = "",
keyword = "Scenario",
line = scenario.Source.SourceLine + 1,
name = scenario.Title,
tags = scenario.Tags.Select(t => new { name = t, line = 1 }),
steps = from step in lastExecutionResult.Result.TraceEvents
where IsRelevant(step) && (step.ResultType == TestNodeResultType.Succeeded || step.ResultType == TestNodeResultType.Failed || step.ResultType == TestNodeResultType.Pending)
&& (step.Type == TraceEventType.Test || step.Type == TraceEventType.TestAct || step.Type == TraceEventType.TestArrange || step.Type == TraceEventType.TestAssert)
let keyword = step.StepBindingInformation == null ? "" : step.StepBindingInformation.StepInstanceInformation == null ? "" : step.StepBindingInformation.StepInstanceInformation.Keyword
let matchLocation = step.StepBindingInformation == null ? "" : step.StepBindingInformation.MethodName
let name = step.StepBindingInformation == null ? "" : step.StepBindingInformation.Text
let cucumberStatus = step.ResultType == TestNodeResultType.Succeeded ? "Passed" : step.ResultType.ToString()
select new
{
keyword = keyword,
line = 0,
match = new
{
location = matchLocation
},
name = name,
result = new
{
duration = step.Duration.TotalMilliseconds,
error_message = step.StackTrace,
status = cucumberStatus
}
},
type = "scenario"
}).ToList(),
id = "",
keyword = "Feature",
line = f.Source.SourceLine + 1,
tags = f.Tags.Select(t => new { name = t, line = 1 }),
name = f.Title,
uri = f.Source.SourceFile
});
}
@Raw(JsonConvert.SerializeObject(features, Formatting.Indented, serializationSettings)) |
Default.srprofile
<?xml version="1.0" encoding="utf-8"?>
<TestProfile xmlns="http://www.specflow.org/schemas/plus/TestProfile/1.5">
<Settings projectName="TestProject" projectId="{xxx-xxx-xxx-xxx}" />
<Execution stopAfterFailures="2" testThreadCount="1" testSchedulingMode="Sequential" />
<TestAssemblyPaths>
<TestAssemblyPath>TestProject.dll</TestAssemblyPath>
</TestAssemblyPaths>
<DeploymentTransformation>
...
</DeploymentTransformation>
<Report>
<Template name="CucumberJson.cshtml" outputName="data.json"/>
</Report>
</TestProfile> |
Tests can be run from within the IDE (e.g. Visual Studio) or by the command line; in the later case, make sure to specify the profile name and all the paths properly.
Since there is code-behind file generation, it is required to have the NuGet "SpecFlow.Tools.MsBuild.Generation" package.
msbuild /t:Clean;Rebuild
cd bin\debug
..\..\..\packages\SpecRun.Runner.1.7.2\tools\SpecRun.exe run Default.srprofile /outputFolder:..\..\..\TestResults
cd ..\..