SpecFlow integration

Specflow by default generates the output results in SpecFlow JSON format. AssertThat plugin can consume it as Cucumber JSON format. The following solution will generate the output result in Cucumber JSON format in parallel and the result could imported to AssertThat plugin using https://assertthat.atlassian.net/wiki/spaces/ABTM/pages/2208366593 or running a https://assertthat.atlassian.net/wiki/spaces/ABTM/pages/725843969 tool.

Option 1 (.NET 4.5)

Step 1 : Add Dependencies

1 2 https://www.nuget.org/packages/SpecNuts/ https://www.nuget.org/packages/SpecNuts.Json/

Step 2 : Add the below code in .cs of feature file

1 2 3 4 5 6 7 8 9 10 11 12 13 14 [BeforeTestRun] public static void BeforeTestRun() {     SpecNuts.Reporters.Add(new JsonReporter());       SpecNuts.Reporters.FinishedReport += (sender, args) => {           String pathName = "specflow_cucumber.json";           System.IO.File.WriteAllText(pathName, args.Reporter.WriteToString());           Console.WriteLine("Result File: " + System.IO.Directory.GetCurrentDirectory().ToString() + System.IO.Path.DirectorySeparatorChar + pathName);       }; }

Step 3 : Open Text Explorer in Visual Studio by Test > Windows > Test Explorer -> Choose Run All

Step 4 : In Visual Studio console, select show output from Test and open Report File as path given in Console

Step 5 : The generated output result file will be in Cucumber JSON format with the name "specflow_cucumber.json". The generated JSON can be imported into the plugin now.

Option 2

  • Install SpecFlow and the SpecFlow+ Runner 1.7.2 or newer along with the msbuild helper package; If you're using Visual Studio, just go to NuGet's Console  (Tools | NuGet Package Manager | Package Manager Console)

    • Install-Package SpecRun.SpecFlow

    • Install-Package SpecRun.SpecRun

    • Install-Package SpecFlow.Tools.MsBuild.Generation

      packages.config 

      1 2 3 4 5 6 7 8 9 10 <?xml version="1.0" encoding="utf-8"?> <packages>   <package id="Newtonsoft.Json" version="9.0.1" targetFramework="net452" />   <package id="SpecFlow" version="2.3.2" targetFramework="net452" />   <package id="SpecFlow.Tools.MsBuild.Generation" version="2.3.2" targetFramework="net452" />   <package id="SpecRun.Runner" version="1.7.2" targetFramework="net452" />   <package id="SpecRun.SpecFlow" version="1.7.2" targetFramework="net452" />   <package id="SpecRun.SpecFlow.2-3-0" version="1.7.2" targetFramework="net452" />   <package id="System.ValueTuple" version="4.3.0" targetFramework="net452" /> </packages>
  • Use the CucumberJson.cshtml report template provided in this page

Before compiling and running the tests, you have to use a proper SpecFlow report template file in order to generate a valid Cucumber JSON report and you have to configure the test profile to use it.

CucumberJson.cshtml 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 @inherits TechTalk.SpecRun.Framework.Reporting.CustomTemplateBase<TestRunResult> @using System @using System.Collections.Generic @using System.Linq @using System.Globalization @using Newtonsoft.Json @using Newtonsoft.Json.Converters @using TechTalk.SpecRun.Framework @using TechTalk.SpecRun.Framework.Results @using TechTalk.SpecRun.Framework.TestSuiteStructure @using TechTalk.SpecRun.Framework.Tracing @{     var serializationSettings = new JsonSerializerSettings     {         ReferenceLoopHandling = ReferenceLoopHandling.Ignore,         Converters = new List<JsonConverter>() { new StringEnumConverter(false) }     };       var features = GetTextFixtures()         .Select(f => new         {             description = "",             elements = (from scenario in f.SubNodes                         let lastExecutionResult = GetTestItemResult(scenario.GetTestSequence().First()).LastExecutionResult()                         select new                         {                             description = "",                             id = "",                             keyword = "Scenario",                             line = scenario.Source.SourceLine + 1,                             name = scenario.Title,                             tags = scenario.Tags.Select(t => new { name = t, line = 1 }),                             steps = from step in lastExecutionResult.Result.TraceEvents                                     where IsRelevant(step) && (step.ResultType == TestNodeResultType.Succeeded || step.ResultType == TestNodeResultType.Failed || step.ResultType == TestNodeResultType.Pending)                                     && (step.Type == TraceEventType.Test || step.Type == TraceEventType.TestAct || step.Type == TraceEventType.TestArrange || step.Type == TraceEventType.TestAssert)                                     let keyword = step.StepBindingInformation == null ? "" : step.StepBindingInformation.StepInstanceInformation == null ? "" : step.StepBindingInformation.StepInstanceInformation.Keyword                                     let matchLocation = step.StepBindingInformation == null ? "" : step.StepBindingInformation.MethodName                                     let name = step.StepBindingInformation == null ? "" : step.StepBindingInformation.Text                                     let cucumberStatus = step.ResultType == TestNodeResultType.Succeeded ? "Passed" : step.ResultType.ToString()                                     select new                                     {                                         keyword = keyword,                                         line = 0,                                         match = new                                         {                                             location = matchLocation                                         },                                         name = name,                                         result = new                                         {                                             duration = step.Duration.TotalMilliseconds,                                             error_message = step.StackTrace,                                             status = cucumberStatus                                         }                                     },                             type = "scenario"                         }).ToList(),             id = "",             keyword = "Feature",             line = f.Source.SourceLine + 1,             tags = f.Tags.Select(t => new { name = t, line = 1 }),             name = f.Title,             uri = f.Source.SourceFile         }); } @Raw(JsonConvert.SerializeObject(features, Formatting.Indented, serializationSettings))

Default.srprofile 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 <?xml version="1.0" encoding="utf-8"?> <TestProfile xmlns="http://www.specflow.org/schemas/plus/TestProfile/1.5">   <Settings projectName="TestProject" projectId="{xxx-xxx-xxx-xxx}" />   <Execution stopAfterFailures="2" testThreadCount="1" testSchedulingMode="Sequential" />   <TestAssemblyPaths>     <TestAssemblyPath>TestProject.dll</TestAssemblyPath>   </TestAssemblyPaths>   <DeploymentTransformation>    ...   </DeploymentTransformation>     <Report>     <Template name="CucumberJson.cshtml" outputName="data.json"/>   </Report>     </TestProfile>

 

Tests can be run from within the IDE (e.g. Visual Studio) or by the command line; in the later case, make sure to specify the profile name and all the paths properly.

Since there is code-behind file generation, it is required to have the NuGet "SpecFlow.Tools.MsBuild.Generation" package.

1 2 3 4 msbuild /t:Clean;Rebuild cd bin\debug ..\..\..\packages\SpecRun.Runner.1.7.2\tools\SpecRun.exe run Default.srprofile /outputFolder:..\..\..\TestResults cd ..\..