VS2017 + Specflow = challenging!

This week I tried to set up a Specflow acceptance test project in one of the applications I'm working on. I already knew that Specflow support for netcore is work in progress. So thinking myself very clever, I created a project targeting net462 with the assumption that it would JustWork^TM

Of course, it didn't. The code behind files were not generated from the add-in on save 😕

Okay, so I knew you could let Specflow generate the code behind on build, or actually BeforeBuild. Good! Let's try that!

<Import Project="packages\specflow\2.2.1\tools\TechTalk.Specflow.targets" />

that should just.... shit:

The imported project "C:\git\project\packages\specflow\2.2.1\tools\TechTalk.Specflow.targets" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.

Problem #1:

packages\ is not in the solution directory anymore because reasons (hint: VS2017 + new NuGet) 😶. Great, so where do we reference the Specflow targets from? Luckily there is a MSBuild property $(NuGetPackageRoot) that points to that path, sooooooo:

<Import Project="$(NuGetPackageRoot)\specflow\2.2.1\tools\TechTalk.Specflow.targets" />

whoop, project loads again! Let's build!

Hmpf. No tests in the test explorer, NCrunch shows nothing... bugger.

Problem #2:

Code behind files still not being generated. Why didn't it work?

A lot of digging turned up a StackOverflow post that explained why the BeforeBuild wasn't called. TL;DR: in VS2017 the build system uses sensible defaults to reduce the noise in .csproj files. That has the unfortunate side effect that BeforeBuild isn't called, which Specflow depends on.

Cool, applied the import magicking done, let's build:

TechTalk.Specflow.targets(47,5): error : Object reference not set to an instance of an object.

$^$%^#@$@#$^$^ not again!

This one took some digging in the Specflow sources. This little bit of source code spelunking led me to MsBuildProjectReader.cs and the aptly named method IsNewProjectSystem(). The method checks if the root node of the .csproj file contains an Sdk property, which new style projects do!

Oh but I just removed that to fix the BeforeBuild problem.... shit.

I wonder if.... ok yeah, adding Sdk="" still loads the project, let's see if it builds....

TechTalk.Specflow.targets(47,5): error : Object reference not set to an instance of an object.

(╯°□°）╯︵ ┻━┻

More source code spelunking later, it turns out that we need a specflow.json in the project because for VS2017 style projects Specflow is also changing to JSON instead of app.config (see here)

Adding a specflow.json with just {} as it's contents....

TechTalk.Specflow.targets(47,5): error : Object reference not set to an instance of an object.

Double (╯°□°）╯︵ ┻━┻

Ok, contents matter. Digging into JsonConfigurationLoader I created the following specflow.json:

{
	"SpecFlow": {
		"UnitTestProvider": {
			"Name": "xUnit"
		}
	}
}

Hit build.....

BOOM! A wild MyFeature.feature.cs appears!

NCrunch suddenly turns green, TestExplorer shows a test, life is good!

Recap

Pulling this all together, you'll need a .csproj that looks like this:

<Project Sdk="">
  <Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" />

  <PropertyGroup>
    <TargetFramework>net462</TargetFramework>
  </PropertyGroup>
  
  <Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" />
  
  <Import Project="$(NuGetPackageRoot)\specflow\2.2.1\tools\TechTalk.Specflow.targets" />
</Project>

and a specflow.json like this:

{
	"SpecFlow": {
		"UnitTestProvider": {
			"Name": "xUnit"
		}
	}
}

You will get the code behinds to be generated on-build. It's a far cry from the generate-on-save behaviour of the add-in, but at least it gets us going.