Current State / Future Development

[matkoch]

Hi maintainers,

over the years, you put a lot of effort into MSpec gaining high popularity in the community. MSpec really achieved a lot for us, for our code, for documentation. Now it feels more and more like it's dying, mostly due to missing contributions by the community, hard times upgrading the ReSharper runner, and in my honest opinion also code quality. I don't like the fact, that a project of this size, just disappears, and also I know of the pain that goes along with that, like deciding between new ReSharper or keeping the testing framework.

So what is the reason for me writing here? Some of you already know, that I started writing a new test framework quite a long time ago (also emerged from my pains with MSpec). With this framework I was able to easily write a spike which proves that it can run MSpec tests in both ReSharper and console. I know that there are ongoing efforts using the NUnit runner as execution engine. However, this will change some things like the output in the ReSharper runner and the base class of the specs. Therefore, I'm not a big fan of that.

My suggestion / question is therefore, if you would like to participate in this movement. I'm really willing to do good work there. My own project is already 2 years old with a very continuous activity. Everything required for creating the new MSpec implementation can be done by myself; but of course I would also appreciate if someone wants to help. The output will be the same, except for behaviours - they will not have a dedicated node; instead, they're flattened with the other It fields. Besides, I have very good connections that will help for future upgrades of the ReSharper runner. The only thing I really need from you, is your commitment to do this step, and maybe some discussions about little details if you want.

Looking forward to hear from you.

[BrunoJuchli]

Sounds interesting.

TBH, i do believe that a better syntax for spec-testing could be invented. I'm not a complete fan of xBehave either and i'm decidedly against all the complexity SpecFlow brings with it.

May i ask you whether you've investigated running tests based on the xunit 2.0 platform? It seems quite configurable to me, too. Why develop another test platform? I know that currently the R# runner is lacking xunit 2.0 based test-discovery which is required to be able to use the custom testing aspects. But surely that is an obstacle which can be overcome.
Looking at the Test Runner Compatibility list here: https://xunit.github.io/ it seems like there's a ton of work involved in developing and maintaining runner support.

I know that MSpec is not supporting all of these platforms, either ;-)

To come back to the topic of "better spec test syntax": While i'd appreciate there being a runner for the next 2-3 years, supporting state of the art IDE / tools, i think development should head in the direction of a new, improved syntax (shouldn't be called mspec because it's a majorly major breaking change). After 2-3 years, hopefully, users will have changed to the "new" spec-testing tool and MSpec can be left for dead. Your offer seems very helpful in that regard.

MSpec only needs to live on after that point in case there's no better alternative around...

[matkoch]

TBH, i do believe that a better syntax for spec-testing could be invented. I'm not a complete fan of xBehave either and i'm decidedly against all the complexity SpecFlow brings with it.

Here is one possible approach: https://github.com/matkoch/TestFx/wiki/SpecK

May i ask you whether you've investigated running tests based on the xunit 2.0 platform?

As far as I can see, the assumption for tests being members (methods) is inevitable for xunit frameworks (including NUnit and xUnit.net). I almost always stumble across something like ITestMethod and so on. So I don't believe it would be possible. Of course there is always a way, but on the other hand, it's also always combined with effort. And in this particular case, my opinion was that it's worth more to have a dedicated platform with all the freedom.

The initial spike for MSpec with my framework was just ~150 LOC (including ReSharper + Console!). I guess with xUnit that would be much higher, especially considering the ReSharper runner support.

[NameOfTheDragon]

This is an interesting proposal but there isn't enough detail to really come to a decision. Is it possible that you could show us a demo of your framework running MSpec tests?

My concern here is twofold:

1. The original maintainers of MSpec don't want to develop MSpec further but seem unwilling to let go of it completely. This is strangling the project.
2. There hasn't been any thought given to running on CoreCLR so we can test things like UWP apps and ASP.Net vNext. This renders the project somewhat irrelevant in the mid-term.

I'm not willing to let this project die without a fight, so I think something needs to happen; I'm just not sure what at this stage.

[matkoch]

As I already said, this is a spike, so there are few things missing. I only put like half a day in it. The actual MSpec code is, like I said, very small. Missing things are recognizing inherited subject attributes; and visualizing It fields. Both should be possible with little effort

[ivanz]

Just fyi the support for .NET Standard (.Net Core + UWP/UAP + whatever else support .Net Standard 1.3) is in progress and work can be tracked here: #297 and here: #296

[matkoch]

@ivanz: skype?

[NameOfTheDragon]

@ivanz @matkoch You two should really get together and talk about the future of MSpec.

I've worked with Matt a little bit and really like his approach; his TestFX framework holds a lot of promise. However, he currently has no support for .Net Core which is a showstopper for me. Unfortunately I don't really have the skills to help make it happen, but I think that migrating MSpec onto the TestFX framework would really make life simpler in the future, because the R# test runner and other runner support all then comes for free. I've tried Matt's spike and it works! It would be excellent if you two could join forces and make that happen. It's got to be worth a look.

I'd like to be able to run ASP.NET CORE tests in R# runner but I also develop stuff for Windows IoT Core running on Raspberry Pi so it would also be useful to have a "device runner" similar to https://github.com/xunit/devices.xunit - I would be willing to work on that if the Core support becomes available.

[ivanz]

The dotnet cli (dotnet test) defines a simple generic IPC based test adapter protocol for both run-time and design-time (e.g. test discovery). If (when?) ReSharper integrates with that - then it will by default support all possible testing frameworks.

I will have a look at TestFX.

I have now just completed the update of Machine.Specifications to make it compatible with .NET Core/.Net Standard. "dotnet test" support for .NET (full) and .NET Core works fine too (no design-time support). There is only a handful of small things left to do and one nuget packaging issue to resolve before I can ship a beta of 0.10. I will then update Machine.Specifications.Should. That's my only aim at this point for MSpec.

[NameOfTheDragon]

While you’re looking at Machine.Specifications.Should, could you have a look at whether it is possible to break the dependency on SpecificationException ? I believe that is the only runtime dependency between the two packages now, and it leads to an awkward “backwards” dependency in the NuGet packages.

Thanks,
Tim

From: Ivan Zlatev [mailto:notifications@github.com]
Sent: 28 July 2016 23:58
To: machine/machine.specifications machine.specifications@noreply.github.com
Cc: Tim Long Tim@tigranetworks.co.uk; Comment comment@noreply.github.com
Subject: Re: [machine/machine.specifications] Current State / Future Development (#286)

The dotnet cli (dotnet test) defines a simple generic IPC based test adapter protocolhttps://docs.microsoft.com/en-us/dotnet/articles/core/tools/test-protocol for both run-time and design-time (e.g. test discovery). If (when?) ReSharper integrates with that - then it will by default support all possible testing frameworks.

I will have a look at TestFX.

I have now just completed the update of Machine.Specifications to make it compatible with .NET Core/.Net Standard. "dotnet test" support for .NET (full) and .NET Core works fine too (no design-time support). There is only a handful of small things left to do and one nuget packaging issue to resolve before I can ship a beta of 0.10. I will then update Machine.Specifications.Should. That's my only aim at this point for MSpec.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com//issues/286#issuecomment-236049831, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABNbCsoz4TEbua4CdYyo_6hl-6mO2BSzks5qaTQGgaJpZM4Glq_R.