NUnit vs. Visual Studio 2008's test projects for unit testing

C#asp.net Mvc Visual Studio-2008 Unit Testing Nunit

C# Problem Overview

I am going to be starting up a new project at work and want to get into unit testing. We will be using Visual Studio 2008, C#, and the ASP.NET MVC stuff. I am looking at using either NUnit or the built-in test projects that Visual Studio 2008 has, but I am open to researching other suggestions. Is one system better than the other or perhaps easier to use/understand than the other?

I am looking to get this project set up as kind of the "best practice" for our development efforts going forward.

C# Solutions

Solution 1 - C#

Daok named all the pro's of Visual Studio 2008 test projects. Here are the pro's of NUnit.

NUnit has a mocking framework.
NUnit can be run outside of the IDE. This can be useful if you want to run tests on a non-Microsoft build server, like CruiseControl.NET.
NUnit has more versions coming out than visual studio. You don't have to wait years for a new version. And you don't have to install a new version of the IDE to get new features.
There are extensions being developed for NUnit, like row-tests, etc.
Visual Studio tests take a long time to start up for some reason. This is better in Visual Studio 2008, but it is still too slow for my taste. Quickly running a test to see if you didn't break something can take too long. NUnit with something like Testdriven.Net to run tests from the IDE is actually much faster. Especially when running single tests. According to Kjetil Klaussen, this is caused by the Visual Studio testrunner. Running MSTest tests in TestDriven.Net makes MSTest performance comparable to NUnit.

Solution 2 - C#

The unit-testing framework doesn't actually matter much, because you can convert test classes with separate project files and conditional compilation (like this, Visual Studio → NUnit):

#if !NUNIT
using Microsoft.VisualStudio.TestTools.UnitTesting;
#else
using NUnit.Framework;
using TestClass = NUnit.Framework.TestFixtureAttribute;
using TestMethod = NUnit.Framework.TestAttribute;
using TestInitialize = NUnit.Framework.SetUpAttribute;
using TestCleanup = NUnit.Framework.TearDownAttribute;
using TestContext = System.String;
using DeploymentItem = NUnit.Framework.DescriptionAttribute;
#endif

The TestDriven.Net plugin is nice and not very expensive... With only plain Visual Studio 2008 you have to find the test from your test class or test list. With TestDriven.Net you can run your test directly from the class that you are testing. After all, unit tests should be easy to maintain and near the developer.

Solution 3 - C#

Benefits/changes of the Visual Studio 2008 built-in unit testing framework:

The 2008 version now is available in professional editions (before it required expensive versions of Visual Studio, and this is just for developer unit testing) that left a lot of developers with the only choice of open/external testing frameworks.
Built-in API supported by a single company.
Use the same tools to to run and create tests (you may run them using the command line also MSTest).
Simple design (granted without a mock framework, but this is a great starting point for many programmers).
Long term support granted (I still remember what happened to NDoc, and I don't want to commit to a testing framework that might not be supported in five years, but I still consider NUnit a great framework).
If using Team Foundation Server as your backend, you can create work items or bugs with the failed test data in a simple fashion.

Solution 4 - C#

I have been using NUnit for two years. All is fine, but I have to say that the unit testing system in Visual Studio is pretty nice, because it's inside the GUI and can more easily do a test for private function without having to mess around.

Also, the unit testing of Visual Studio lets you do covering and other stuff that NUnit alone can't do.

Solution 5 - C#

One slight annoyance of Visual Studio's testing framework is that it will create many test run files that tend to clutter your project directory - though this isn't that big of a deal.

Also, if you lack a plugin such as TestDriven.NET, you cannot debug your NUnit (or MbUnit, xUnit, etc.) unit tests within the Visual Studio environment, as you can with the Microsoft Visual Studio testing framework, which is built in.

Solution 6 - C#

Slightly off-topic, but if you go with NUnit I can recommend using ReSharper - it adds some buttons to the Visual Studio UI that make it a lot easier to run and debug tests from within the IDE.

This review is slightly out-of-date, but explains this in more detail:

Using ReSharper as an essential part of your TDD toolkit

Solution 7 - C#

xUnit is another possibility for a greenfield project. It's got perhaps a more intuitive syntax, but it is not really compatible with the other frameworks.

Solution 8 - C#

My main beef with Visual Studio unit tests over NUnit is the Visual Studio test creation tends to inject a bunch of generated code for private member access.

Some might want to test their private methods, and some may not. That's a different topic.

My concern is when I'm writing unit tests they should be extremely controlled so I know exactly what I'm testing and exactly how I'm testing it. If there's auto generated code I'm losing some of that ownership.

Solution 9 - C#

I have done some TDD using both and (maybe I'm a little dumb) NUnit seems to be a lot faster and simpler to use to me. And when I say a lot, I mean a lot.

In MSTest, there is too many attributes, everywhere - the code that do the real tests is the tiny lines you may read here and there. A big mess. In NUnit, the code that do the test just dominates the attributes, as it should do.

Also, in NUnit, you just have to click on the tests you want to run (only one? All the tests covering a class? An assembly? The solution?). One click. And the window is clear and large. You get clear green and red lights. You really know what happens in one sight.

In VSTS, the test list is jammed in the bottom of the screen, and it's small and ugly. You have to look twice to know what happened. And you cannot run just one test (well, I did not find out yet!).

But I may be wrong, of course - I just read about 21 blog posts about "How to do simple TDD using VSTS". I should have read more; you are right.

For NUnit, I read one. And I was TDDing the same day. With fun.

By the way, I usually love Microsoft products. Visual Studio is really the best tool a developer can buy - but TDD and Work Item management in Visual Studio Team System sucks, really.

Solution 10 - C#

First I want to correct a wrong statement: you can run MSTest outside of Visual Studio using the command line. Although several CI tools, such as TeamCity, have better support for NUnit (probably would change as MSTest becomes more popular).

In my current project we use both and the only big difference we found that MSTest always runs as a 32 bit while NUnit runs as either 32 bit or 64 bit tests which only matters if your code uses native code that is 32/64 bit dependent.

Solution 11 - C#

I got messages that "NUnit file structure is richer than VSTest"... Of course, if you prefer the NUnit file structure, you can use this solution to the other way, like this (NUnit → Visual Studio):

 #if !MSTEST
     using NUnit.Framework;
 #else
     using Microsoft.VisualStudio.TestTools.UnitTesting;
     using TestFixture = Microsoft.VisualStudio.TestTools.UnitTesting.TestClassAttribute;
     using Test = Microsoft.VisualStudio.TestTools.UnitTesting.TestMethodAttribute;
     using SetUp = Microsoft.VisualStudio.TestTools.UnitTesting.TestInitializeAttribute;
     using TearDown = Microsoft.VisualStudio.TestTools.UnitTesting.TestCleanupAttribute;
 #endif

Or any other conversion... :-) This use here is just an alias to the compiler.

Solution 12 - C#

I started with MSTest, but I switched for one simple reason. MSTest does not support inheritance of test methods from other assemblies.

I hated the idea of writing the same test multiple times. Especially on a large project where test methods can easily run into 100's of tests.

NUnit does exactly what I need. The only thing that is missing with NUnit is a Visual Studio addin which has can display the red/green status (like VSTS) of each test.

Solution 13 - C#

If you are considering either MSTest or NUnit, then I recommend you look at MbUnit. My reasons are

TestDriven.Net compatibility. Nothing beats have TestDriven.Net.ReRunWithDebugger bound to a keyboard combination.
The Gallio framework. Gallio is a test runner like NUnit's. The only difference is it doesn't care if you wrote your tests in NUnit, MSTest, xUnit or MbUnit. They all get run.
Compatibility with NUnit. All features in NUnit are supported by MbUnit. I think you don't even need to change your attributes (will have to check that), just your reference and usings.
Collection asserts. MbUnit has more Assert cases, including the CollectionAssert class. Basically you no longer need to write your own tests to see if two collections are the same.
Combinatorial tests. Wouldn't it be cool if you could supply two sets of data and get a test for all the combinations of data? It is in MbUnit.

I originally picked up MbUnit because of its [RowTest ....] functionality, and I haven't found a single reason to go back. I moved all my active test suites over from NUnit and never looked back. Since then I've converted two different development teams over to the benefits.

Solution 14 - C#

As far as I know, there are four frameworks available for unit testing with .NET these days:

NUnit has always been out in front, but the gap has closed in the last year or so. I still prefer NUnit myself, especially as they added a fluent interface a while back which makes tests very readable.

If you're just getting started with unit testing it probably doesn't make much difference. Once you're up to speed, you'll be in a better position to judge which framework is best for your needs.

Solution 15 - C#

I don't like the Visual Studio's built-in testing framework, because it forces you to create a separate project as opposed to having your tests as part of the project you're testing.

Solution 16 - C#

MSTest is essentially NUnit slightly reworked, with a few new features (such as assembly setup and teardown, not just fixture and test level), and missing some of the best bits (such as the new 2.4 constraint syntax). NUnit is more mature, and there is more support for it from other vendors; and of course since it's always been free (whereas MSTest only made it into the Professional version of Visual Studio 2008, and before that it was in way more expensive SKUs), and most ALT.NET projects use it.

Having said that, there are some companies who are incredibly reluctant to use something which does not have the Microsoft label on it, and especially so OSS code. So having an official Microsoft test framework may be the motivation that those companies need to get testing; and let's be honest, it's the testing that matters, not what tool you use (and using Tuomas Hietanen's code, you can almost make your test framework interchangeable).

Solution 17 - C#

With the release in .NET 4.0 of the Code Contracts system and the availability of a static checker, you would need to theoretically write fewer test cases and a tool like Pex will help identify those cases. Relating this to the discussion at hand, if you need to do less with your unit tests because your contracts are covering your tail, then why not just go ahead and use the built-in pieces since that is one less dependency to manage. These days, I am all about simplicity. :-)

Solution 18 - C#

I would prefer to use MS's little test framework, but for now am sticking with NUnit. The problems with MS's are generally (for me)

Shared "tests" file (pointless) that must be maintained
Tests lists cause conflicts with multiple developers / VCSs
Poor integrated UI - confusing setup, burdensome test selection
No good external runner

Caveats

If I were testing an aspx site, I would definitely use MS's
If I were developing solo, also MS would be fine
If I had limited skill and couldn't configure NUnit :)

I find it much easier to just write my tests and fire up NUnitGUI or one of the other front ends (testDriven is far far far far overpriced). Setting up debugging with the commandline version is also pretty easy.

Content Type	Original Author	Original Content on Stackoverflow
Question	Ryan Skarin	View Question on Stackoverflow
Solution 1 - C#	Mendelt	View Answer on Stackoverflow
Solution 2 - C#	Tuomas Hietanen	View Answer on Stackoverflow
Solution 3 - C#	Simara	View Answer on Stackoverflow
Solution 4 - C#	Patrick Desjardins	View Answer on Stackoverflow
Solution 5 - C#	Tarsier	View Answer on Stackoverflow
Solution 6 - C#	Richard Ev	View Answer on Stackoverflow
Solution 7 - C#	Ben Fulton	View Answer on Stackoverflow
Solution 8 - C#	Ian Suttle	View Answer on Stackoverflow
Solution 9 - C#	Sylvain Rodrigue	View Answer on Stackoverflow
Solution 10 - C#	Dror Helper	View Answer on Stackoverflow
Solution 11 - C#	Tuomas Hietanen	View Answer on Stackoverflow
Solution 12 - C#	Th3Fix3r	View Answer on Stackoverflow
Solution 13 - C#	AlSki	View Answer on Stackoverflow
Solution 14 - C#	gilles27	View Answer on Stackoverflow
Solution 15 - C#	Gene Mitelman	View Answer on Stackoverflow
Solution 16 - C#	David Keaveny	View Answer on Stackoverflow
Solution 17 - C#	Norman H	View Answer on Stackoverflow
Solution 18 - C#	Andrew	View Answer on Stackoverflow