Unit Tests vs. Acceptance Tests

Are you for one or the other? Or both?

My understanding is unit tests:

• validate the system from the developer's point of view
• help developers practice TDD
• keep code modular
• assist in detecting errors at low levels of granularity

Acceptance tests:

• validate the system from the business and QC / QA points of view
• tend to be high level as they're often written by people not familiar with the inner workings of the code

I feel both are necessary. However, for minimization of redundant work, is it a good idea to try to incorporate unit tests into acceptance tests? In other words, have the latter call the former. Does going in the opposite direction make any sense?

What are your thoughts in general on unit tests vs. acceptance tests, and how to manage them in relation to each other?

Acceptance and integration tests tell you whether your code is working and complete; unit tests tell you where it's failing.

If you've done a good job with acceptance and integration tests, and they're passing, your code is implementing all the functionality it's supposed to, and it's working. That's great to know (it's also great to know that it isn't). But if it isn't working, an acceptance test won't give you much insight into what has gone wrong; since it tests many units of functionality, it can be kind of a bird's-eye view of failure. This is where unit tests shine. Good unit tests tell you exactly what went wrong, with exactly what part of your code. It's harder to know whether you've written enough unit tests than acceptance tests, but when you have a failing acceptance test without a corresponding failing unit test - it's time to write that unit test.

That is all from the testing perspective. And, of course, TDD isn't (and ATDD isn't) about testing. With respect to driving your design, acceptance tests give you a broad roadmap ("here's where you want to go") while unit tests take you to the next intersection ("turn left"). They're both valuable in this regard and, again, their value complement one another.

Don't confuse them; don't miscegenate them. Unit tests, in particular, shouldn't depend on anything else, and it would be a mistake to constrain your unit tests by making acceptance test dependent on them. Of course they can share some framework code, but they should be independent.

As a summary of all the above,

• Acceptance tests make sure that you're building the right thing
• Unit tests make sure that you're building the thing right

> However, for minimization of redundant work, is it a good idea to try to incorporate unit tests into acceptance tests?

No.

> In other words, have the latter [acceptance] call the former [unit]. Does going in the opposite direction make any sense?

Don't bother.

Acceptance tests are often political. You show them to people who -- based on their gut instinct -- decide to accept or reject.

Then you argue about the validity of the acceptance tests.

Then you argue about the scope of work and the next release.

Acceptance tests aren't -- generally -- technical. If they were, then you'd have formal unit tests and that would be that.

Don't try to finesse the politics. Embrace it. Let it happen.

---

You can hope that Acceptance Test-Driven Development (ATDD) leads to "acceptance tests are written and agreed upon by the entire team before development begins." But you have to reflect the reality that anything written in advance is specious at best and negotiable at worst.

The premise behind all Agile methods is that you can only agree to get to something releasable. Everything after that is negotiable.

The premise behind all test-first (TDD, ATDD, or anything else) is that a test is an iron-clad agreement. Except it's not. With any TDD (or ATDD) method you can agree -- in principle -- to the test results, but you haven't really agreed to the test itself.

It may arise that the test cannot easily be written. Or worse, cannot be written at all. You may agree to results that seem testable, but turn out to be poorly-defined. What now? These are things you can't know until you start development and get to details.

All testing is important. And no particular kind of testing can be a superset or subset of any other kind testing. They're always partially overlapping sets. Trying to combine to somehow save some work is likely to turn out to be a waste of time.

More testing is better than anything else. The union of all tests has more value than trying to force a subset-superset relationship among tests.

Unit Tests - my specific function does what it's supposed to do, nothing more, and nothing less.

Acceptance Test - my application does what it's supposed to do.

Example: Application to calculate roots of quadratic functions. Takes inputs of a, b, and c, returns roots x1 and x2. This application is built by functions I write to add two numbers, subtract two numbers, multiply two numbers, divide two numbers, and take the square root of two numbers.

Unit tests - check that my divide and multiply functions work correctly, my square root works correctly, my add and subtract work correctly.

Acceptance tests - check that my application calculates roots of quadratic functions.

Since my entire application is to calculate roots, I should not have a unit test that also calculates roots because there is no individual function which does so.

These are just my personal opinion on the issue of some kinds of tests:

> However, for minimization of redundant > work, is it a good idea to try to > incorporate unit tests into acceptance > tests?

I'd second S. Lott's no on this and add that there is a danger here of the unit tests being rigged to some extent that may have some bugs pass through. For example, on a drop down someone may test a few states but likely not all of them, where a tester may use different data to uncover a potential bug.

> In other words, have the latter call > the former. Does going in the opposite > direction make any sense?

I'd be careful of ever coupling them together. The unit tests represent tests of the smallest bits of functionality, often so small that an end-user wouldn't understand that there may be hundreds of tests just to get a web form to enter data into a CRM system. Acceptance tests are more about what the user of the application wants which can be more subjective,e.g. "Does this look pretty?" versus "Does this look right?" There can be that mark of "good enough" with acceptance tests that I'm not sure would work with unit tests. Generally if a unit test fails, then someone has to decide either to fix the code or remove the test as each can be a good option depending on circumstance.

> What are your thoughts in general on > unit tests vs. acceptance tests, and > how to manage them in relation to each > other?

Unit tests are about verifying the simplest pieces of code. There can be integration tests but this is a level higher as once all the little pieces are checked, do the combination of the pieces work together,e.g. there were Saturday morning cartoons I watched growing up that had toys one could put together like "Voltron" or various Transformers like the Constructicons that formed Devastator. Acceptance tests are generally from an end user perspective of, "Can I do X with the application now?" having an answer of "Yes," before something goes out the door. While some error cases may be checked in an acceptance test, it isn't common to do a thorough test of every possible combination that one could enter into an application. However, unit tests may cover boundary conditions and a few other random-like cases.