A shot at definitions beyond “unit test”

When doing research on which kinds of programmatic tests different developers and companies utilize and how they handle them, I realized that there is no common definition of terms and concepts. While most sources agree on what is and what is not a unit test, there are various contradictory definitions of what a test is, if it is not a unit test. In this blog post I’d like to present a brief overview of the definitions we are currently using. Since we steadily try to enhance and refine our development process and tools, the terms and concepts presented here are almost certain to change in the future.

Please note that this post is not intended to fully describe all the details of the different test approaches, but rather to give an idea and first impression on how we distinguish them.

Unit Tests

The most basic kind of programmatic tests, unit-tests, are likely to be the most commonly used kind of test. They help to determine that a small piece of code, e.g. a single method or class, behaves as intended by its developer. If properly applied, unit-tests provide a solid foundation to build an application upon. Figure 1 schematically depicts the scope of a unit-test in an exemplary software system.

Depending on the complexity of the tested system, techniques like mocking of dependencies may be required. Especially system resources need to be replaced by mocks, since unit tests need to be completely independent from them (Michael Feathers describes this and some other requirements of unit tests in his blog post “A Set of Unit Testing Rules”). Furthermore, unit tests are not meant to be long running, but instead have to execute within a split second.

Schematic view of a unit test of a component in an exemplary system

Figure 1: Schematic view of a unit test of an component in an exemplary system

Integration Tests

A more sophisticated approach to testing are integration tests which challenge a part or sub-system of an application made up of several units in order to determine whether these units properly cooperate. In contrast to unit tests, integration tests may include system resources and may also determine the test’s outcome by checking the state of these resources. This larger scope and the fact that the tested functionality is typically made up of several actions, leads to integration tests taking a multitude of the time taken by unit tests. Figure 2 schematically illustrates an integration test’s view on an exemplary system.

Schematic of an integration test in an exemplary system

Figure 2: Schematic of an integration test in an exemplary system

Acceptance Tests

The by far most involved technique to test the behavior of an application is the utilization of acceptance tests. While the other approaches challenge only parts of an application, acceptance tests are meant to challenge the application as a whole from a user’s point of view. This includes using system resources, as well as to control the application and verify its proper function as a user would: Through its (G)UI and without knowing anything about the internals of the software.

Schematic of an acceptance test in an exemplary system

Figure 3: Schematic of an acceptance test in an exemplary system

Conclusion

While some developers only distinguish between unit tests and other tests, defining the latter ones more clearly proved very useful when creating, using and explaining them to other developers and customers. Yet, these definitions are not carved in stone and certainly need to be refined over time. Thus, I would like to get to know your opinion on these definitions. Do you agree or do you have a completely different way of distinguishing between test approaches? How many kinds of tests do you distinguish? And why do you do so?

About these ads

3 Responses to A shot at definitions beyond “unit test”

  1. [...] the system behaves (or not) and document it at the same time, I decided to implement the interface integration test [...]

  2. […] Uncle Bob’s presentation, we discussed different test categorization schemes. We’ve invented our own, but there is also a widely used definition from the International Software Testing Qualifications […]

  3. […] Die in der Vorlesung vorgestellte Abgrenzung der Test-Typen ist nur eine von vielen möglichen. Hier ist die Abgrenzung nochmal detailliert beschrieben: http://schneide.wordpress.com/2011/09/05/a-shot-at-definitions-beyond-unit-test/ […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 76 other followers

%d bloggers like this: