On this page
- What You Should Know
- Why Isolate - Focus on Unit Tests
- Motivations for isolation
- Motivations against isolation
- How to Isolate - Substitution Patterns and Test Doubles
- Substituting - Dependency Injection and Dependency Lookup
- Substituting - Overriding and Overwriting
- Substituting - Test Hocks
- How to Test - Applying Testing Knowledge
- Unit Test Organization
- Same Project
- Same Package
- Test Case Naming
- Test Method Naming
- Summary - Test Infected Teams
- References
Agile Unit Testing
What You Should Know
This book page is not for developers that are new to application testing. Therefore, it does not describe unit testing concepts and techniques. The following list describes the level of experience or knowledge required to understand this book:
- Familiarity with unit tests, programming on PHP 5.x with the corresponding development tools for that platform, and the process of writing tests in general.
- A general understanding of the structure and concept of xUnit testing frameworks; the object model; dependency management (composition, inheritance patterns) including database mocking.
Agile testing combines engineering with testing activities into one activity and can be expressed by following Test Driven Development where a developer would write tests first. In particular, this book page focuses on unit testing where units represent objects or single architectural layers in contrast to functional testing (customer tests) where a unit represents the complete application (many architectural layers). Functional testing is also vitally important to Drupal but the focus of a separate book page.
Why Isolate - Focus on Unit Tests
Most objects of a system depend on other objects (depended-on objects). Objects contain other objects and delegate work to them. When unit testing one object the decision is up to the developer to touch the depended-on object with the same unit test or substitute the depended-on object with an alternative object (test double).
The motivations to substitute (isolate) depended-on objects with test double are many-fold and have to be contrasted with the motivations against it.
Motivations for isolation
Motivations for isolation (concrete depended-on objects are substituted with test doubles) are:
- Simulate hard to test behaviour If the object under test contains behaviour that is hard to simulate in a unit testing environment (i.e. specific error conditions, or asynchronous events), a substituted test double could simulate it and could allow more test scenarios and faster test runs.
- Granular defect localization If an implementation changes and a test flags up an unexpected result, it is easy to localize the defect if the object under test is isolated from its environment.
- No triggering of unwanted behaviour Without any isolation, the behaviour of the object under test might trigger behaviour on the depended-on object that causes other unexpected effects in unit tests. By substituting the depended-on object, the behaviour the object under test triggers is controlled.
- Less setup code to satisfy dependencies of depended-on objects Without any isolation, depended-on objects might require further dependencies in order to execute without errors. These dependencies would need to be satisfied by the unit test, increasing the knowledge of the unit test unnecessarily. Would the object under test use test doubles instead of concrete depended-on objects, the test doubles can be designed not to require further dependencies, reducing setup code and making the test easier to read and maintain.
- Higher coverage through testing interactions In order to test how an object under test interacts with depended-on objects, the unit test needs access to the depended-on objects. Test doubles can be designed to offer flexibility on how unit tests can observe the interaction with their object under test (mocks). Depended-on objects are often not designed for it, and developers are in danger of sacrificing the encapsulation of a concrete implementation of a depended-on object.
- Code coverage numbers are more significant When concrete depended-on objects are substituted, they don't get coverage through unit tests that focus on other objects. Code coverage tools can easier identify the need for additional tests of the concrete depended-on objects when no code coverage on concrete depended-on objects has been registered via other unit tests.
Motivations against isolation
When concrete depended-on objects are executed by the object under test, then isolation is not performed and the motivations against isolation can be:
- Over-specified unit tests hinder refactorings If all interactions with depended-on objects are tested, refactorings on objects under test that change any of the interactions also need to change the unit tests and this increases the overall effort of refactorings. To counter this danger, unit tests should only focus to test one piece of behaviour and not repeat tests of other tests and not test interactions that are not significant.
- More difficult to write and read unit tests If the object under test has to create test doubles for all its depended-on objects, it increases the size of the unit test and makes it more difficult to read. However, automatic test double creation libraries such as Mockery or PHPUnit ease the time it takes to write test doubles.
- Higher code coverage with fewer tests While objects under test might not be able to and want to test all interactions with depended-on objects, they automatically cover some behaviour of depended-on objects and can achieve a higher code coverage with fewer tests. However, the higher code coverage through unit tests that triggered code of depended-on objects might lead to a false sense of security as the focus of the unit test was not on the depended-on object but on the object under test.
- The danger of substituting behaviour to test When isolating too many developers might accidentally substitute objects they actually would want to test and the risk of integration increases. Watch code coverage results to ensure behaviour continues to be tested. The more depended-on objects the unit test touches, the more it steers testing efforts away from unit testing towards functional testing. At some point, different tooling than unit testing frameworks, additional qualified personnel such as Quality Engineers and separate test runners are more appropriate for complete functional and black box testing (See Agile Functional Testing).
How to Isolate - Substitution Patterns and Test Doubles
Isolation can be achieved by substituting the depended-on object with an object that is controlled by the unit test suite; a test double. The test double takes the same interface as the production object but does have a different implementation, usually much simplified and targeted for a particular test case or suite.
Test doubles can come in many flavours. The most important test doubles are stubs and mocks. A stub directs inputs into the object under test. A mock object observes outputs of the object under test.
Substituting - Dependency Injection and Dependency Lookup
Two of the most popular approaches to get substituted behaviour into objects under test is dependency injection and dependency lookup. Dependency injection injects the test double via the API of the object under test. Dependency lookup retrieves the test double via another object. The other "locator" object should then provide some mechanism to configure it during unit tests with a test double.
Here is an example of an object that might use dependency injection substitution:
<?php
class KittenService {
public function __construct(QueryInterface $query) {
$this->query = $query;
}
public function doWhatever() {
// ...
if (/* $condition */) {
// ...
$this->query->execute();
}
else {
// ...
$this->query->abort();
}
}
}
// We want to test abort function was called so we inject a mocked object.
$mock = new QueryMock(); // Implements QueryInterface.
$service = new KittenService($mock);
$service->doWhatever();
$this->assertTrue($mock->abortCalled(), 'Abort function was called.');
?>
Here is the same example of an object that might use dependency lookup substitution:
<?php
class KittenService {
public function doWhatever() {
// ...
if (/* $condition */) {
// ...
db_query()->execute();
}
else {
// ...
db_query()->abort();
}
}
}
// We want to test abort function was called so we set the service locator to return our mocked object.
db_query_set(new QueryMock()); // Implements QueryInterface.
$service = new KittenService();
$service->doWhatever();
$this->assertTrue(db_query()->abortCalled(), 'Abort function was called.');
?>
Substituting - Overriding and Overwriting
Substitution by overriding can be achieved with subclassing the depended-on object with an object only used in tests that provide an alternative implementation for the behaviour, not of interest in unit tests of the object under test.
Substitution by overwriting simply sets (overwrites) a depended-on object with a test double. However, this is only feasible if the depended-on object is exposed via an API of the object under test and typed as an interface or common base class.
Here is an example of an object that might use overriding substitution:
<?php
class KittenService {
public function __construct(QueryInterface $query) {
$this->query = $query;
}
public function doWhatever() {
if ($result = $this->query->execute()) {
// Do whatever...
}
else {
throw MyAPICheckedException();
}
}
}
class QueryStub extends Query {
// Simulates the query execution failed.
public function execute() {
return FALSE;
}
}
// We want to test the API function throws an exception when the db fails.
$stub = new QueryStub(); // Extends Query.
$service = new KittenService($stub);
try {
$service->doWhatever();
$this->fail('The query passed.');
catch (MyAPICheckedException $e) {
$this->pass('An exception was thrown.');
}
?>
Here is the same example of an object that might use overwriting substitution:
<?php
class KittenService {
public function __construct(QueryInterface $query) {
$this->query = $query;
}
public function doWhatever() {
if ($result = $this->query->execute()) {
// Do whatever...
}
else {
throw MyAPICheckedException();
}
}
}
class QueryStub implements QueryInterface {
// Implements full interface.
// Simulates the query execution failed.
public function execute() {
return FALSE;
}
}
// We want to test the API function throws an exception when the db fails.
$stub = new QueryStub(); // Implements QueryInterface.
$service = new KittenService($stub);
try {
$service->doWhatever();
$this->fail('The query passed.');
catch (MyAPICheckedException $e) {
$this->pass('An exception was thrown.');
}
?>
This is different as the QueryStub has to implements the full interface and might be cumbersome to maintain, but does not require the dependencies of the Query object in the former example and offers the full flexibility over the test-double behaviour.
Substituting - Test Hocks
A Test Hock is a control point (i.e. if-statement) within the object under test that decides to behave differently based on if it's being exercised during unit tests or during production. This can require a known object with a global state to tell objects if they are in unit tests or not. However, instead of hardcoding a conditional statement in the production code, this could also be done configuring a dependency injection container. In any case, this approach adds risk, size and complexity to production code that would not need to be there following the approaches above and should be used with caution.
<?php
if (drupal_valid_test_ua()) {
//
}
?>
Example of a Drupal Test Hock.
How to Test - Applying Testing Knowledge
Unit Test Organization
While there are many solutions to how to organize unit tests, Drupal attempts to provide a convention to improve consistency between projects.
Same Project
See PHPUnit file structure, namespace, and required metadata
Same Package
See PHPUnit file structure, namespace, and required metadata
Test Case Naming
See PHPUnit file structure, namespace, and required metadata
Test Method Naming
Self-documenting tests i.e. should or Given/When/Then naming.
http://dannorth.net/introducing-bdd
Writing Behaviour Driven Test for Drupal
Summary - Test Infected Teams
Test-infected developers never write their tests days after their code. Test-infected developers want to write tests because that's the way they think about software development. They don't want to think otherwise. Test-infected developers never have excuses not to test. They are never too busy to test, their environments never take up too much time to create test data, and their customer never complains that testing is too expensive because it takes too much time.
If it is difficult to create an environment of test infected developers,
- analyze the design of the application for testable code, prevent repetitive and over-specified tests, keep tests small, easy and close to the way objects are used in production.
- don't attempt to test every single line of code in your module. Realize that functional tests have a role. Focus your unit tests to test behaviour and not structure and wiring.
- ensure libraries, frameworks, API projects, any other code that is shared by multiple developers achieve the highest coverage and quality of tests.
- keep quality of test code as high as application code. Maintain tests with refactorings but realize that it's sometimes easier to throw away and rewrite tests as it's sometimes easier to throw away and rewrite code.
References
This page book is taken from http://sourceforge.net/adobe/cairngorm/wiki/BestPracticesAgileUnitTesting, where you may have a look at the original paper. All the credits should go to them.
Help improve this page
You can:
- Log in, click Edit, and edit this page
- Log in, click Discuss, update the Page status value, and suggest an improvement
- Log in and create a Documentation issue with your suggestion