Acceptance Criteria, Part 2: Spreading the seeds of knowledge

New Thinking 25 November 2016 Nigel Charman

Part 1 of this series described how discovery and understanding of requirements is improved by combining discussion of generalised acceptance criteria with specific examples. But once you have that level of understanding, how do you capitalise on it and make it a useful asset for maintaining the system?

Traditional projects often build elaborate documentation and models during initial development, but fail to maintain them as the system changes. As personnel come and go, institutional knowledge about what the system does and, more importantly, why it does it, is lost.

The system and the documentation get out of sync and confidence in the documentation degrades. Any traceability to the original requirements is gone forever.

The Specification by Example practice addresses this by including the examples within the specification. By automatically validating these examples against the system on each code change, you ensure that the examples still match the actual system behaviour, or are notified if the two diverge. The specification becomes Living Documentation.

Recently, we have experienced two systems that needed changes to existing features, several years after the features were first developed. Both systems had living documentation that was validated frequently.

The development teams based their discussions with the business around the documentation, with full confidence that the documentation matched the actual system behaviour. The documentation revealed features that the business didn’t know existed, with the examples giving the team the confidence to say “yes it really does that!”.

Creating living documentation

As discussed in Part 1, you gain an early, deep understanding of requirements by collaboratively specifying acceptance criteria and examples. In complex cases, you may find that not all requirements are known and you need to perform additional discovery.

Once the requirements are agreed, you document the story, acceptance criteria and examples in a specification. By explicitly stating the general acceptance criteria and specific examples in the specification, you increase its readability. As David Peterson points out:

"If we don't make the rule explicit we only have the concrete examples, so they have to be made much more verbose—and potentially implementation-specific—so that readers can correctly interpret them."

By including the acceptance criteria in the specification, you no longer need to infer the generalisations from the examples.

One thing to be wary of is polluting the specifications with too much detail. If you find the acceptance criteria getting too wordy, you should seek to simplify them. If the detail really does help the specification, consider creating a separate "additional details" section for it or link to an external document.

During development of the feature, your tester or developer will automate the examples as tests. With an appropriate tool, such as Concordion, Fitnesse or Cucumber, the results of running the tests are included in the documentation. By running these tests frequently, using a Continuous Integration server, you create living documentation that your maintenance team can rely upon.


An example specification might look like:

with additional examples to cover all acceptance criteria.

In this illustration, the Concordion tool executed the specification against the system, validating that the examples are correct and highlighting the results in green. Any failing examples would be shown in red, requiring you to investigate whether the system or tests need to be updated to keep the two in sync.


In summary, discussing acceptance criteria as well as examples will help you better understand the requirements. Documenting both the acceptance criteria and examples helps pass that understanding on to those reading your specifications.

Using automated tools to frequently check these examples against the system creates authoritative, living documentation that increases the team’s ability to maintain and evolve the system with confidence.