Zero, One, or More Testable Requirements

Silvie   Spreeuwenberg
Silvie Spreeuwenberg Founder / Director, LibRT Read Author Bio || Read All Articles by Silvie Spreeuwenberg

There was a project, there was a deadline, the software was delivered late, it was the holiday season, and the person responsible for the system integration test was on holiday.  So I was asked to perform a system integration test of a new protocol against the requirements set for the protocol.  According to the still-prevailing practice in many organizations today, the requirements were written in, and made available as, a plain text document.

It struck me that many of the requirements were not testable.

For example, I found the requirement:

A configuration message includes for each service a list with 0, 1, or more objects that are involved with the execution of the service.

Well, no matter what test I designed, the requirement would always be met, while a test should potentially have either a positive or a negative outcome.  I wanted to change the requirement to something like this:

A service in a configuration message must include a list with the objects that are involved in the execution of the service.

If no objects were involved with the execution of the service this rule would obviously not apply so the zero situation would be covered as well.  But that was not my assignment, and I would do what I normally do:  write business rules.

But I had a bad feeling about this situation because I like to do a good job.  What was the cause?  Were these requirements bad?  Should we update the requirements document?  Well, when the tester updates the requirements we then need to have a new tester to test the updates of the tester … that does not make any sense.

Given my background, my reaction was not surprising.  I am used to working with business rules and I always check that they are testable.  But does that also apply to requirements?  Apparently not.

A survey of the literature and some google searches confirmed my intuition.  While business rules are required to be testable,[1] a requirement may be testable.  There are "tips for writing testable requirements"[2] and there are criteria to distinguish between testable and non-testable requirements.[3]  However, a requirement does need to be verifiable,[4] being defined as:

The implementation of the requirement can be determined through basic possible methods:  inspection, demonstration, test (instrumented), or analysis (to include validated modeling & simulation).

Does this confirm that requirements and rules are just very different things as most experts in both communities agree?  Or can we learn something from this example?  When I read the requirement I couldn't resist asking myself what the intent of the author must have been.  I believe the author must have intended to say:

The system must be able to process a list of the objects that are involved in the execution of the service.

which is much more precise AND testable — although the word 'process' is much too generic in my opinion.  What does it mean?  To be fair, I don't know exactly, so that's why I started with that word.  But if I were the analyst this would be on my list to find out.

Also, it would be good if this statement could be complemented with a business rule stating when the optional object is mandatory and what objects to expect:

A service in a configuration message must include a list with all objects that are involved in the execution of the service.

Now, being a business rules expert (and thus very biased!), I believe these latter two statements are simply more precise than the original.  And isn't that what would be best in all situations related to system development? 

The irony is that I was inspired and influenced some ten years ago by the writing guidelines in The Handbook of Ambiguity[5] which is all about requirements.  We have used these guidelines and further restricted business rule authors by suggesting rule patterns and promoting rigid knowledge representation techniques like decision tables.  And now I conclude that requirements engineers (or authors) should learn from our experience with these more controlled natural languages.

Here are some resources for further reading:

  • The Handbook of Ambiguity is a great source of examples about the consequence of loosely-written requirements.

  • SBVR Structured English,[6] RuleSpeak,[7] and TableSpeak[8] are examples of controlled natural languages developed in the business rules community.  Many organizations use a subset or extension of these language patterns.

  • Do you want only one page to start with?  From my website, you can download and print (and follow) these most important guidelines for avoiding ambiguity.[9]

References

[1]  The IIBA's BABOK Guide:  Appendix A - Glossary defines business rule as:  a specific, actionable, testable directive that is under the control of the business and supports a business policy.  URL:  http://www.iiba.org/babok-guide/babok-guide-online/appendix-a-glossary.aspxreturn to article

[2]  From a presentation on the CQAA website.  URL:  www.cqaa.org/Resources/Documents/Presentations%202010/Writing%20Testable%20Requirements.pdf return to article

[3]  From Wikipedia, on "Software testability":  http://en.wikipedia.org/wiki/Software_testability return to article

[4]  From Wikipedia, on "Requirement":  http://en.wikipedia.org/wiki/Requirement return to article

[5]  Dr, Daniel M. Berry, From Contract Drafting to Software Specification: Linguistic Sources of Ambiguity, (The Ambiguity Handbook).  URL:  https://cs.uwaterloo.ca/~dberry/handbook/ambiguityHandbook.pdf return to article

[6]  "Annex A - SBVR Structured English," Semantics of Business Vocabulary and Business Rules (SBVR) v1.2, Object Management Group (Nov. 2013).  URL: http://www.omg.org/spec/SBVR/1.2/ return to article

[7]  Business Rule Solutions, RuleSpeak® Practitioner Kits, Business Rule Solutions, LLC, 1996-2009.  URL: http://www.RuleSpeak.com return to article

[8]  Business Rule Solutions, Decision Tables - A Primer: How to Use TableSpeak™, Business Rule Solutions, LLC.  URL: https://www.brsolutions.com/b_ipspeakprimers_toc2.php return to article

[9]  Silvie's website.  URL: http://www.silviespreeuwenberg.com/checklist/ return to article

# # #

Standard citation for this article:


citations icon
Silvie Spreeuwenberg , "Zero, One, or More Testable Requirements" Business Rules Journal, Vol. 15, No. 12, (Dec. 2014)
URL: http://www.brcommunity.com/a2014/b788.html

About our Contributor:


Silvie   Spreeuwenberg
Silvie Spreeuwenberg Founder / Director, LibRT

Silvie Spreeuwenberg has a background in artificial intelligence and is the co-founder and director of LibRT. With LibRT, she helps clients draft business rules in the most efficient and effective way possible. Her clients are characterized by a need for agility and excellence in executing their unique business strategy or policy. Silvie's experience has resulted in the development of tools and techniques to increase the quality of business rules. She writes, "We believe that one should focus on quality management of business rules to make full profit of the business rules approach." LibRT is located in the Netherlands; for more information visit www.silviespreeuwenberg.com & www.librt.com

Read All Articles by Silvie Spreeuwenberg
Subscribe to the eBRJ Newsletter
In The Spotlight
 John A. Zachman
';
The Issue Is THE ENTERPRISE By John A. Zachman Jan. 2017 | Vol. 18, Iss. 1
 Jim  Sinur

Online Interactive Training Series

In response to a great many requests, Business Rule Solutions now offers at-a-distance learning options. No travel, no backlogs, no hassles. Same great instructors, but with schedules, content and pricing designed to meet the special needs of busy professionals.