by Gary E. Mogyorodi
This article provides an overview of the Requirements-Based Testing (RBT) process. RBT is a rigorous process for improving the quality of requirements and for deriving the minimum number of test cases to cover 100% of those requirements. RBT is comprised of two techniques: Ambiguity Reviews and Cause-Effect Graphing.
An Ambiguity Review is used in the requirements phase of software development to identify ambiguities in functional1 requirements. The intent of an Ambiguity Review is to identify anything that is unclear, ambiguous or incomplete in the requirements. The elimination of these ambiguities improves the quality of those requirements.
Cause-Effect Graphing is a test case design technique that is performed once requirements have been reviewed for ambiguity, and after they have been reviewed for content. Requirements are reviewed for content to insure that they are correct and complete. The Cause-Effect Graphing technique derives the minimum number of test cases to cover 100% of the functional requirements to improve the quality of test coverage.
This article addresses the first RBT technique – the Ambiguity Review. A follow-on article will discuss the Cause-Effect Graphing technique.
So What is it about Requirements?
A requirements document describes what the behavior of a software system is expected to be. It is the starting point for all remaining phases of software development. Requirements should be written in a detailed manner so that there is no misunderstanding regarding what the expected system should do. I call this a testable requirement. Too often, not enough time is spent on the requirements document, so when the requirements are written, there is confusion as to:
1.what the stakeholders expect will be delivered
2. what the developers are expected to deliver
3. what the testers are expected to test
Therefore, it would benefit the entire project team if there is one clear, detailed, testable set of requirements that they can work from.
Why should we care about having Testable Requirements?
Testable requirements reduce development time by avoiding costly defects in later phases of the software development life cycle.
The Relative Cost to Fix an Error
The cost of fixing a software error is lowest in the requirements phase of the software development life cycle. This is because there are very few deliverables at the beginning of a project to correct if an error is found. As the project moves into subsequent phases of software development, the cost of fixing an error rises dramatically, since there are more deliverables affected by the correction of each error, such as a design document or the code itself. See Table 1 for the relative cost to fix an error at different phases in the development life cycle.
The Distribution of Bugs
A study by James Martin showed that the root causes of 56% of all of the software bugs identified in projects are a result of errors introduced in the requirements phase. Of the bugs rooted in requirements, roughly half of them are due to poorly written, ambiguous, unclear and incorrect requirements. The remaining half of these bugs are due to requirements that were completely omitted. So there is plenty of room for improvement for writing clear, concise, unambiguous and complete requirements.
What’s an Ambiguity Review?
An Ambiguity Review is a test of the requirements to ensure that they are written in a clear, concise and unambiguous manner. An Ambiguity Review occurs at the earliest phase of development, i.e. in the requirements phase. At this point in a project, all the project team has is a description of what the system is intended to do. The Ambiguity Review occurs prior to the review of requirements for content by the domain experts. The intent of the Ambiguity Review is to provide the domain experts with a better-quality set of requirements to work from, so they can better identify missing requirements, and improve the content (completeness and accuracy) of all requirements. The Ambiguity Review is based on a checklist of 15 common problems people have writing requirements.
Let’s Do an Ambiguity Review
An example is often a good way to illustrate a technique. I will use a requirements document entitled “The Postal Regulation” to illustrate the use of an Ambiguity Review. The draft version of the Postal Regulation requirements is provided in the box below.
The first step in an Ambiguity Review is to identify any words or phrases that are potentially ambiguous. I have highlighted a number of words that are ambiguities, and we will take a closer look at each one.
1. The word “other” in the phrase “other types of postal items” does not clearly define what types of postal items there are other than parcels, so we would want to get clarification as to what these other types of postal items are.
2. The word “will” identifies an ambiguity called a “Dangling Else”. The sentence with “will” in it tells us what is normally expected to happen, i.e., that the Country & Weight Surcharge is applied for parcels that people mail to South America between December 1 and December 24. But the requirements do not tell us what happens for any other set of conditions. For example, the requirements do not tell us what the postage surcharges are if the parcels are mailed to South America but NOT between December 1 and December 24? So we have a dangling else.
3. The word “these” is a pronoun that makes a reference to what surcharges the system will apply. It would be clearer to make the reference explicit, such as the surcharges in Table 1, and eliminate the word “these”.
4. The word “standard” does not clearly define the postage amount prior to the application of postage surcharges being applied. The author of the requirements probably understands what standard postage means. The sentence is written as if the author assumes that the reader knows what standard postage means, which may not be true.
The next step in the Ambiguity Review is to review the requirements and identify anything else that is ambiguous. From reviewing the requirements, the following additional Ambiguity Review questions can be asked:
• Is there such a thing as non-standard postage?
•What postage surcharges are applied to parcels mailed to South America between December 1 and December 24 going to Brazil that weigh exactly 33 pounds?
•What postage surcharges are applied to South American countries other than Argentina or Brazil?
•What currency are the postage surcharges in?
•What is the postage surcharge for postal items other than parcels?
After getting clarification from the domain experts on these issues, the ambiguities are eliminated and the requirements are revised (see Figure
3) to look like this (new text in red letters):
The goal is to derive a set of requirements that are testable. The requirement is testable if it is written in such a way that each statement is deterministic. Deterministic means that for a given starting condition and a set of inputs, the reader can determine exactly what the expected outcomes will be. Each statement in the requirements can then be used to prove or disprove whether the behavior of the software is correct. Eliminating ambiguities is the first step in deriving a testable requirement.
Once the requirements have been reviewed and corrected for ambiguity, they can be reviewed by the domain experts to ensure that they are correct and complete. Then the requirements are corrected for any errors or omissions.
Now the development team can move into the Design Phase of the software development life cycle with a complete set of requirements. At the same time, the testing team can begin test case design. In my next article, I will pick up from where we left off and show the use of the Cause-Effect Graphing technique to derive the minimum number of test cases that cover 100% of the functionality defined in the requirements for the Postal Regulation (for South America) problem.