When someone gives you a requirements specification how do you know whether the specification is complete and unambiguous? How do you know whether the specification is fit for its intended purpose? In short, how do you know whether it is a good specification? Requirements come in many different formats and levels of detail, and whenever they pass from one person to another there is a risk of misinterpretation. In these days of large distributed organisations, global projects and outsourcing, the risk of ambiguous or missing requirements is multiplied. A misinterpreted requirement can cost thousands and can mean the difference between project success and failure.

You need some way of determining whether the specification is complete and some way of spotting assumptions and ambiguities. In short you want to know whether the specification is good enough to successfully communicate the intended details to the intended audience. This paper talks about our approach to auditing requirements specifications and finding requirements defects before they cause damage.

Audits in other fields

Every year a professional firm of accountants audits our company’s financial records. The auditors make sure that we have been charging and paying the correct taxes, they check our invoices, payments, purchase orders, receipts and bank accounts. They make sure that we have been running our business according to the laws of the land. The auditors know what our records should contain and how much detail is necessary. They also know how to make sure that the details are consistent. Just like other companies, we keep our records organised according to long established accounting principles; this makes it easier for the auditors to follow what we have done. It also makes it easier for us to find the answers to questions. The point of having the audit is to provide an official checkpoint that we are operating correctly and that our records are consistent and understandable. There are parallels between a requirements audit and a financial records audit. The purpose of both audits is to avoid future problems by identifying problems early. However a key difference is that financial audits are obligatory and have a well-defined set of standards. So the auditors know where to start and they have an objective way of communicating deficiencies and raising questions. The records have a reasonably consistent form, the expected content is well defined, the level of detail is defined and the rules for consistent connections between financial records are not open to debate.

But in the case of requirements specifications we do not have a set of standards that define the obligatory contents. Instead, the form, content, detail and connectivity are all variable depending on the characteristics and choices of the particular project. However, by taking a closer look at these variables we have learnt how to do relevant and productive audits for all manner of requirements specifications.

Form, content, detail and connectivity

The form of a specification refers to characteristics like the way that it is organised, the way the contents are grouped, the use of text and diagrams, the indexing, the typefaces, the colours, the medium (paper or electronic) – everything that determines the document’s arrangement, format and appearance. Particular document design standards or the use of an automated tool quite often dictate the form of the specification.

The content is the classes of subject matter that are contained in the specification. For example a requirements specification might be expected to contain things like an unambiguous definition of the scope of the requirements, a definition of the stakeholders, a specification of the functional and non-functional requirements, a definition of the terms used in the requirements and other pieces of requirements-related information as agreed by the particular project team.

The detail relates to the granularity of each of the essential pieces of subject matter. For example should this document contain measurable atomic requirements or is it a preliminary document that just summarises the business use case scenarios?

The connectivity is the formal structure that connects the requirements knowledge and makes it traceable. If the specification contains atomic requirements then can each requirement be traced to the related parts of the business? Are all the terms in each requirement defined? Can each requirement be proved to be relevant to the scope of the project?

The challenge in doing a requirements audit is to be able to look past the form to discover whether the content, detail and connectivity are adequate for the intended purpose of the specification. In other words, if this specification is given to the people for whom it is intended, then what decisions will they be able to take and what assumptions will they need to make?

Doing an audit

We developed the Volere requirements specification template as a structure for guiding requirements discovery and organising requirements knowledge. Similar to financial audits, if you have used a formal structure for organising your requirements then it is easier to audit them. The auditor knows precisely what he is looking for, what level of detail should be provided and how all the elements should be connected to each other.

But we come across many specifications that have not been produced using a well-defined structure. So we do the requirements audit by using the requirements template as a checklist for driving the audit. Here’s how to do it:

You need:

• A copy of the Volere template
• A selection of coloured post-it notes (the coloured tabs are best) for marking pages
• Coloured pencils
• Index cards
• The specification that you wish to audit
• A spreadsheet program like Excel
• Some peace and quiet

Figure 1: Volere Template table of contents

Tagging the specification

Use the table of contents of the template (Figure 1) as the guide to running your audit. In essence you are matching the actual contents of the requirements specification that you are auditing with the generic content defined by the template. You are looking for subject matter that is missing, duplicated, ambiguous, superfluous or irrelevant. You can start with any one of the sections in the template and use it as the driver for finding matches in the specification for that type of information. I am going to tell you the order that we have learnt (by doing many specification audits) that normally proves to be most logical and productive. Start with section 1 the purpose of the project and remind yourself what you are looking for by reading the guidance in section 1 of the template. Choose one of your coloured post-it notes for tagging section 1 information—suppose you choose red. Now look through the specification you are auditing and put a red tag on all pages of the specification that refer to the purpose of the project. Incidentally I have suggested using coloured post-it notes because it is helps to distinguish between different types of information. If you haven’t got coloured notes then use standard yellow ones and write the relevant section number on each one. In addition to tagging the appropriate pages with a red tag it also helps to highlight the information with a red coloured pencil. This is because you will often have information about several sections mixed together on the same page and this approach will make it easier for you when you are analysing the results of your audit.

Now turn your attention to sections 2 and 3, choose another coloured tag, and review your specification from the point of view of how well it defines the stakeholders. I have audited specifications that make no mention of any type of stakeholder (or just mention the stakeholders by name of role without defining their input to requirements) and have learnt that these are indicators that there will be assumptions and ambiguities in the specification. Once again the template, along with the other references at the end of this paper, contains a great deal of guidance about the type of stakeholder information that should be in the specification.

The next section we review is section 7 the scope of the work, because this specifies the boundaries of the work that the project is investigating. The width of the boundary varies depending on the subject matter of the project. It might be the work done by part of a business, or it might be all the work that will be done by a new business, or it might be the boundaries of the part of the world that will be affected by a new consumer product. As explained in the template, the scope of the work should be precisely defined by adjacent systems and defined inputs and outputs. Also, the scope should be partitioned according to business events (or some other agreed and traceable partitioning theme) and each event should be traceable to one or more business use cases.

Next we review and tag the specification for: section 4 mandated constraints, section 6 relevant facts and assumptions, section 8 the scope of the product.

Throughout the audit you should be looking for terms that are vague, contradictory or ambiguous. This corresponds to section 5 naming conventions and definitions. If the specification contains a dictionary/lexicon/glossary or some section that defines the terminology, then you can use this as a starting point. In parallel with all of your auditing activities, whenever you come across a term then ask the question – is this term unambiguously defined and is it being consistently used? A convenient way to keep track of this is to write each term on an index card – or you can use a spreadsheet if you prefer, and note on the card whereabouts in the specification the term is defined. If you come across more than one definition of the same term then note the location on the card. To keep track, I use a specific coloured pencil – green – to mark in the specification the terms that I have noted on cards.

Now you have built some familiarity with the organisation and content of the specification and you are ready to look for and tag the atomic functional and non-functional requirements; sections 9-17 in the template cover these. Also, the requirements shell that you will find in the template’s introductory material identifies all the attributes of an atomic requirement.

Use the requirements shell to assess the completeness of each atomic requirement. If any of the attributes is missing, then make a note on the specification and tag the requirement with one of your coloured notes.

Looking for connections

Each atomic requirement should possess attributes that define its details. But a problem in many of the specifications that I audit is that there is no formal way of assessing how these requirements work together in groups, or how they affect each other.

To provide functional traceability each of the atomic requirements in the specification there should have a connection to one or more business use cases and, if the project has progressed to that stage, one or more product use cases. Depending on how the project has been partitioned, the requirements in your specification might be grouped using a different theme like features or components. The partitioning theme should support the way that you are working and should reflect the way that the requirements need to be grouped in order to communicate “chunks” of requirements to your relevant stakeholder groups. On many of the projects that I work on the requirements are tagged to use cases as a way of discovering and maintaining functional groups. Then they are also tagged to features, components or whatever other grouping the project needs to satisfy its communication needs.

Analysing the results

After comparing sections 1-17 of the template with the specification, you now you have a specification that is bristling with coloured tags and marked with coloured pencils. To prepare your audit report of the specification you need to identify:

• what is missing?
• what is redundant?
• what is inconsistent?

Missing requirements knowledge

If any of the sections (or subsections) of the template could not be matched to the some part or parts of the specification then you potentially have some missing requirements knowledge. I say potentially because you need to consider the intended detail (as discussed earlier) of the specification. If, for example, the intention is that the specification is a first cut summary of the requirements then maybe the exclusion of atomic requirements (or some of their attributes) is intentional.

If the specification is intended as input to design and implementation, then it should contain a detailed definition of the atomic functional, non-functional and constraint requirements. And each definition should include all the attributes of an atomic requirement. It should also be possible to trace each atomic requirement to the business use cases and product use cases to which it relates.

Spotting redundancies

In the first pass of your audit you used coloured tags and coloured pencils to identify which parts of the specification correspond to which parts of the requirements knowledge described in the template. If there are any parts of the specification that are not tagged, then those parts are potentially redundant. When you spot these untagged parts of the specification then you might discover that they are relevant but you missed them when you were doing your matching. However there might be some things in the specification that are not relevant to the requirements and are only going to confuse the people who read it.

Another reason for a section being untagged is that it might duplicate something else in the specification. If this is the case then mark it as a duplicate. Uncovering inconsistencies

Review the parts of the specification dealing with stakeholders (sections 2 &3) the goals (section 1) and the scope (sections 7 and 8) and look for inconsistencies:

• Does the work (investigation) scope concur with the stated goals?
• Are the stakeholders identified, not just by role but by name and reason for involvement?
• Are the goals measurable or open to interpretation?
• If the specification contains a product scope, then does that scope conform to the goals?
• Are all the interfaces on the context diagram defined in the definitions of terms? (Ref section 5).
• Are all the terms used in individual requirements defined in the specification? Earlier I suggested using either index cards or a spreadsheet as a way of keeping track of which terms occur where and whether or not they are defined.

Communicating the results

Once you have done your audit, then it is best to talk to the people who produced the specification and to ask questions about your findings. They are in a position to shed light on misunderstandings and to identify how they will address any problems that you have highlighted.

Then you can summarise the findings of your audit according to the sections on the requirements template.

Section Number:

Section Name:

• Missing requirements information
• Redundant requirements information
• Inconsistent requirements information
• Other comments and questions
• Business risk of non-repair

When to do an audit

The best time to do a requirements audit is whenever you have a handover point in your project. This might be because you are handing the specification over to another group in your organisation or to an external supplier. If you are giving the specification to people in another location, regardless of whether they are inside or outside your organisation, then it is vital to spend time doing the audit. You will be surprised at how many questions arise and how many assumptions you will uncover.

Suzanne Robertson is a principal and founder of The Atlantic Systems Guild and joint originator of the Volere requirements techniques. She specialises in helping organisations to set up relevant requirements processes, does requirements audits and teaches organisation how to write and audit their requirements. suzanne@systemsguild.com

Resources

Robertson, James, & Suzanne Robertson. Requirements-led Project Management: Discovering David’s Slingshot. Addison Wesley, 2005. Contains a requirements knowledge model that is helpful in doing requirements audits.

Robertson, James, & Suzanne Robertson. Volere Requirements Specification Template. Use this as a checklist for helping you to audit requirements specifications.

Robertson, Suzanne. Stakeholders, Goals, Scope: The Foundation for Requirements and Business Models. This article that will help you to audit the foundation of the requirements specification. http://www.volere.co.uk/articles.htm

Robertson, James, & Suzanne Robertson. Mastering the Requirements Process Addison Wesley, 1999. Contains details and examples of the subject matter covered in the requirements template.