SOFTWARE QUALITY - AN INTERNATIONAL CONCERN BY JEAN-NORMAND DROUIN, CQA BELL CANADA ISO 9000 Canadian businesses that want to survive in today's economy must be able to compete on the international market place. This in turn means that they must meet whatever international standards that apply to their business area. Over the last few years, the ISO 9000 series of standards has become such a standard. In fact, starting in 1993, any company wishing to enter the european market place must be certified to the appropriate ISO 9000 standard. The benefit to the customer is that is provides some reassurance that the organization has in place a recognized quality system. This certification does not guaranty that the product will be perfect all the time but it helps reduce the risk of having poor quality products. For the product manufacturer, ISO 9000 certification has become necessary to keep/expand its present business. The Problems of the Software Industry The software industry is unique in the fact that its product is intangible. The consumer cannot touch a computer program. At best, one can witness the actions that the program produces on a computer. Because of the size of today's computer programs, verifying their quality has become very difficult if not impossible. ISO 9001(Quality Systems Model for quality assurance in design, development, production, installation and servicing) is the standard that applies to the software industry. Because it has been designed for industries designing, developing, producing, etc. tangible products, it is quite hard to apply to an industry such as software which produces in tangible products. In principle, software quality is no different from car quality; one must ensure that the final product performs the way it was designed to. This is done by evaluating the final product, a method known as product evaluation. It is just as important to know that the product was put together call it manufactured, developed or engineered - in the best possible way. This type of evaluation is called process assessment. In certain market segments such as off-the-shelf software, product evaluation is possible. For instance, a word processor manufacturer can conduct a large-scale product trial to ensure that the final product performs as expected and is free of defects. Because of the sheer size of this market segment, the manufacturer can count on a large base of customers that are willing to participate in such a trial. Industry watchdogs also conduct and publish in the appropriate trade literature extensive comparative product studies that help consumers compare one product to the next. Finally, there is an ample choice of equivalent products on the market today. These three factors give the consumers the tools they need to select a product and feel confident that they are getting the best possi- ble quality for the money they are spending. The rest of the software industry does not always lend itself to product evaluation. One has only to think of the programs used to control nuclear power plants where the concept of trialing is ludicrous. Product evaluation of Telecom software presents a different set of problems because of the sheer size of these programs. A typical switching program is made of tens of millions of instructions that execute (and therefore must be tested) in real-time. For these products, process assessment becomes neces- sary to ensure that the quality was built-in. Software Development Process Evaluation In simple terms, the software development process is the set of methods and practices that an organization uses during the planning, design development, delivery, support, maintenance and support costs increase. For a development organization, one of the best ways to reduce these costs is to continuously improve its software development practices (build quality in). And the first step in this direction is to perform a software development process assessment. The concept of software development process assessment is just about 10 years old. Although many companies have started their own private efforts in this direction, it became widely known when the US department of defense (DoD), tired of getting soft- ware of unpredictable quality levels, mandated the Carnegie Melon University to develop such an evaluation methodology. The Soft- ware Engineering Institute (SEI) that was born from this created the Capability Maturity Model (CMM), a method and technique that scrutinizes and quantifies certain aspects of the software development process that were important to the DoD. Although incomplete in certain areas, the CMM and its associated rating scheme has provided many purchasing organizations in the US and some in Canada with a tool to help in the selection of a suitable development organization. But the real benefit of the CMM is that it provides development organization with a means to look for improvement opportunities in their development process. Many organizations that have used the CMM to improve themselves have found that continuous improvement is the best way to stay ahead of the competition. Are ISO 9000 or CMM Enough? ISO 9000 specifies how to develop and implement a Quality system in an organization. As we saw earlier, it is based towards manufacturing which makes it difficult to apply to the software industry. Furthermore, ISO 9000 does not address a number of key software engineering issues (e.g. process engineering, technical maturity, total Quality Management, etc.). The CMM on the other hand does provide a means to evaluate the software development process. And although it is well suited to organizations that produce software for the DoD, it is sometimes difficult to apply to other types of software. What is needed in addition is an international standard on software process assessment that would combine the best elements of the various process assessment methods that are presently available (CMM and others). The SPICE Standard Last June in London England, the ISO committee on Software Engineering (JTC1/SC7) held its annual plenary meeting. This committee has a number of key projects such as metrics, require- ments traceability, product quality, life-cycle processes etc. At that meeting, a large majority of the countries endorsed a proposal to create a new international standard for Software Process Assessment Called Software Process Improvement and Capability determination or SPICE, the project will develop a software process assessment standard that can be used for both capability determination (i.e. prior to contract signature to help in the selection of a suppli- er) and for process improvement as well. The standard will be based on the best methods currently available. It will also be designed so that it can be tailored to address the concerns of specific industry sectors. The project plan for the development of this new standard in- cludes an aggressive schedule calling for the drafting of seven standard documents by April 1994. Most importantly, SPICE in- cludes a trial of the standard that will be conducted in the 1994-1995 time frame. During this trial, companies will be asked to apply the standard to their own organization and provide feedback to the standard drafting team. Benefits of SPICE Many software supplier have to submit to a number of process assessment schemes as required by their customers. Having an international scheme for process assessment will help reduce this cost. And for the purchaser, dealing with a supplier that uses an international standard provides again reassurance that the supplier is seriously concerned with meeting and exceeding its customers quality expectations. Impact of SPICE On The Industry ISO 9001 is having an important impact on the software industry as many companies are finding out. The upcoming SPICE standard will have an even more significant impact. Companies that will participate in the SPICE trial or contribute to the drafting of the standard will acquire advanced knowledge on the standard and on how to use it. This will give them a significant advantage over their competitors and help them become competitive on the international marketplace. It is also important to recognize the software is now part of many products (power, transportation, medecine, telecommunication, etc.). SPICE will have a similar impact on these industries. In these times of economic troubles, it is important that the various levels of governments recognize the growing impor- tance of software on the Canadian economy and strongly support Canada's contribution and participation to the project SPICE. This kind of support will help increase the competitiveness of our industry on the international market place, adding billions of dollars and thousands of high-tech jobs to the Canadian economy, in the years to come. If you wish to know more about the Canadian SPICE project (de- scription, project plan, tasks, etc.), please contact: Jean-Normand Drouin, CQA Quality Assurance Bell Canada 2205 Rolland Therrien Longueuil, Quebec J4N lC5 Tel.: (514) 408-5531 Fax.: (514) 448-2090 [Originally printed in _Software_Quality_, Spring/Summer, 1993]