A tester's perspective on quality vs value.

  • Artigo

Quality is a mysterious thing. It seems difficult to articulate definatively, but it is relatively appearant when it's lacking. The term quality has different meanings to different people, and even the dictionary defines the word broadly. But, 'quality' is integral to the discipline of software testing, so at some point we simply have to wrap our heads around it and understand how it relates to our profession. We can say that at one end of the specturm is Jerry Weinberg's relativist point of view that "Quality is value to some person." At the other end of the specturm is a more prescriptive perspective by Philip Crosby who stated "Quality is conformance to requirements."

Several years ago when I first became a test manager at Microsoft I read Weinberg's seminal works entitled Quality Software Management, and I still highly recommend leads and managers responsible for leading engineering teams and shipping software products read all 4 volumes. In Volume 2; First Order Measurment, Weinberg stated "Quality is value" and 'value is always perceived value'.  The quality as value proposition is important for managers to understand because it largely determines business success. Basically, if our customers realize a value then for them there is also some perception of intrinsic quality. In this context quality is largely regarded as a value proposition for the business goal of increased customer satisfaction which Weinberg refers to as first-order measurement.

But, the quality as value is a business proposition. Quality as conformance to requirements implies that there are essential or distinctive characteristics, properties, or attributes that must be evaluated and measured. In fact, even Jerry Weinberg clearly states "Whatever else it means, quality at least means meeting specifications for each job."  As software testers our most important objective is to provide accurate, qualitative information. So, in this context quality is composed of measurable goals that are indirectly related to customer value, and Weinberg refers to this as second-order measurement.

So, when some people talk about quality in terms of 'value' they are perhaps approaching the topic from a high level perspective which is mostly external customer focused. However, since I often interact with new individual contributor testers and try to teach them important concepts and skills of the profession I tend to have a more measurement-centric approach to defining quality. It is the second-order measures that testers are generally responsible for obtaining and relating back to management as quality metrics and/or risk assessment information. But, qualitative software measures are also important for managers to understand as well. If senior managers (the decision makers) don't understand second-order measures they may not use them effectively as part of their decision process, or they will likely ignore or do away with second-order measures when the project spins out of control, and the result will likely be what Weinberg refers to as "quality crisis."

Certainly, understanding the customer needs may be first-order measurement of quality. But, defining the indirect measures of quality is a much harder problem. Testers must interpret customer values into goals and goals into measurable tasks as second-order measurements that correlate to 'quality' assessment. Things that make this an especially challenging task include collecting the wrong measures, abuse or ignore the metrics, and we must constantly make adjustments based on the information and on changing customer values. Also, the measures for one project are often different than the metrics needed on another project. But, one thing is for certain; without the ability to measure the "essential or distinctive characteristics, properties, or attributes" that are critical for the success of our project then we really have no way of knowing whether or not we achieve/exceed our goals other than feel-good or best guess.

"Thinking about measurability from the beginning is an essential part of creating a well-formed effort." - Jerry Weinberg.

Quality is the totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs. Not to be mistaken for "degree of excellence" or "fitness for use" which meet only part of the definition.
[ISO8402].

Comments