PONC - Those who speak of progress measure it by “Quantity” and not by “Quality”

  • [アーティクル]

Test Manager A: Already 3 days of testing is over and you have found just 8 bugs. I am disappointed with the way …….

Tester B: but if you look at the bugs you will find…

Test Manager A: Don't make excuses…when I was a tester, I found "101" bugs in just 2 days. Take example of Tester B007, he has already found 22 bugs and…

Tester B: I understand you point and let me explain you that I have found 5 Severity one bugs and they are really critical.

Test Manager A: That's fine but I want you to file at least 10 more bugs before you leave today.

Tester B: Okay L

Rings any bells? Sounds familiar? You are very lucky if you are not tester and it didn't happen to you ever J

Okay folks, I am not telling you a story, this happened to one of my friend in a CMM Level 5 company.

Why Quantity kills Quality? L

There is an article on "Quantity, Quality, and the Function of Knowledge " by Hartley B. Alexander

https://links.jstor.org/sici?sici=0160-9335%2819050817%292%3A17%3C459%3AQQATFO%3E2.0.CO%3B2-Y&size=LARGE&origin=JSTOR-enlargePage

I am presenting an interesting excerpt from this wonderful piece of work on philosophy and human psychology to prove my point:

'The human mind knows only numbers adequately.', 'The mind only grasp fully and clearly which is reducible to the term of ratio', 'we reduce our impressions to mathematical formulas, and simply by means of this reduction of a confused state to a clear idea, we manage adequately to grasp the real'

What do we understand by Quantity?

Quantity is a kind of property which exists as magnitude or multitude. Quantity was first introduced as quantum, an entity having quantity. Being a fundamental term, quantity is used to refer to any type of quantitative properties or attributes of things. In our simple industry terms "Quality is conformance to requirements."

Where to Apply?

  1. Defects / Bugs: Finding Quality bugs / defects are more important than finding lots of not that severe bug.
  2. Test Cases: Writing few quality test cases is more effective than writing 10001 test cases? In the end a good test case is one which has a high probability of find defects.
  3. Automated Test Scripts: Automating few test cases which can give you high ROI are better to automate rather than trying to automate 100 % test cases and then ending up doing lot of rework.

Some on the common ways to measure / improve quality are:

  1. KAIZEN (Contribution Process Improvement)
  2. Quality Management System
  3. Total Quality Management
  4. Six Sigma
  5. Zero Defect
  6. SPC (Statistical Process Control)

Suggestions:

  1. Severity: When you prepare test plans, define "Severity" definition very clearly.

  2. Prioritization: Attack high risk areas first and something which is too critical for customer / end user.

  3. Unit Testing: Encourage your dev team to perform a good unit testing so that you can find quality bugs in Integration and System testing phase.

  4. Impact: For each bug logged state "Business Impact" of that bug if not been fixed.

  5. Worst Case Scenario: Always identify and mention the "WORST CASE SCENARIO" of the bug.

  6. Quality of the Build: If the bug is Sev 1 and that is critical enough and might be possible cause of many other bugs then immediately stop testing and reject the build. Business users don't have time to analyze your complex bug matrix.

  7. Reports with GRAPHS: Give daily reports / weekly report / build summary / test coverage reports with "Graphs" with bug classification as Sev 1, Sev 2 and Sev 3.

  8. Triage: Always include the stakeholders (recommended – Customer, Business User) in the BUG TRIAGE meetings and let them know that you have found something critical.

  9. Talking with Customer always helps: If you feel testing estimation are too aggressive then always let the customer know that if test team is given more time then you can discover many more quality bugs and that way over a period of time you will have "many quality bugs and that will increase the bugs count too J"

My viewpoint: "Quality is reconcilable with Quantity" J There is tradeoff that exists between quality and quantity.

Final Verdict: Left to readers J

Lesson Learnt: I should try to write "Quality" blogs over "Quantity" L

If you are among the "vital few" who are still thinking what is PONC ...then it's "PRICE OF NON CONFORMANCE" and that signifies the importance of quality over just quantity.

Disclaimer: There is absolutely nothing wrong with assessing progress by looking at the no. of bugs but I would rather give more importance to **severity** of those bugs.