Software Testing Metrics and KPIs. In this article, I will explain you several software testing metrics and KPIs and why we need them and how should we use them. Bs Player For Windows 7 32 Bit here. This article based on my experiences and understanding. A software bug is an error, flaw, failure or fault in a computer program or system that causes it to produce an incorrect or unexpected result, or to behave in. What is Defect Priority Definition of priority Defect Priority signifies how important urgently it is to fix this defect. In other words Priority means how fast. Writing a good bug report is very necessary in order to get your bug resolved. Developer should be able to reproduce the bug easily using your bug reproduce steps. Also, I will use several quotes from various books and articles. They are listed at references part of this article. I want to start with metrics. Metrics can be very useful as well as very harmful to your development and testing lifecycle. It depends on how to interpret and use them. In any kind of organization, people managers, testers, developers, etc. Some of them use very dangerous metrics to assess team members performance, on the other hand, some of them use relevant, meaningful, and insightful metrics to improve their processes, efficiency, knowledge, productivity, communication, collaboration and also psychology. If you measure the correct metrics in a right way and transparently, they will guide you to understand the team progress towards certain goals and show the teams successes and deficiencies. In lean software development, we should focus on to use concise metrics that lead our continuous improvement. In reference 1, there is a quotation from Implementing Lean Software Development From Concept to Cash, by Mary and Tom Poppendieck2. They call this measurement cycle time. The focus is on the teams ability to repeatedly and reliably deliver new business value. Then, the team tries to continuously improve their process and reduce the cycle time. These kinds of metrics need whole team approach because all the team members efforts enhance these metrics results. Time based metrics are critical to enhancing our speed. We should ask ourselves some questions based on speed, latency, and velocity. In the agile world, one of the most used metrics is Team Velocity. It shows us, how many stories point SP that the team tackles during a single sprint and this is one of the fundamental key metrics of Scrum. It is calculated at the end of each sprint by summing up the completed User Story points. Lean development focuses to delight end users customers, which should be the goal for all the team. Thus, we need to measure also business metrics such as Return on Investment ROI and business value metrics. If the main business goal of the software is to reach and gain more customers, we need to deliver a software which serves this purpose. High Severity And Low Priority In Software Testing' title='High Severity And Low Priority In Software Testing' />In another reference, Sarialiolu states that without any metrics and measures, software testing becomes a meaningless and unreal activity. Imagine, in any project, that you are not aware of the total effort you have expended and the total number of defects you have found. Under these circumstances, how can you identify the weakest points and bottlenecks 2 This approach is more metric based and with this way you should talk with the numbers such as Total Test Effort and Test Efficiency with schedule and cost variancesTest Execution Metrics passedfailedin progressblocked etc. Test Effectiveness number of defects found in systemtotal number of defects found in all phasesTotal number of Defects and their Priorities, Severities, Root Causes dev, test, UAT, stage, liveDefect Turnaround Time Developers defect fixing timeDefect Rejection Ratio Ratio of rejectedinvalidfalseduplicate defectsDefect Reopen Ratio Ratio of successfully fixed defectsDefect Density per development day or line of codeDefect Detection Ratio per day or per testing effortTest Coverage, Requirement Coverage, and so onIn Mobile Testing Tips book 5, metrics importance stated as follows without the knowledge you would obtain through proper test metrics, you would not know how goodbad your testing was and which part of the development life cycle was underperforming. The whole team approach also critical on the metrics that you will measure and report. Some of the tips are listed below for the rest of them I suggest you read the book. Tell people why metrics are necessary. Explain each metric that your gather to all team members and stakeholders not only the test team. Make people believe in metrics. Try to be informative, gentle, and simple in your reports. Try to evaluatemonitormeasure processes and products rather than individuals. Try to report point in time and trend metrics. Try to add your comments and interpretation with your metrics. Try to be 1. 00 objective. Metrics should be 7. X2. 4 accessible. Also, metrics categorized in three sections in the book. These are test resources, test processes, and defects. Resources metrics are about time, budget, people, effort, efficiency, etc. Process metrics is about test case numbers, statuses, requirement coverages, etc. Defects metrics is about the number of defects, defect statuses, defect rejection ratio, defects reopen ratio, defects root causes, defects platforms, defects types, etc. At last, the book states that metrics make the test process transparent, visible, countable, and controllable and in a way allows you to repair your weak areas and manage your team more effectively. Some approaches such as Rapid Software Testing RST Bach Bolton states that you need to use discussion rather than KPIs and objective metrics. Bach emphasized that you need to gather relevant evidence through testing and other means. Then discuss that evidence. Also, Bach wrote that in Lessons Learned in Software Testing 4 book at Lesson 6. Never use the bug tracking system to monitor programmers performance. If you report a programmers huge number of defects, he gives his all to fix his bugs and tries to postpone all other his tasks. Also, the other crucial mistake is to attack and embarrass a developer for his bugs. This will cause very big problems on team collaboration and whole team approach. The other developer team members also respond this action very defensively and they will start to argue on each bug and they dont accept most of them. They generally tell, This works in my machine, Have you tried it after clear the browser cache with CTRLF5, This is a duplicate bug, It is not written in requirements, and so on. Also, the worst thing is they may start to attack you on your testing methods, approaches, strategies, and skills. This causes a terrific mess in the team, big problems on team members communications, and reduce team efficiency. And also lesson 6. Never use the bug tracking system to monitor testers performance. If you start to evaluate the testers with the number of bugs they found, they may start to behave not intended way. They are starting to focus on easy bug such as all kinds of cosmetic defects, they try to focus only bug counts rather than questioning the requirements and examine all kinds of edge cases. They may report the same bugs several times and this also irritates the developers and leads waste of time. Testers less likely to spend their time for coaching to other testers or self improvement activities etc. Also, it affects their psychology in a bad way. For example, in team A, if developers wrote unit tests which have 9. X may not find too many defects. On the other hand, in team B, if developers do not have the necessary things to do before the testing phase, this time tester Y may find too many defects. In these conditions, if you assess tester X and tester Y with the bug counts, this will be very unfair.