Software Test Metrics

Software Test Metrics is used in decision making. The test metrics is derived from raw test data.
Because what cannot be measured cannot be managed. Hence Test Metrics is used in test management. It helps in showcasing the progress of testing.

Benefits of Metrics

  • Provide status visibility.
  • Form a basis for decision making.
  • Set realistic expectations.
  • Focus people’s activities.
  • Supports process improvement.
  • Improve morale.

    Using Metrics

  • Display in form of Equations, tables, Pie charts, graphs.
  • Develop or refine development processes.
  • Use for Pareto analysis (A technique used for decision making) - Determine where most of the project’s time should be spent.
  • Discussing metrics with staff can be more valuable than the metrics themselves

    Risks of Metrics

  • Misuse for employee evaluations (your productivity was low - you’re fired!)
  • Misuse for employee incentives (I’ll pay $10 for each bug found)
  • Dependence on metrics outside of their scope
  • Use of unreliable data

    Limitations of Metrics

  • Over-reliance on statistics
  • Data inaccuracy

    Some of the Software Test Metrics are as below,

    What is Test Summary

    It is a document summarizing testing activities and results, and it contains an evaluation of the test items.

    Requirements Volatility

    Formula = {(No. of requirements added + No. of requirements deleted + No. of requirements modified) / No. of initial approved requirements} * 100
    Unit Of measure = Percentage

    Review Efficiency

    Components - No. of Critical, Major & Minor review defects
    - Effort spent on review in hours
    - Weightage Factors for defects:
    - Critical = 1; Major = 0.4; Minor = 0.1
    Formula = (No. of weighted review defects/ Effort spent on reviews)
    Unit Of measure = Defects per person hour

    Productivity in Test Execution

    Formula = (No. of test cases executed / Time spent in test execution)
    Unit Of measure = Test Cases per person per day

    Here the time is the cumilative time of all the resources. example - If there were 1000 Test cases executed in a cycle by 4 resources.
    If resource 1 executed 300 test cases in 2 days,
    resource 2 executed 400 test cases in 3 days
    resource 3 executed 75 test cases in 1 day
    resource 4 executed 225 test cases in 4 days.
    Then the cumulative time spent for executing 1000 test cases is 10 man days.
    Then the Productivity in Test Execution = 1000/10=100
    So the productivity in test execution is 100 test cases per person per day.

    Defect Rejection Ratio

    Formula = (No. of defects rejected / Total no. of defects raised) * 100
    Unit of Measure = Percentage

    Defect Fix Rejection Ratio

    Formula = (No. of defect fixes rejected / No. of defects fixed) * 100
    Unit of Measure = Percentage

    Delivered Defect Density

    Components - No. of Critical, Major & Minor review defects
    - Weightage Factors for defects:
    - Critical = 1; Major = 0.4; Minor = 0.1
    Formula = [(No of weighted defects found during Validation/customer review + acceptance testing)/ (Size of the work product)]
    Unit Of measure = Defects for the work product / Cycle.

    Outstanding defect ratio

    Formula = (Total number of open defects/Total number of defects found) * 100
    Unit Of measure = Percentage

  • style="display:block"