Lecture 10 - Testing and Verification

Lecture 10 - Testing and Verification

1. Overview

   • Testing a significant component of most life-cycle models
   • An integral component of development
     • Should occur during every phase of life-cycle
   • Divide into:
     • Execution based
     • Non-Execution Based
   • Independence of testing
     • Responsibility of different people
     • Originators have blind spots

2. Non-Execution Based Testing

   • Walkthrough
     • Group discussion of design documents
       • Concentrates on fault finding
       • Not about presenting a solution
     • Two Main Techniques
       • Authors of various design documents present/justify their proposals
         • Team provide feedback
   • Inspection
     • More detailed and structured than walkthrough
     • Five Main Steps
       • Overview
         • General statement by one individual
       • Preparation
         • Participants try to understand documentation
       • Inspection
         • Detailed presentation of each document
           • Ensure every brand covered
           • Faults identified
           • Report of faults produced by discussion moderator
       • Rework
         • Author responds to faults presented
           • Proposes solution to faults
       • Follow-Up
         • Ensure all faults addressed, no new ones introduced

3. Testing during development phases

   • Requirements Phase
     • Allow various users to experiment with prototypes
     • Ensure people consulted are representative of end users
   • Object-Orientated Analysis Phase
     • Walkthrough/Inspection of design documentation
       • CRC Cards, Initial Class Diagrams
   • Design Phase
     • Walkthrough/Inspection Again (Personnel restricted to design team)
       • Detailed Class Diagrams, Behavioural Diagrams
     • Show how new documentation follows from earlier stages

4. Criteria for Execution based Testing

   • Correctness
     • Satisfies its out requirements
   • Utility
     • Extent to which user's needs are met
   • Reliability
     • Frequency of product failure
   • Robustness
     • Error Handling, adaptability to differing operating conditions
   • Performance
     • Response Time, Memory Utilisation

5. Implementation

   • Class Testing
     • Informal (By Programmers)
     • Methodical (SQA Group)
       • Non-Execution Based
       • Execution Based
   • Black-Box (Test to specifications)
     • Choose test data based on specifications
   • White-Box/Glass-Box (Test to Code)
     • Test-Data chosen by examining code

6. Black-Box Testing

   • Not possible to test all input permutations
     • Two variables
       • One takes one of seven values
       • One takes one of nine values
       • Implies 56 permutations and is a very simple case
   • Art to choosing manageable sets of data
   • Equivalence testing and boundary value analysis
   • Equivalence Class
     • et of data where one element is effectively as good as another
   • Can apply similar technique to output
     • Devise test data to produce output in each equivalence class and around boundaries
   • Alternative: Functional Testing
     • Test functional of whole system
     • Each item of functionality for the system tested

7. White-Box Testing

   • To test every path through the code
     • Permutations very large
     • Not possible to test all paths
   • Structural Testing
     • Statement Coverage
       • Test Data formulated so that every statement is executed at least once
       • Weakness on slide
     • Branch Coverage
       • Devise test data such that every branch of very statement executed
         • More difficult but more errors found
     • Path Coverage
       • Tests all paths through code
         • Too Many
         • Use Linear Code Sequences
         • Identify points L from which control flow may jump
           • Control statements (if, do, switch etc), entry and exit points
         • Linear code sequence begins at an element of L and ends at an element L
         • Testing only these paths significantly reduces the total number
       • Alternative
         • All-Definition-Use-Paths
           • Identify paths between variable definition and its use
             • Test these paths
           • Very efficient technique, many faults found
           • Far fewer paths than maximum
         • Infeasible paths (Dead Code)
           • Some code cannot be reached -- indicates fault
         • Integration/Maintenance
           • Each class tested as completed
           • Test the remainder still works
           • Maintenance: Additional Problem: Personnel unfamiliar with product

8. Formal Verification

   • Problems may remain after all testing
   • Alternative
     • Formally prove program correct
   • Proof mathematical and time consuming