Transition

Transition

1. Testing and Optimisation

2. Testing

   • Goal of Testing
     • Assesses quality of software at the time
       • a process to provide stakeholders with information about the quality of the software with respect to the context in which it is intended to operate.
     • To show presence, not absence of bugs.
       • debugging then finds cause of bugs
   • Principles
     • Use both positive test (tests designed to pass to see if they don't) and negative tests (tests designed to fail to see if they don't)
     • Test the boundaries, the 0s, the full, the empty

3. Strategies

   • Unit Testing
     • Concentrate on modules most likely to contain errors
       • complex
       • novel
       • infamous
     • order of testing
       • start with smaller modules
         • add new ones gradually
       • get I/O fixed first
       • critical modules early
     • should be done during development
   • Manual techniques
     • systematic walkthrough
       • work in teams
         • designer, coder, tester and moderator
     • Software Inspections
       • read the source code in execution order, following the instructions as a computer must.
       • an expert tries to find common errors
         • - array bound errors - off-by-one errors - file I/O - default values - comparisons - reference vs copy
   • Black box testing
     • based on the specifications for what the unit should do
     • Analyse input ranges to derive test cases
       • boundary values
       • invalid inputs outside bounds
     • Equivalence types
       • Identify data values that are known to be equivalent
   • White box testing
     • Design test cases by looking at internal structure
       • Test each independent path at least once
       • Focus on infamous situations
     • Control structure tests
       • test all possible branches
       • data flow, confirm path through parameters
       • loops, especially changing how many times executed, 0, 1, many
       • ensure exceptions occur
   • Stress Testing
     • to discover at what point the system will fail
     • it has to actually break the system to be a valid test.
     • consider both volume and speed
     • not used for finding errors
   • Random Testing
     • Cannot possibly test exhaustively all combinations of data
     • Think about the system in a new way
     • Supply random data in the hope of finding a bug
   • Regression Testing
     • protects against reintroducing errors that were previously dealt with
     • Include all bugs reported
     • Every failure should produce a new test case
     • Test automization tools are the bomb
   • Usability
     • See how long it took someone to do something or how many errors they made
       • Compare averages over trials and over people
         • get normal distribution
     • Experimental (A/B) testing
       • we change something within the interface and see what effect it has on some measurable quantity
       • Null hypothesis states that there is no difference caused by the change
         • use t-test to try to disprove that, ie that the difference in data is not due to random variation
     • Think-aloud testing
       • evaluate user's mental model
     • Cognitive Walkthrough
       • like think-aloud, but no user involved
       • development team evaluates usability by playing part of user
   • Security
   • Efficiency
     • optimisation
   • performance