EVALUATION

EVALUATION

1. Types

   • Formative
   • Summative

2. Heuristic

   • When to use?
     • Conducted at various stages of development cycle
   • What?
     • To identify general usability problems that operators can be expected to encounter when using a computer or a telematics application
   • Benefit
     • Quick and relatively cheap
     • General acceptance that the design feedback is valid and true
     • Can also be obtained early in the design process
   • Quality issues
     • Criteria for successful application of the method
       • Little experience
       • Good at both application area + human factor
       • Minimizing conflicts involving ‘the same’ and ‘different

3. Usability walkthrough

   • What?
     • To identify problems in paper-based screen design, draft training plans, draft documentation, etc
   • When to use?
     • early on in the design cycle, along with an assessment of user reactions
   • Benefit
     • Quick and little expenses for detailed user feedback
     • F/back is on the paper design (not on) before the development
     • Rapport between users and members of dev. team
   • Quality issues
     • Criteria for successful application of the method
     • Minimum procedure required
     • Common errors

4. CELLO

   • What?
     • largely derived from expert-based heuristic
   • When to use?
     • can be used throughout the lifecycle
   • Benefit
     • Easy to adopt
     • Cheaper than user-based testing and performance analysis
     • fast, relatively cheap, simple first step to usability testing
   • Quality issues
     • Criteria for successful application of the method
     • Minimum procedure required
     • Common errors

5. SUMI

   • What?
     • to analyze subjective opinions of users
   • When to use?
     • Can be used in conjunction with many other techniques
   • Benefit
     • Provides an objective way of assessing user satisfaction
     • SUMI’s authors have calibrated the results against large numbers of business systems
     • SUMI scores are based on a standardized questionnaire