CHAP 3: EVALUATION TECHNIQUES

CHAP 3: EVALUATION TECHNIQUES

1. Evaluation

   • Test usability system and functionally of system
   • Occurs in laboratory, field and/or in collaboration with users
   • Evaluates both design and implementation
   • Should be considered at all stages in the design life cycle

2. Goals of evaluation

   • Assess extent of system functionally
   • Assess effect of interface on user
   • Identify specific problem

3. Evaluating design

   • Cognitive walkthrough
   • Review based evaluation
   • Heuristics evaluation
     • Nielson's 10 heuristics: visible -> Match -> Control -> Consistent -> Prevention -> Recognition -> Flexible -> Aesthetic -> Help -> Documentation
   • Model based evaluation
     • Design rationale, Dialog model, GOMS model, Keystore- level model.

4. Evaluating through user participation

   • Laboratory studies
   • Field studies

5. Experimental factors

   • Statistical measures
   • Experimental design
     • Within groups
     • Between groups
   • Hypothesis
   • Variable
     • Independent variable
     • Dependent variable
   • Participant

6. Analysis of data

   • Types of test- parametic, non-parametic, contingency table.

7. Analysis of data

   • Analysis- type of test
     • Parametric
     • Non-Parametric
     • Contingency table

8. Evaluation techniques

   • Query techniques
     • Questionnaires
       • Multi-choise
       • Ranked
       • open-ended
       • General
       • Scalar
     • Interviews
   • Observational techniques
     • Think aloud & cooperative evaluation
     • Protocol analysis
     • post-talk walkthrough
     • Automatic protocol analysis tools

9. Factors distinguishing evaluation techniques

   • The level of interference implied
   • The immediacy of the response
   • The information provided
   • The type of measures provided
   • The level of subjectivity or objectivity of the techniques
   • The style of evaluation
   • The stage in the cycle at which the evaluation is carried out
   • The resource required