How to evaluate empirically

How to evaluate empirically

1. How to evaluate analytically

   • Heuristic Evaluation
   • Walkthrough
     • Think as the user would
   • Claims analysis
   • cognitive modeling
     • user control and freedom
   • Test
   • prototype
   • personalize
     • motivations
     • information processing style (top down or bottom up)
     • computer self efficacy
     • risk taking in software use
     • strategy for learning
   • Consistency and standards
   • Error prevention
   • Recognition vs recall
   • Flexibility and user efficiency
   • Help users recover and diagnose errors

2. Critique

3. How to "observe" whether it will work

4. Evaluation

   • Control
   • Variation

5. High fidelity functional prototype

6. A/B Tests

7. Ecological validity

   • Technology Probes
   • Experience Sampling

8. How to be Critical

   • Engage deeply
     • help someone understand what you were trying to do and why
   • Critiques
     • divergent and convergent
       • generate ideas and point in new directions
     • say something positive if you want to soften the blow of something negative
   • get clarification
   • challenge the designers assumptions
     • consider alternative perspectives
   • consider the implications and consequences of such a design
     • to truly consider all, you need expertise in the domain of design