Introducing Evaluation

Introducing Evaluation

1. Evaluation is integral to the design process

   • Evaluators collect information about users’ or potential users’ experiences when interacting with a prototype, an app, a computer system, etc
     • They do this in order to improve its design.
       • Evaluation enables to check that the design is appropriate and acceptable for the wider user population.

2. Evaluation focus

   • Usability of the system
   • Users’ experience when interacting with it

3. Methods

   • Which to use depends on the goals of the evaluation
     • Controlled settings involving users
       • Users’ activities are controlled in order to test hypotheses and measure or observe certain behaviors.
         • The main methods are usability testing and experiments.
     • Natural settings involving users
       • There is little or no control of users’ activities in order to determine how the product would be used in the real world.
         • The main method used is field studies.
     • Any settings not involving users
       • Consultants and researchers critique, predict, and model aspects of the interface in order to identify the most obvious usability problems.
         • The range of methods includes inspections, heuristics, walkthroughs, models, and analytics.

4. How You Interpret Data

   • Reliability
     • How well it produces the same results on separate occasions under the same circumstances
   • Validity
     • Is concerned with whether the evaluation method measures what it is intended to measure
   • Ecological Validity
   • Biases
     • Occurs when the results are distorted
   • Scope
     • How much its findings can be generalized