Mind map

Testing and modeling users

Ara Estrada Spanish

Map content

A central aspect of interaction design is user testing. User testing involves measuring the performance of typical users doing typical tasks in controlled laboratory-like con- ditions.
- Its goal is to obtain objective performance data to show how usable a system or product is in terms of usability goals, such as ease of use or learnability.
User Testing
- User testing is an applied form of experimentation used by developers to test whether the product they develop is usable by the intended user population to achieve their tasks
  - Variables
    - The aim of an experiment is to answer a question or hypothesis to discover new knowledge. The way that scientists do this is by investigating the relationship between variables. Changing one of them and observing what happens to the other.
  - Because user testing has features in common with scientific experiments, it is sometimes confused with experiments done for research purposes. Both measure performance.
    - Research requires that the experimental procedure be rigorous and carefully documented so that it can be replicated by other researchers.
    - Typically 5-12 users are involved in user testing, but often there are fewer and compromises are made to work within budget and schedule constraints. "Quick and dirty7'
MEDLINE EXAMPLE
- The goal of the study was to identify usability problems in the revised interface. More specifically,the evaluators wanted to know if the revised way of categorizing information, suggested by the expert evaluators, worked.
  - MEDLINEplus was tested with nine participants selected from primary health care practices in the Washington, DC metropolitan area.
    - The topics chosen for the tasks were identified from questions most fre- quently asked by website users:
      - The procedure involved five scripts that were prepared in advance and were used for each participant to ensure that all participants were given the same information
        Before starting the main tasks the participants were invited to explore the web- site for up to 10 minutes and to think aloud as they moved through the site.
        Criteria for successfully completing each task were developed in advance. For ex- ample, participants had to find and access between 3-9 web page URLs. Each user's search moves were then recorded for each task.
        Analysis of the data focused on such things as: website organization such as arrangement of topics,
        The main finding was that reaching external sites was often difficult. Furthermore, analysis of the search moves revealed that several participants experienced diffi- culty finding the health topics pages devoted to different types of cancer.
Doing user testing
- Controlling the test conditions is central, so careful planning is necessary. This involves ensuring that the conditions are the same for each participant, that what is being measured is in- dicative of what is being tested
  - Working through the DECIDE framework will help you identify the nec- essary steps for a successful study
- Determine the goals and Explore the questions
  - User testing is most suitable for testing prototypes and working systems. Although the goal of a test can be broad, such as determining how usable a product is
  - Choose the paradigm and techniques
    - User testing falls in the usability testing paradigm and sometimes the term "user testing" is used synonymously with usaplity testing.
    - ldentify the practical issues: Design iypical tasks
      - Deciding on which tasks to test users' performance is critical. Typically, a number of "completion" tasks are set, such as finding a website, writing a document
        The type of test prepared will depend on the type of prototype available for testing as well as study goals and questions.
        Generally, each task lasts between 5 and 20 minutes and is designed to probe a problem. Tasks are often straightforward and require the user to find this or do that
      - Identify practical issues: Select typical users
        Knowing users' characteristics will help to identify typical users for the user testing. But what is a typical user? Some products are targeted at specific types of users, for example, seniors, children, novices, or experienced people
        Identify~practicai lssues:Prepare the testing conditions
        User testing requires the testing environment to be controlled to prevent unwanted influences and noise that will distort the results.
        The space may be arranged to superficially mimic features of the real world. For example, if the product is an office product or for use in a hotel reception area,
        The observation room is usually separated from the main laboratory by a one- way mirror
        The reception area also has bathroom facilities so that testers do not have to go into the outside world during a session
        jdentify practical issues: plan how to run the tests
        A schedule and scripts for running the tests, such as those used in MEDLINEplus, should be prepared beforehand. The equipment should be set up and a pilot test
        Deal with ethical issues
        you need to prepare and plan to administer an in- formed consent form. If the study is situated in a usability laboratory, it is also nec- essary to point out the presence of one-way mirrors,
        Evaluate, analyze, and present the data
        Typically performance measures (time to complete specified actions, number of er- rors, etc.) are recorded from video and interaction logs. Since most user tests involve a small number of participants, only simple descriptive statistics can be used to pre- sent findings: maximum, minimum, average
        Preece, Jennifer. (2002). Interaction Design. United States of America: John Wiley & Sons Inc.
        ARANZA ESTRADA A01176250
        Experiment
        The development schedule was flexible, and the evaluators knew that quantitative results would be well received by their clients, so they adopted a more experimental approach than usual.
        The aim of an experimentis to answer a question or to test a hypothesis that pre- dicts a relationship between two or more events known as variables.
        Variables and conditions
        In order to test a hypothesis, the experimenter has to set up the experimental condi- tions and find ways to control other variables that could influence the test result.
        Experiments are carried out in user testing usually to compare two or more condi- tions to see if users perform better in one condition than in the other.
        In matched-participants design, participants are matched in pairs based on certain user characteristics such as expertise and gender.
        Data collection and analysis
        Data should be collected that measures user performance on the tasks set. These usually include response times, number of errors, and times to complete a task.
        Fitts' Law
        predicts the time it takes to reach a target using a pointing device.
        One of its main benefits is that it can help designers decide where to locate buttons, what size they should be and how close together they should be on a screen display.
        In a nutshell the bigger the target the easier and quicker it is to reach it.
        Allocation of participants to conditions
        there are not enough partici- pants and at other times it is preferable to have all participants take part in all condi- tions.
        In same-participant design, all participants perform in all conditions so only halfthe number of participants is needed
- Practical Issues
  - Where will the experiment be conducted, how will the equipment be setup, how will participants be introduced to the experiment
  - Predictive models
    - provide various measures of user performance without actually testing users.
      - This is especially useful in situations where it is difficult to do any user testing.
    - The GOMS model
      - Goals: refer to a particular state the user wants to achieve (e.g., find a website on interaction design).
        Operators: refer to the cognitive processes and physical actions that need to be performed in order to attain those goals (e.g., decide on which search en- gine to use, think up and then enter keywords in search engine). The differ- ence between a goal and an operator is that a goal is obtained and an operator is executed.
        Methods are learned procedures for accomplishing the goals. They consist of the exact sequence of steps required (e.g., drag mouse over entry field, type in keywords, press the "go" button).
        Selection: rules are used to determine which method to select when there is more than one available for a given stage of a task.
        The Keystroke level model
        differs from the GOMS model in that it provides actual numerical predictions of user performance. Tasks can be compared in terms of the time it takes to perform them when using different strategies.
        The predicted time it takes to execute a given task is then calculated by describing the sequence of actions involved and then summing together the approximate times
        Benefits and limitations of GOMS
        allows comparative analyses to be performed for different interfaces or computer systems relatively easily.