Please wait - loading…

Measuring the User Experience (Tullis and Albert:2013)

Description

Albert, W. and Tullis, T., 2013. Measuring the user experience: collecting, analyzing, and presenting usability metrics. Elsevier.
William Dunn
Mind Map by William Dunn, updated more than 1 year ago
William Dunn
Created by William Dunn over 3 years ago
46
0

Resource summary

Measuring the User Experience (Tullis and Albert:2013)
  1. 4. Performance metrics
    1. Task success
      1. Binary success
        1. Levels of success
          1. Issues in measuring success
          2. Time on task
            1. Importance
              1. Collecting and measuring
                1. Analyzing and presenting
                  1. Issues to consider
                  2. Errors
                    1. When to measure
                      1. What is
                        1. Collecting and measuring
                          1. Analyzing and presenting
                            1. Issues to consider
                            2. Efficiency
                              1. Collecting and measuring
                                1. Analyzing and presenting
                                  1. As a combination of task success and time
                                  2. Learnability
                                    1. Collecting and measuring
                                      1. Analyzing and presenting
                                        1. Issues to consider
                                      2. 5. Issue-based metrics
                                        1. What is an issue?
                                          1. Real vs. false
                                          2. How to identify
                                            1. In-person study
                                              1. Automated study
                                              2. Analyzing and reporting metrics for usability issues
                                                1. Freq of unique issues
                                                  1. Freq of issues per participant
                                                    1. Freq of participants
                                                      1. Issues by category
                                                        1. Issues by task
                                                        2. Consistency and bias
                                                          1. Number of participants
                                                            1. 83% of issues by 1st 5 people (31% avg prob of any one issue)
                                                              1. Other study only 35% of issues
                                                                1. Tullis: Catch big issues; Homogenous user group; Limited design
                                                                  1. More people when all issues must be caught; various user groups; Large design
                                                            2. Severity ratings
                                                              1. Based on UX
                                                                1. Based on combination of factors
                                                                  1. Using rating system
                                                                    1. Caveats
                                                                2. 6. Self-reported metrics
                                                                  1. Importance: User perception; Insight into feelings
                                                                    1. Rating scales
                                                                      1. Likert
                                                                        1. Semantic differential scales
                                                                          1. How and when:
                                                                            1. Analyzing
                                                                            2. Post-task rating
                                                                              1. Ease of use
                                                                                1. After-Scenario Questionnaire (ASQ)
                                                                                  1. Expectation measure
                                                                                  2. Post-session ratings
                                                                                    1. Aggregating individual task ratings
                                                                                      1. System Usability Scale
                                                                                        1. Computer System Usability Questionnaire
                                                                                          1. Questionnaire for UI satisfaction
                                                                                            1. Usefulness, Satisfaction, and Ease-of-Use Questionnaire (USE)
                                                                                              1. Product reaction cards
                                                                                              2. Other types
                                                                                                1. Specific attributes
                                                                                                  1. Specific elements
                                                                                                    1. Open-ended questions
                                                                                                      1. Awareness and comprehension
                                                                                                        1. Awareness and usefulness gaps
                                                                                                      2. 7. Behavioural and Physiological Metrics
                                                                                                        1. Unprompted verbal expressions
                                                                                                          1. Eye tracking
                                                                                                            1. Scan path
                                                                                                              1. Heat map
                                                                                                                1. Areas of Interest (AOIs)
                                                                                                                  1. Binning chart
                                                                                                                    1. Metrics
                                                                                                                      1. Dwell time
                                                                                                                        1. # of fixations
                                                                                                                          1. Fixation duration
                                                                                                                            1. Sequence
                                                                                                                              1. Time to First Fixation
                                                                                                                                1. Revisits
                                                                                                                                  1. Hit ratio
                                                                                                                      2. Analysis
                                                                                                                        1. Pupillary response
                                                                                                                        2. Emotion
                                                                                                                          1. Affectiva; Q-Sensor; Blue Bubble Lab; Emovision; Seren; Emotiv
                                                                                                                          2. Stress and physiological measures
                                                                                                                            1. Heart rate variance
                                                                                                                              1. Skin conductance
                                                                                                                              2. PressureMouse; Posture Analysis Seat
                                                                                                                              3. 8. Combined and comparative metrics
                                                                                                                                1. Single Usability Scores
                                                                                                                                  1. Based on target goals
                                                                                                                                    1. Based on Percentages
                                                                                                                                      1. Based on Z scores
                                                                                                                                        1. Using single usability metric
                                                                                                                                        2. Usability scorecards
                                                                                                                                          1. Comparisons against
                                                                                                                                            1. Goals
                                                                                                                                              1. Expert performance
                                                                                                                                            2. The basics
                                                                                                                                              1. 1. Introduction
                                                                                                                                                1. Definitions
                                                                                                                                                  1. UX: User involved, interact with interface, measurable and interesting
                                                                                                                                                  2. Value of UX metrics
                                                                                                                                                    1. Structure, extent, evaluation of changes, ROI, patterms
                                                                                                                                                    2. Ten myths

                                                                                                                                                      Annotations:

                                                                                                                                                      • 1. Take too much time 2. Too expensive 3. Not useful for small improvements 4. Don't help understand causes 5. Too noisy 6. Trust your gut 7. Don't apply to new products 8. No metrics exist for this 9. Management doesn't care 10. Can't collect data with small sample
                                                                                                                                                      1. 1. Take too much time
                                                                                                                                                        1. 2. Too expensive
                                                                                                                                                          1. 3. Not useful for small improvements
                                                                                                                                                            1. 4. Don't help understand causes
                                                                                                                                                              1. 5. Too noisy
                                                                                                                                                                1. 6. Trust your gut
                                                                                                                                                                  1. 7. Don't apply to new products
                                                                                                                                                                    1. 8. No metrics exist for this
                                                                                                                                                                      1. 9. Management doesn't care
                                                                                                                                                                        1. 10. Can't collect data with small sample
                                                                                                                                                    3. 2. Background / Statistics
                                                                                                                                                      1. Independent and dependent variables
                                                                                                                                                        1. Dependent: Things you measure (e.g. # of errors)
                                                                                                                                                          1. Independent: Things you control (e.g. Age, designs)
                                                                                                                                                          2. Types of data
                                                                                                                                                            1. Nominal
                                                                                                                                                              1. Ordinal
                                                                                                                                                                1. Interval
                                                                                                                                                                  1. Ratio
                                                                                                                                                                  2. Descriptive statistics
                                                                                                                                                                    1. Central tendency
                                                                                                                                                                      1. Variability
                                                                                                                                                                        1. Confidence intervals
                                                                                                                                                                        2. Comparing means
                                                                                                                                                                          1. Independent samples
                                                                                                                                                                            1. Paired samples
                                                                                                                                                                              1. >2 samples
                                                                                                                                                                              2. Relationships between variables
                                                                                                                                                                                1. Correlations
                                                                                                                                                                                2. Nonparametrics tests
                                                                                                                                                                                  1. Chi square (X2)
                                                                                                                                                                                  2. Graphical presentation
                                                                                                                                                                                    1. Column or bar chart
                                                                                                                                                                                      1. Line graphs
                                                                                                                                                                                        1. Scatterplots
                                                                                                                                                                                          1. Pie or donut chart
                                                                                                                                                                                            1. Stacked bar or column
                                                                                                                                                                                  3. 3. Planning
                                                                                                                                                                                    1. Study goals
                                                                                                                                                                                      1. Formative study
                                                                                                                                                                                        1. Summative study
                                                                                                                                                                                        2. User goals
                                                                                                                                                                                          1. Performance
                                                                                                                                                                                            1. Satisfaction
                                                                                                                                                                                            2. Choosing the right metrics
                                                                                                                                                                                              1. TEN TYPES OF USABILITY STUDIES
                                                                                                                                                                                              2. Evaluation methods
                                                                                                                                                                                                1. Traditional
                                                                                                                                                                                                  1. Online
                                                                                                                                                                                                    1. Online surveys
                                                                                                                                                                                                    2. Other study details
                                                                                                                                                                                                      1. Budget and timeline
                                                                                                                                                                                                        1. Partcipants
                                                                                                                                                                                                          1. Data collection
                                                                                                                                                                                                            1. Data cleanup
                                                                                                                                                                                                        Show full summary Hide full summary

                                                                                                                                                                                                        Similar

                                                                                                                                                                                                        User Prototyping UX/UI Design Quiz
                                                                                                                                                                                                        jglawson
                                                                                                                                                                                                        User experience: Jeff Johnson's principles
                                                                                                                                                                                                        Martha Mathews
                                                                                                                                                                                                        46. People Can't Actually Multitask
                                                                                                                                                                                                        Jesus Zepeda
                                                                                                                                                                                                        45. People Pay Attention Only to Salient Cues
                                                                                                                                                                                                        Jesus Zepeda
                                                                                                                                                                                                        62. People are motivated by autonomy
                                                                                                                                                                                                        Jesus Zepeda
                                                                                                                                                                                                        User Prototyping for UX/UI Design Flash Cards
                                                                                                                                                                                                        jglawson
                                                                                                                                                                                                        71. People Can Tell if a Smile is Fake or Real More Accurately with Video
                                                                                                                                                                                                        Machaela Buckley
                                                                                                                                                                                                        What is Responsive Design?
                                                                                                                                                                                                        Martha Mathews
                                                                                                                                                                                                        Polymer 2.0 - Custom Element - Register Element
                                                                                                                                                                                                        Ravi Upadhyay
                                                                                                                                                                                                        ...More on Salient Cues
                                                                                                                                                                                                        Jesus Zepeda
                                                                                                                                                                                                        User experience and responsive design
                                                                                                                                                                                                        Martha Mathews