Flashcards
SCC 202 - Lecture 7A - Measuring User Experience
Cards in this deck (33)
- 1What are the two types of data user studies can provide?Qualitative
Quantitative - 2Describe formative evaluation? What do you use the outcome for?Formative: before a design is finalized.
Goal: make improvements prior to release.
Identify and diagnose problems
Use outcome to make recommendations
Iterative, periodical. - 3Describe Summative evaluation? What do you use the outcome to do and give examples?Summative: evaluate the final product Goal: check how well objectives have been met
Evaluate against a set of criteria
Use outcome to inform follow-up actions n e.g., plan future extensions; marketing; seek investment - 4List two variable metrics?Observable, and
Quantifiable - 5List some aspects of user interaction we can measure using the two variables metrics?Effectiveness: being able to complete a task
Efficiency: the amount of effort required to complete a task
Satisfaction: the degree to which the user was happy with his or her experience while performing the task - 6Define some things metrics can help you do?A decision making tool
Help reveal patterns - 7What two aspects of user experience does user goal boil down to?Performance – covers what users actually do in the interaction
- Especially critical when they don’t have a choice in using the system
Satisfaction – covers what they say/think/feel about it
- Especially important when they have a choice (other products) - 8List 5 typical user studies?Study completion of a transactions
Comparing products
Study frequent use of the same product
Study navigation / information architecture
Increasing awareness - 9Name the four aspects preparing a user study?Sampling - Selection of participants Measurement- Choice of metrics and method of data collection
Design- Design of the setup, tasks and procedure
Analysis- How the data is analysed and presented - 10What was J.Nielsen view on the number of participants you need?“Five is Enough”, J. Nielsen n Arguing 80% of issues found with first 5 participants
- 11When sampling, the more participants yoy have, .......The more, the higher the confidence that the data is representative
- 12When might small numbers(5-10) be sufficient?- It’s ok to miss some problems
- There is only one distinct user group, expected to perform/think similarly
- Increase numbers if you have different groups (e.g. Novice v expert) - 13Difference between variables and attributes?- Variables can take on different values
- An attribute is a specific value on a variable - 14What are independent variables?The things you manipulate or control for in usability study
e.g., alternative UI designs you want to test, or products you compare - 15What are dependent variables?- The things you measure in the study
- e.g. time on task, number of errors, ease of use, ranking of designs - 16Name the four different types we can collect variables?Nominal
Ordinal
Interval
Ratio - 17Define Nominal ScaleSet of named attributes
Not ordered: all you can say is that they are different
Analysis limited to counts and frequencies - 18Examples of Nominal scale?- Different user groups / categories (e.g., novice/expert)
- Users who clicked link A instead of link B - 19What is the task success rate formula?Number of correctly completed tasks / Total number of attempts
- 20Define Ordinal scale?- Ordered groups or categories
- Attributes are ordered
- But distance between attributes is not meaningful – relative ranking only
- Analysis limited to counts/frequencies (can’t average) - 21Ordinal scale examples?Discrete rating scales, e.g. “excellent, good, fair, poor”
Rank order of preference for designs - 22Where are ordinal scale commonly used?Common type of data collected in surveys
- 23Define Interval scales?- Ordered set where distance between attributes is meaningful
- E.g., days of week; years
- But there is no absolute zero, and there are no “multiples”
- Analysis: can produce averages - 24Interval data examples?- Continuous rating scales
- SUS: system usability score (based on 10 statements that users agree/disagree on 5-point-scale) - 25Define ratio scale?Ordered series
Includes a zero with inherent meaning Can compare in terms of ratios, multiples, percentages - 26Ratio scale examples?Time
Number of errors - 27List the types of user metrics?Performance
Issues
User reported
Behavioural - 28Define performance metrics and give examples?Measure how the user performs with the interface
Task success: ability to complete a given set of tasks
Time on task: how long it takes
Errors: mistakes/slips made during a task - 29Define issue based metrics and give examples?Usability issues are based on behaviour in using an interface
Behaviours that prevent task completion
Number and frequency of unique issues Frequency per participant – how many issues encountered per user
Frequency of participants – how common is this issue with user - 30Define user reported metrics and give examples?Rating scales - Likert scales: each item is a statement; users rate their level of agreement
Post-task ratings - ease of use
Post session ratings - System usability scale)
Accessing specific attributes - visual appeal, perceived efficiency. - 31Define behavioural metrics and give examples?Unprompted verbal expressions - Coding to quantify
Eye tracking - Analysing visual behaviour n Measuring emotion- Electrodermal activity (EDA)
Measuring stress- Heart rate variance - 32Break down the study design process?What, When, Where, Who (participants), Why (for what purpose), hoW (what technology)
What will users actually interact with in the study?
Tasks: what will users be asked to do with the interface?
Procedure: how will the study unfold, step by step
Control: how will other influences on the results be minimized - 33—
Information
- Description
- Flashcards on SCC 202 - Lecture 7A - Measuring User Experience, created by Ayo Olutobi on 31/01/2017.
- Created by
- Ayo Olutobi
- Views
- 4
- Rating