Lecture 5b: Evaluation of UI

Lecture 5b: Evaluation of UI

1. evaluating UI

   • integral to the design process; info abt user / potential user's experience when interact with prototype; focus on both system usability & UX when interact with it
   • evaluation
     • find out user appeal the design / use the system; collect info abt user, survey aft use the system, UX
   • why evaluate?
     • cater to all type of user; usable, pleasing & engaging experience
   • what to evaluate?
     • ranges from low-tech prototypes to complete system
   • where to evaluate?
     • natural setting; remote studies of online behavior [social networking] (cheaper & quicker); living laboratories
   • controllled setting involving users
     • usability testing & experiments
   • natural settings involving users
     • online communities & products tht r used in public places [dh control to control user activity]
   • any settings not involving users
     • inspections, heuristics [apply knowledge of typical users guided by rules of thumb], walkthroughs [involve stepping through a scenario / answer ques for a detailed prototype], models & analytics

2. heuristic evaluation

   • experts, guided by usability principles; evaluate whether UI elements conform to tried & tested principles
   • 3 stages for doing heuristic evaluation
     • 1) briefing session to tell expert what to do
     • 2) evaluation period [seperate work > 1st pass to get a feel for the product > 2nd pass focus on specific features]
     • 3) devriefing session in which expoerts work tgt to prioritize prob & suggest solutions
   • pros
     • no special facilities needed; no participants required; no user testing; quick
   • cons
     • time-consuming; prob identify diff with user; prob by expert maybe wrong; tends to uncover many low severity prob; severe prob can be missed

3. expert review

   • not involve users; check cinsistency of all interface; identify prob of the interface
   • guidelines review; consistency inspection; cognitive walkthrough; metaphors of human thinking [focus on how user think when interacting with interface]; formal usability inspection [discuss the merits & weakness of the interface]
   • 3 - 5 expert reviewers can be highly productive

4. cognitive walkthrough

   • focus on ease of learning; conduct by expert; walkthrough the design prototype with scenario

5. usability testing [controlled setting involving users]

   • ollecct data using combination method; goals to determine usable of interface; recording performance of user; user are observe & timed; investigate how user perform the task; questionnaire aft testing; identify & explain errors; comparison of prototypes; measure performance based on task; test abt particular variable; user think aluod whe perform task; reocrd video for ltr review & identify other prob
   • paper mockups & prototyping (conduct usability testing); discount usability testing (quick, formative-evaluation, summative-evaluation); competitive usability testing (compare diff version of interface, compare with competitors products); universal usability testing (target for universal design, diff types of users use the design); filed test & portable labs; remote usability testing (synchronous & asynchronous)
   • 5 - 12 participants in diff profiles

6. type of data

   • time to complete a task; number & type of error per task; number of error per unit of time; number of navigation to online help; number of user making the particular error; number of user completing task successfully

7. typical usability test

   • 1) scheduling the test [usability team work with other product teams to plan a test; determine profile of test participant; contact, interview, schedule test sessions]
   • 2) pretest preparation [prepare agreement for user (paperwork); equipment setup; explain to participants what need to take note]
   • 3) during the test [ask user to think aloud; get comment from participants; observe all activity perform by participants]
   • 4) aft test [facilitator write report; follow up the result with developers]
   • short comings of usability testing [limited coverage of interface features; hard to predict success in long-term usage; lab environment diff with real work environment]

8. acceptance test

   • for large implementation projects, set goals for hardware & software performance; system must be reworked until success demonstrate to meet the criteria
   • time to learn specific functions; speed of task performance; rate of errors by users; human retention of commands over time; subjective user satisfaction

9. evaluation during active use

   • keep on collect data while user use the system; interviews & focus grp discussion; continuous user-performance data logging; online / telephone consultant / email / online suggestion boxes