MacKenzie, 2013, Chapter 4 - Scientific Foundations

MacKenzie, 2013, Chapter 4 - Scientific Foundations

1. 4.2 What is empirical reserach?

   • Definitions of empirical
     • Originated or based in experience or observation alone, often without due regard for system and theory
     • Capable of being verified or disproved by observation or experiment
   • Principle in science - not be blinded by preconceptions, researches should be guided by direct observations an experiences about phenomena, without prejudice to, or even consideration of, existing theories.

2. 4.1What is research?

   • Research must be published
     • Publication is the final step
   • Citations, references, impact
     • Support intellectual honesty
     • Many purposes and uses
     • Few citations or many citations from researches of a paper, describes the impact in the field
   • Research must be reproducible
     • If cannot be replicated is useless
     • Easy to read, easy to follow and importantly, ease to reproduce
   • Research versus engineering versus design
   • Definitions
     • An exercise as simple as careful or diligent search.
     • Collecting information about a particular subject.
     • Investigation or experimentation aimed at the discovery and interpretation of facts an revision of accepted theories or laws in ligth of new facts.

3. 4.3 Research methods

   • Experimental method (also called the scientific method)
     • Knowledge is acquired through controlled experiments in laboratory settings
       • Requires at least 2 variables: manipulable variable (also called an independent variable or factor) and a response variable (also called dependent varaible, must be observable, quantificable and measureable)
       • The relationship between the independent and dependent variable is equivalent to the cause and effect, which are manipulaions in the interface or interaction techniques to have caused differences and comparisson in the response variable.
     • Knowledge may imply gathering new knowledge or also studying existing knowledge for verifying, refuting, correcting, integrating or extending
     • Cause-and-effect conclusions are not possible in research using the observational method or the correlational method.
   • Correlational method
     • Looking for relationships between variables
     • Characterized by quantification since the magnitude of variables must be ascertained (e.g., age, income, and so on), for nominal-scale varaibles, categories are established.
     • Data may be collected through a variety of methods, susch as observation, interviews, on-line surveys, questionnaries, or measurement.
     • Provides a balance between relevance and prevision. Since the data were not collected in a controlled setting precision is sacrified.
     • Data collected bring balance to a connection with real life experiences. However the data obtained using correlational methods are circumstancial, not causal.
   • Observational method
     • In HCI (Human Computer Interaction) research
       • Techniques as interviews, field investigations, contextual inquiries, case studies, field studies, focus groups, think aloud protocols, storytelling, walkthroughs, cultural probes, and more are used
       • Is the why or how of the interaction, as opposed to the what , where, or when
       • Focus on human thought, feeling, attitude, emotion, passion, sensation, relection, expression, sentiment, opinion, mood, outlook, manner, style, approach, strategy, and so on
     • Approach tends to be qualitative rather than quantitative
     • Achieves relevance while sacrificing precision
     • Involve note-taking, photographs, videos, or audio recordings rather than measurement

4. 4.4 Observe and measure

   • Observation
     • Manual observation
       • Notebooks., log sheets, timing by hand in some activity, used for preliminary testing, sometimes called pilot testing.
     • Apparatus
       • Desktop computers, smartphones, others. Data is stored in a file for further analysis. Not always the best optionfor meassuring events.
   • Measurement scales
     • NOMINAL
       • Simple, involves arbitrarily assigning a code to an attribute or category.
       • Examples of nomical scale data: gender (male, female) and mobile phone usage (not using, using)
     • ORDINAL
       • Provide an order or ranking to an attribute.
       • Example of a questionnaire item soliciting an ordinal response.
     • INTERVAL
       • More sophisticated, interval data have equal distances between adjacent values.
       • Interval data are commonly used in HCI where a response on a linear scale is solicited.
     • RATIO
       • The most sophisticated data of the four scales of measurement.
       • Ratiod ata have an absolute zero and support a myriad of calculations to summarize, compare, and test the data. In HCI, the most common ratio-scale measurement is time (time to compelte a task)
       • Data can be added, substracted, multiplied, divided; means standar deviations, and varainces can be computed.

5. 4.5 Research questions

   • Initial thoughts
     • Is it viable?
     • Is it as good as or better than current practice?
     • What are its strengths and weaknesses?
     • Which of several alternatives is best?
     • What are the human performance limits and capabilities?
     • Does it work well for novices, for experts?
     • How much practive is required to become proficient?
   • Testable questions
     • Is the new technique any good?
     • Is the new technique better than QSK (Qwerty Soft Keyboard)?
     • Is the new technique faster than QSK after a bit of practice?
     • Is the measured entry speed (in words per minute) higher for the new technique than for a QSK after one hour of use?
     • Is the new technique faster than QSK?

6. 4.6 Internal validity and external validity

   • Graphical comparison of Internal Validity and External Validity
     • There's tension between these, immproving one comes at the expense of the other.
   • When a question is testable means that is possible to craft a methodology to answer it thtough observation and measurement.
   • When positioned as low breadth (that's bad) yet answerable with high accuracy (that's good)
   • Internal Validity: is the extent to which an effect observed is due the test conditions.
   • External Validity: is the extent to which expreimental results are generalizable to other people and other situations.

7. 4.7 Comparative evaluations

   • From a research perspective, more meaningful and insightful results are obtained if a comparative evaluation is performed.
   • Including a baseline condition serves as a check on the methodology andfacilitates the comparison of results between studies. e.g.,

8. 4.8 Relationships: circumstancial and causal

   • Cause and effect conclussion are not possible in certain types of controlled experiments. If the variable manipulated is a naturally ocurring attribute of participants, then the cause and effect conclusion are unreliable, beacuse is not possible to avoid confoundinf variables or because it's not possible to know wheter the experimental effect was due to the independent variable or to the confounding variable.
   • In HCI research usually the variable manipulated is often a norminal-scale, and the varaible measured is usually ratio-scale.
   • Finding a relationship does not mean a causal relationship exists, many relationships are circumstantial.
     • Classic example: smoking and cancer, there is a real relationship, cancer is more prevalent in the people who smoked, but not enough to affirm that skmow causes cancer.

9. 4.9 Research topics

   • Ideas
   • Finding a topic
   • Tips
     • #1: Think small!!
     • #2: Replicate!
     • #3: Know the literature
     • #4: Think inside the box!

Images and files in the map