Lecture 6: Reasoning about probabilities

Nature of probability

• Bayesians view on probability: Subjective degree of confidence. One can express confidence that a single event will occur one can express the probability of a single event. Can express confidence that a single event will occur
• Frequentist: Probability is always defined over a reference class. It cannot have a relative frequency or probability. Can only express probabilities over a series of events.
• The reason why we can't reason about probability is because we can't make rational decisions.
• Bayes theorem is normative theory of probability by now it should be clear that:
• Man is apparently not a conservative bayesian: Kahneman & Tversky
• Our minds are not built to work by rules of probability: Gould 1992

Do frequentist representations elicit B.R.

• B.R. ( Bayesian Reasoning)
• Heuristic and biases literature people do not appear to reason about probabilities normatively.
• Evolutionary psychologists we can according to Bayes Theorem if information is presented in a format that we have evolved to process.
• Both sets of theories make claim that people aren't rational
• Not because we can't rationalize decisions, but because we weren't evolved to do so
• No evolutionary reason why Bayesian reasoning should not have developed.
• If prediction is based on long-line of memory is it more reliable
• Further Reading.
• In summary, the difference is that in the Bayesian view, a probability is assigned to a hypothesis. In the frequentist view, a hypothesis is tested without being assigned a probability. (Gershoff)

Advantages of a frequentist mind

• Basically, using a Frequentist method means making predictions on underlying truths of the experiment using only data from the current experiment. (Birkett)
• Number of events indicates the reliability of the decision is retained in frequency format but not in a probability format
• Permits easy updating as new information is collected
• Reference classes can be constructed post-hoc as the reference class changes.

Base Rate Neglect: Tversky & Khaneman

• T&K 1982
• Blue/Green Cab example. 85% are green 15% blue.
• Witness identified cab as blue. Correct 80% incorrect 20%
• People focus on 80% rather than the 15%
• Chance that witness incorrectly identifies green taxi is higher than correctly identifying blue taxi.
• T&K attribute this to representativeness heuristics: Tendency to see someone or something belonging to a certain group or category by evaluating how similar this object is to another particular object in the same group. (Sandersson)
• Base rate neglect is the tendency for people to mistakenly judge the likelihood of a situation by not taking into account all relevant data.
• Base rate: base rate is the a priori chance or prior odds that a member of a specified population will have a certain characteristic, assuming that we know nothing else about this person other than that he or she is a member of the population we are examining (Kamphuis & Finn, 2002).
• Explanation: People integrate two items of
• information only if both seem to them equally relevant. Otherwise,
• high relevance information renders low relevance information irrelevant. (FR: Bar-Hillel, 1979)

Base rate neglect: Cascells et al. (1978)

• If a test to detect a disease whose prevalence is 1/1000 has a
• false-positive rate of 5%, what is the chance that a person found
• to have a positive result actually has the disease, assuming you
• know nothing about the person’s symptoms or signs?
• Most frequent answer: 95% --> Reasoning: if false-positive rate is 5%, then test will be correct
• 95% of the time.
• Correct answer: about 2% --> Reasoning: assume you test 1000 people; only about one person
• actually has the disease, but the test will be positive in another 50
• or so cases (5%). Hence the chance that a person with a positive
• result has the disease is about 1/50 = 2%
• About 12% gave correct answer
• Suggest some sort of concern about the way general defects in human reasoning may affect decision making. (Boterill & Carruthers)

Base rate neglect: Counter-argument

• Cosmides & Tooby ( 1996) presented problem in both probability and frequency formats.
• Found that people can't reason because format isn't it whole numbers
• Instead of producing probabilities it was found that when asked out of how many people who test postitive for the disease will actually have the disease
• Found that the majority of participants answered correctly.
• Interpreted their data in terms of modules. The modules cannot reason about probabilities because it is domain specific. Can only accept the input in a particular frequency format.
• HOWEVER!! Both Griffin and Beuhler (1999) and Evans (2000) actually failed to replicated this effect.
• Further Reading
• Substantial body of evidence to show that human participants are prone
• to a range of errors and biases when given statistical word problems in the psychological laboratory (Evans et al. 2000)
• One argument that has been advocated by the optimistic or pro-ration-
• ality camp is that people are much more competent in reasoning about frequencies
• than one case probabilities (Evans et al. 2000)
• Since most research on quantitative word problems has employed probabilistic formats, it is argued, a quite misleading picture of people's actual competence
• in statistical reasoning and risky decision making has emerged in this literature. (Evans et al. 2000)
• Cosmides and Tooby (1996) have linked the claimed superiority of
• reasoning about frequencies to their research programme on evolutionary psychology.
• Problem with these experiments is that the frequency versions used do not
• simply present frequency information. Also a cue a simple mental model of set
• inclusion by focusing every statistic on a group of 1000 people.

Monty Hall Revisisted

• Aaron & Spivey ( 1989) and Krauss & Wang (2003) compared behavior on monty-hall problem using frequency and probability formats. Found that frequency formats elicited a higher switch rate than probability formats.

Preference Reversals

• Key phenomenon that violates rational choice theory
• Lichtenstein & Slovic 1971: Participants are presented with parts of monetary gambles or "bets"
• Contain possibility of winning or losing certain amounts
• One bet in each pair is relatively safe and has a high chance of winning small amount
• The other bet is more risky and has a small possibility of winning a large amount
• Participants prefer to play P-bet over $-bet.
• Monetary value of bet reflects utility. $-bet has higher utility.
• Inconsistent behavior because Ss have rated $ has higher utility.
• Further Reading
• Tunney et al. 2006 examined if reversals are diminished when presented as frequencies:
• When the options were presented in the standard probability format 59% of participants’ preferences were reversed. However, reversals were significantly decreased in the frequency format group, with only 40% of participants’ responses being inconsistent.
• Preference reversals were not eliminated when the gambles were presented as frequencies they were significantly diminished and may not be such a robust example of irrational decision making after all.

Teaching Bayesian Reasoning

• Theory is that frequencies elicit normative reasoning because it is easier to picture and understanding than probabilities.
• Sedlemeier & Gierender (2001) examined 3 ways of teaching Bayesian reasoning.
• 1. Rule training: given relevant info. and told which number correspond to which part of the formula
• 2. Frequency Grid: All relevant info. Each square of grid corresponds to which part of formula.
• 3. Frequency tree: Does not represent individual cases but constructs a reference class.
• Results:
• Initially all groups showed an improvement.
• After retention interval rule trainings groups performance declined.
• Frequency representation group retained their knowledge for a period of at least 5-weeks

Why do frequencies elicit normative R.?

• Further reading
• Gigerenzer and Hoffrage (1995) reported that there were more correct answers to the P(H/D) question in the frequency format (around 50\%) than in the probability format (around 20\%). The authors claimed that “Bayesian algorithms are computationally simpler when information is encoded in a frequency format rather than a standard probability format”; i.e., fewer calculations are needed and they can be done on natural numbers. (Ayal & Beyth-Marom 2014)
• Controversially: Despite evidence replicating the frequency-probability effect (e.g., Brase et al., 1998; Cosmides & Tooby, 1996), there is still controversy regarding its underlying process (e.g., Girotto & Gonzalez, 2001, 2002; Lewis &Keren, 1999; Sloman, Over, Slovak & Stibel, 2003). For example, Lewis and Keren (1999)
• “mammography problem” (adapted from Eddy, 1982; see also Cosmides & Tooby, 1996) has been the subject of many Bayesian experimental studies in the last 30 years and illustrates the base rate fallacy
• Fewer pieces of information needs to be stored
• Just absoloute frequencies
• Base rates are not needed i.e.. to know how often the disease occurs in a pop.