Chapter 8

Página 1

Chapter 8: Experimental Designs

A. Purpose of experiments

i. To demonstrate that one variable caused changes in other variables (cause and effect)

ii. Possible experimental questions:

a. Do students learn more effectively with individual or group assignments?

b. Is behavioral activation as efficacious (: Has to do with treatment success- deals with random control trials [rtc’s])

c. Do signs with “no texting allowed while driving” lead to decreasing texting while driving?

B. Causal inference

i. Covariation of X and Y

a. As X varies, Y varies

ii. Temporal order/precedence

a. The variation in X occurs before the variation in Y

iii. Absence of plausible alternatives

a. X is the only factor in a situation that varies prior to a change in Y (high internal validity)

C. Important experimental elements

i. Manipulation

a. One variable is manipulated to take two or more different forms (IV)

ii. Control

a. All other variables must be controlled so that there is no influence on the two variables

iii. Measurement

a. A second variable is measured in order to obtain scored on the variable for the manipulated condition (DV)

iv. Comparison

a. Scored in one condition are compared to those in the other condition

b. Consistent differences in scores provided evidence of the manipulation’s causal influence

Simplest experiment example (posttest only)

D. The value of 2 or more levels of the IV

i. One group (one level) is recruited, the IV is manipulated, and they are measured at pretest and posttest (one-group, pretest/posttest design)

ii. Possible main threats to internal validity:

a. Maturation

1. People naturally change over time, independent of being a participant in a study

2. Spontaneous remission

- “like if someone’s mental health improves on its own”

3. Solution: A control group allows one to rule out the effects of maturation

b. History

1. This threat occurs when a historical or external event occurs to everyone in the study, so it is unclear whether the change(s) occurring are due to the IV or the historical event

2. This event must affect just about everyone in the experimental group

3. Solution: A control group allows one to rule out the effects of history

c. Testing

1. This occurs when the act of measuring participants’ responses impacts how they respond on subsequent measures

2. Score improvement or decline

3. Solutions:

- Only administer a posttest

- Use an alternative form of the test so that it is a bit different form of the pretest

- Include a control group

E. Confounding variables

i. “These are factors that covary with the IV in such a way that it can no longer be determined which one (confounding or IV) has caused the changes in the DV”

a. Examples of possible confounding variables

1. When testing cortisol and testosterone in men and women it is important to account for the natural phenomena that these chemicals in men are elevated in the morning

2. When doing a clinical study your participants may also have other treatments occurring in their life.

b. Reducing confounding factors

1. Keep extraneous factors as constant as possible across the different conditions of the experiment

2. Balance extraneous factors that cannot be held constant

F. Making comparisons

i. Experiments allow researchers to make comparisons

a. Between-subjects: participants are only assigned to one level of the IV

1. Simplest experiment example

b. Within-subject: participants are involved with each level of the IV

c. Between-subject and within-subject: mixed design

Between-subjects experimental design

ii. Between-subject

a. Design options

1. Independent-groups design (typically)

-Participants are randomly assigned to the various conditions of the experiment

- If randomization works…check (how?)

2. Matched-groups design

- A set of participants is mated on at least one attribute

- A member of each pair is then randomly assigned to the various conditions

G. Between-subjects

i. Pros

a. Each participant only engages in one condition…

1. So effects caused by exposure to one condition can’t carry over to other conditions

2. So different but equivalent versions of the same task would not need to be developed (required for within-subject’s designs)

3. Example: Changing the name on the same resume to reflect the ethnicity of the person and see if the reader (hiring person) shows hiring biases

b. Some questions can only be examined through a between-subjects design.

c. It could be inadvisable to have each participant engage in all conditions

1. The task could be too time consuming, cause too much fatigue, or be too stressful

ii. Cons

a. It is more difficult to keep constant participants backgrounds, personality traits, and abilities across all groups/ conditions

1. This means that between-subjects designs are less sensitive detracting experimental effects compared to within- subject designs

b. Sample size

1. Many more participants will be recruited compared to a within-subject

iii. Practice

a. Do students learn more effectively with individual or group assignments?

1. IV: individual or group

2. DV= grades

b. Is behavioral activation as efficacious as cognitive-behavioral therapy in the treatment of depression?

1. IV = treatments

2. DV= depressive symptoms

c. Do signs with “No texting allowed while driving” lead to decreased texting while driving?

1. IV= signs or no signs

2. DV= texting and driving

H. Within-subject design

i. This is also called a repeated-measures design

ii. It is a type of experiment

iii. The SAME group of individuals participates in ALL of the experimental conditions

iv. No control group is necessary, since each participant serves as his/her own control

Within-subject design

^^^^^^ Fewer participants are required, which can be helpful if you are recruiting a sample that is hard to find^^

v. Pros

a. This type of design ensures that participants in the different groups are equivalent

1. The same participants are in condition 1 and condition 2

vi. Cons

a. Exposing participants to all conditions make it more likely that they will figure out the hypotheses

b. If data collection with each person takes some time, then scheduling might be problematic or there could be dropouts

c. Counterbalancing is necessary, which complicates the study

1. The levels of the IV are presented to participants in different orders

2. Necessary because:

-Order effects: These occur when participants’ responses are affected by the order of conditions to which they are exposed

-Progressive effects: these reflect changes in participants’ responses that result from their cumulative exposure to prior conditions

- Carryover effects: These occur when participants responses in one condition are uniquely influenced by the particular conditions or the conditions that preceded it

vii. Design options

a. Concurrent: measures designs

1. Participants are exposed to all levels of the IV at roughly the same time, and a single attitudinal or behavioral preference in the DV

2. Cove v. Pepsi

b. Repeated: measures design

1. Participants are measured after exposure to each level of the IV

2. No pretest is included

3. Oxytocin levels in different toddlers

I. RCT: a special kind of experiment

i. Key factors

a. Centering in on a very specific problem so the inclusion and exclusion lists are very long so there can’t really be random sampling-> there needs to be volunteers to the study with the problem

b. Random assignment

1. Critical to the study

2. Randomization list creation

c. At least one treatment condition and one condition control

1. No treatment (empty control group)

2. Wait-list

3. Treatment as usual

4. Alternative treatment

5. Placebo

ii. Additional factors

a. Issues of expectation

1. Since participants are self-selecting into the study they have some expectation as to what is going on

b. Single-blind study (about medication)

1. The researcher will know what group the participants are assigned to but the participants will not

c. Double-blind study (about medication)

1. Neither the researcher nor the participants know who’s in what group

d. Partial blinding (about psychology)

1. Can’t single or double b/c of consent processes- participants need to know exactly what treatment they’re are receiving, and the administrator needs to know what they are administering

2. Someone else collects the actual data from the participants that are not the PI’s or the people who administer treatment. They are apart of the team, not are blind to the assignment.

e. Treatment fidelity: delivering your treatment as you intended to (non-drug treatment)

1. Operationalize the intervention

-Manuals

2. Training interventionists

3. Supervision of interventions

- Fidelity checks

- Video or audio recording of session being reviewed by someone.

4. If this isn’t addressed in any intervention study we look at for our paper that is a huge critique

f. Recruitment

g. Screening

1. Telephone screening

2. In-person screening

3. Inclusion/exclusion criteria

4. Down the line (1->3) the number of participants will be cut down as you figure out who really is elidable

h. Researchers notes

1. Anything happening out of the ordinary should be on the participants record like if they dropped you need to find out why.

i. Intent-to-treat analysis

1. “once randomized always analyze” – meaning even if someone drops or dies, you still have to keep their information and analyze their involvement in the study

2. Per protocol analysis

iii. What are the benefits

a. There is an applied problem that needs to be solved

b. If there are significant findings, the researcher can say they are due to the intervention

1. Most importantly, study participants’ symptoms have improved

iv. What are the limitations

a. Time-consuming and labor intensive

b. Extremely costly