Issues of reliability, validity and sampling

The Issue Of Reliability

• Experimental research-
• Reliability refers to the ability to repeat a study with a consistency in results. Conditions must not be changed as this will affect results.
• Observational techniques-
• Observations need to consistent in their measure, therefore two or more observers should get the same record. Inter-rater reliability, 0.80 would be good. This can be increased through training observers to in the use of coding systems/behaviour checklists.
• Self report techniques-
• Two types of reliability: Internal reliability is the extent to which something is consistent within itself. i.e question on a personality test should all be measuring personality. External reliability is the measure of consistency over several different occasions, i.e if a questionnaire is taking once, when taken again over a time period, it should be same or is otherwise unreliable.
• Inter-interviewer reliability- This is the extent to which two or more interviews present the same outcome.
• Reliability can be tested using:
• Split-half method: compare a person's performance on two halves of a questionnaire or test, as long at the test is measuring the same thing (IR) then there should be a close correlation from both halves.
• Test-retest method: give a test/questionnaire at one time, then after a given time period repeat it again with the same stimuli. If there is external reliability then the outcome should be the same.

The Issue of Validity

• Internal Validity: This concerns the extent to which the researcher tests what they intended to test. External Validity: Concerns the extent to which the results can be generalised to the general public (ecological validity)
• Experimental research: internal validity is usually affected by extraneous variables. It is commonly thought laboratory studies are low in ecological validity and field experiments are are high.
• This is not always necessarily true as sometimes the artificial nature of the lab is not relevant to the IV (memory task) and therefore can be generalised.
• It is important to take into consideration whether the participants are aware they are being studied and if the task is artificial and thus low mundane realism and thus reduces generalisability of results.
• Observational techniques: If the coding/behaviour checklist is flawed then observations will not have internal validity, i.e behaviours belonging in more than one category. Internal validity will be affected by observer bias (predisposed ideas and expectations). High ecological validity to to natural behaviour.