Nonparametric statistics

Nonparametric statistics

1. Logic of hypothesis testing

   • More often we reduce steps 3, 5 and 6 to a single step
   • Compare our obtained value of the test statistic with a critical value
   • The critical value marks the border of the rejection region
   • If our obtained value is more extreme than the critical value
     • Then it must be in the rejection region
     • So its probability must be less than a
     • So reject H0

2. Nonparametric tests

   • The binominal sign tests makes no assumptions about
     • The population from which the data were drawn
     • The level of measurement in the data
   • Are particularly suitable for psychological research
   • They make few assumptions and so can be used in a wide range of situations
   • They are usually easier to understand and conduct
   • They are flexible and can be customised for your own research question
   • They are only slightly, if at all, less powerful than conventional tests

3. Parametric tests

   • Examples are Student's t and ANOVA
   • They assume
     • Data come from normally distributed populations
     • With equal variances
     • At least interval-level measure
   • Is those things are true, they are usually the most powerful tests, so should be used
   • But if the assumptions are violated
     • The shape of the sampling distribution changes
     • The location of the critical value changes
     • The probability of falsely rejecting H0 changes
     • The hypothesis test becomes innaccurate

4. Power, efficiency, and asymptotic relative effuciency

   • The power of a test is the probability that it will reject H0 when H0 is actually true
   • power = 1 - p
   • Power depends on sample size - any test becomes more powerful as N increases
   • Efficiency is the sample size a test needs to achieve a particular powet
   • Relative efficiency is the ratio of sample sizes needed by two tests to achieve the same power
   • Because relative efficiency changes as both sample sizes change, two tests are usually compared in terms of asymptomatic relative efficiency or ARE
   • The ARE of Test B is the limit of its relative efficiency when sample size becomes very large
   • Used to compare power of a nonparametric test with its parametric equivalent
   • Usually, ARE of a nonparmetric test is a little less than 100%, so a little less powerful than its parametric rival
   • But note that this is done making the parametric assumptions
   • If those assumptions are violated the nonparametric test can be much more powerful

5. Classes of nonparametric tests

   • Rank-randomisation
     • What people normally mean when they refer to nonparametric tests
     • Ordinal data
     • Assume ordinal-level measurement
     • Transform data to ranks
     • Easy to carry out, tables of critical values exist
     • Problem
       • Some loss of power
   • Data-randomisation
     • The best possible kind of test but they're hard to carry out with anything more than very small sampels
     • Interval data
     • Assume interval-level measurement
     • ARE = 100%, higher if parametric assumptions violated
     • Problems
       • Difficult to carry out if sample size is medium to large
       • Sampling distribution is different for every data set
         • Can't produce tables of critical values
         • Must always carry them out from first principles
       • Can't be used with ordinal-level data
   • van der Waerden normal-scores
     • A refinement on rank-randomisation tests, making them a little more powerful, but at the cost of being harder to do
     • Ordinal data
     • Assume ordinal-level measurement
     • Slight refinement on rank-randomisation tests, giving increased power
     • Transform data to ranks and then to z-scores
     • ARE = 100%
     • Problems
       • A bit harder to carry out than rank-randomisation tests
       • Not well known