Statistical Inferences Test

Daniel Lakens
Quiz by Daniel Lakens, updated more than 1 year ago
Daniel Lakens
Created by Daniel Lakens over 3 years ago


These are 10 questions from the practice exam from my free Coursera Course "Improving Your Statistical Inferences". If you had a 100% score, there's no need to do this free course. If you didn't score too well, no worries! We can all improve our statistical inferences - check out 21 free lectures, and 10 assignments, here:

Resource summary

Question 1

After finding a single statistically significant p-value we can conclude that _____, but it would be incorrect to conclude that ______.
  • The data is surprising if we assume there is effect; the null hypothesis is likely to be false.
  • The null hypothesis is likely to be false; the alternative hypothesis is true.
  • The data favors the the alternative hypothesis; the null hypothesis is likely to be false.
  • The data is surprising assuming the alternative hypothesis is true; there is a true effect.

Question 2

You analyze your data in two ways. With Frequentist statistics you find an mean effect size of 3, with a 95% confidence interval of 1 to 5. With Bayesian methods you find a mean of 2.75, with a 95% credible interval of 1.5 to 4. Which conclusions can you draw?
  • Both intervals give you estimates for the most probable values of the true mean.
  • The confidence interval gives you the values you believe are most likely based only on the data, the credible interval gives you the most probable values given the data and your prior.
  • Both procedures give intervals which contain the true value 95% of the time, but credible intervals are more accurate.
  • The confidence interval contains the true value 95% of the time, the credible interval contains the 95% most plausible values.

Question 3

What are the benefits of performing a study with a larger sample size, compared to doing the same study with a smaller sample size?
  • Lower Type 1 error rates, lower Type II error rates, same accuracy of estimates.
  • Lower Type 1 error rates, lower Type II error rates, higher accuracy of estimates.
  • Same Type 1 error rates, lower Type II error rates, higher accuracy of estimates.
  • Same Type 1 error rates, lower Type II error rates, same accuracy of estimates.

Question 4

You performed a p-curve analysis and found a skewed distribution of p-values with many more small p-values (around 0.01) than high p-values (around 0.04). What does this mean?
  • This distribution is typical for when there is no true effect.
  • This distribution is typical for when there is a true effect.
  • This distribution suggests the presence of p-hacking.
  • This distribution indicates there is publication bias.

Question 5

How do we know there is publication bias in favor of significant results? Why is it unreasonable to expect articles with 4 experiments that aim for 80% power to exclusively show significant results?
  • We can’t be sure there is publication bias; because of the Type 1 error rate we will eventually get a non-significant result.
  • It’s impossible that 91.5% of findings in psychology confirm the hypothesis; the type 1 error rate makes that unlikely in the long run.
  • Researchers have admitted to not submitting all their results for publication; it is impossible that all hypotheses a researcher examines are true.
  • It’s unlikely that the observed drop in the p-value distribution for p-values higher than 0.05 in the published literature occurs by chance; If all four studies examine a true effect the probability of only significant results is power*power*power*power.

Question 6

You predict that your intervention will significantly increase participants' performance on a test, this is an example of _____. You find a significant result and conclude your theory is true, this is an example of ______.
  • A progressive research line; affirming the consequent
  • A falsifiable prediction; denying the consequent
  • A degenerative research line; denying the consequent
  • A falsifiable prediction; affirming the consequent

Question 7

A researcher reports two significant findings testing the same hypothesis, using an alpha of 5%. The researcher predicted one finding before doing the study, but the other finding was observed during exploratory analyses where many tests were performed. Which statement is correct?
  • Because the researcher performed two analyses, the alpha level should be divided by two for each test to bring the overall error rate back to 5%.
  • Because the second hypothesis was not predicted, the overall error rate is inflated by an unknown amount.
  • The predicted finding is confirmatory, the unexpected finding is exploratory. The error rate for each finding is maintained at 5%.
  • The exploratory finding has increased the Type 1 error rate to 10%.

Question 8

We compare model A (the effect is 0) to model B (the effect is 1) and find a Bayes Factor of 10 in favor of model A which means _____.
  • The prior probability of model A is 10 times larger than that of model B
  • Model A is 10 times more likely to be true than model B
  • The data favor model A 10 times more than model B, given the prior
  • We can have a strong personal belief in the absence of a true effect.

Question 9

You performed 6 studies, only 4 of them had a significant result. The likelihood ratio of this happening assuming H0 (no effect) is true versus assuming H1 (a true effect) is true tells you ______. If you assume the study had around 80% power to detect a true effect, this likelihood ratio will probably show that _____.
  • The likelihood of H0 being true, and the likelihood of H1 being true; it is not unlikely to have some significant effects, even if H0 is false
  • The likelihood of the data, assuming H0 is true; it is unlikely to get 4 significant results out of 6
  • How much more or less likely H0 is, compared to H1; it is likely to have some non-significant effects, even if H1 is true
  • If 4 significant studies out of 6 is likely to happen if H0 is true; it is unlikely to get 4 significant results out of 6, assuming H0 is false

Question 10

An example of a standardized effect size is ______; these should always be reported because the are useful for _____.
  • Minutes; a-priori power analysis
  • Cohen's d; comparing the effect size across studies using different measures
  • Meters; calculating less biased effect size estimates
  • Eta-Squared; controlling Type 1 error rates
Show full summary Hide full summary


8 eLearning trends teachers should know about
How to Increase Student Engagement in the Classroom
Study unit 1
chandra Sinclair
Collaborative Learning
Figure 4.1 Four Categories of Learner Characteristics
Idania Arroyo
How effective is making maths and science mandatory in Yr. 11 & 12 for students in Australia?
6 Teaching Techniques you Should Know!
Andrea Leyden
Putting the Flipped Classroom Model into Practice
Social Media in the Classroom
EDU 340 Final Review Chapter 17 - 19
Stephanie Corlew
EDU 340 Final Review Chapters 20 - 23
Stephanie Corlew