There are multiple statistical tests concerning a single parameter of interest such as t-test or a binomial test of proportions. Those tests can be performed with summary statistics that completely describe the observed data. In this new analysis, we extend the “Summary Statistics” module with a “General Bayesian Tests” analysis that allows us to evaluate the evidence in favor of or against a hypothesis for multiple different likelihoods and prior distribution specifications.
Data
We reanalyze results from a study suggesting that emotional words are remembered better than neutral words. Grider and Malmberg (2008) presented results of several paired t-tests, one of them on the difference in accuracy between neutral and positive words, d = 0.25, t(79) = 2.24, p < 0.05.
Interface
We can quantify the statistical evidence with the help of the General Bayesian Tests analysis which is based on the methodology developed and promoted by Zoltan Dienes (e.g., Dienes 2014; Dienes & Mclatchie, 2018) that was recently added to the Summary Statistics module. General Bayesian Tests allow us to specify different likelihoods for the observed data. We choose the Non-central d and set the observed effect size to d = 0.25 and sample size n = 80.
Next, we follow the re-analysis by Rouder et al. (2009) who specified a point null hypothesis with a Spike distribution at zero and two alternative hypotheses to assess robustness to prior specifications; a Cauchy prior distribution with scale 1 and a standard normal prior distribution. We set both of these under the Null Hypothesis and Alternative Hypothesis headings:
Results
The resulting output summarizes the evidence in favor of each of the alternative hypotheses in the Model Summary table. We find negligible evidence in favor of the null model in comparison to the alternative Cauchy model and negligible evidence in favor of the normal model in comparison to the null model. In other words, from a Bayesian perspective the data show an absence of evidence (Keysers et al., 2020).
To obtain the intuition behind this outcome, we can interrogate the Prior predictions by selecting the corresponding checkbox. When comparing the prior predictions under the alternative hypotheses, we can see that the Cauchy prior distribution spreads the prior predictions more towards the tails (with the prior prediction for 0 only about to reach a density of 0.04) while the normal prior distribution keeps more of the density in the central area (with the prior prediction for 0 almost reaching a density of 0.05).
Since the normal prior distributions assigned slightly more prior plausibility to the area containing the observed data, it fared slightly better against the spike prior distribution than the Cauchy prior distribution.
References
Colling, L. J. (2021). Bayesplay: The Bayes Factor Playground. R package version, 0.9.2, https://CRAN.R-project.org/package=bayesplay
Dienes, Z. (2014). Using Bayes to get the most out of non-significant results. Frontiers in Psychology, 5. https://doi.org/10.3389/fpsyg.2014.00781
Dienes, Z., & Mclatchie, N. (2018). Four reasons to prefer Bayesian analyses over significance testing. Psychonomic Bulletin & Review, 25, 207–218. https://doi.org/10.3758/s13423-017-1266-z
Grider, R. C., & Malmberg, K. J. (2008). Discriminating between changes in bias and changes in accuracy for recognition memory of emotional stimuli. Memory & Cognition, 36, 933-946. https://doi.org/10.3758/mc.36.5.933
Keysers, C., Gazzola, V., & Wagenmakers, E.-J. (2020). Using Bayes factor hypothesis testing in neuroscience to establish evidence of absence. Nature Neuroscience, 23, 788-799. https://www.nature.com/articles/s41593-020-0660-4
Rouder, J. N., Speckman, P. L., Sun, D., & Morey, R. D. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 20, 225–237. https://doi.org/10.3758/PBR.16.2.225
About the authors
