To facilitate the transition to Bayesian inference we recently created the “Learn Bayes module in JASP” (with support from a grant from the APS Fund for Teaching and Public Understanding of Psychological Science and an Erasmus+ grant from the European Union for the QHELP project). The goal of this module is to illustrate all steps of Bayesian parameter estimation and testing.
In the previous blogpost we outlined the Binomial Estimation analysis, which illustrates the key concepts associated with Bayesian estimation of a simple binomial chance parameter. Here, we continue the journey with the Bayesian Testing analysis, which illustrates the key concepts associated with Bayesian testing of a simple binomial chance parameter. In contrast to the estimation, in which we considered each of the specified models individually, in testing we consider the models as competing accounts of the world, and we assess their relative predictive performance. In the JASP GUI, the analysis can be split into five sections, each with its own purpose.
Data
The “Data” section is designed to specify the data input. We can either use a variable from a dataset loaded into JASP (i.e., a variable that contains successes and failures), we can indicate the total number of successes and failures, or we can enter the observations as a sequence. For example, we can specify the order of red and blue balls picked out of a bag and label a red ball a “success” and a blue ball a “failure”, as indicated in the figure below:
Hypothesis
The “Hypothesis” section is used to specify hypotheses that will be used for testing the underlying parameter θ (i.e., the population proportion of successes). We can specify the hypothesis’s name, the prior probability for the hypothesis, the prior distribution for the parameter θ, and parameters that define the distribution. Here, we specify a prior spike distribution (a distribution with the density concentrated on a single point) at value 0.5 for a hypothesis called Cecil, and a beta (1, 1) distribution for a hypothesis called David. We assign prior probability 1 to both of them, making them equally likely a priori (JASP internally standardizes the prior probabilities to sum to one, so we can interpret them as prior odds).
Inference
The purpose of the “Inference” section is to draw conclusions from the specified hypotheses. The figure above already shows the default Testing Summary table with the prior, P(H), and posterior, P(H|D), probability of each hypothesis, their marginal likelihood, and a Bayes factor, BF, quantifying the evidence in support of a hypothesis vs. all remaining hypotheses. Furthermore, the options allow users to visualize the prior, prior predictive, posterior, and prior and posterior distributions using a range of figures. The predictive accuracy of each hypothesis can also be visualized. The following figure shows the prior distributions of both hypotheses scaled by their prior probabilities using the “Joint” option. Notice that the right y-axis quantifies the probability of the single point for the spike, whereas the left y-axis quantifies the probability density of the continuous beta distribution.
The next figure visualizes the prior predictive distributions side by side, with the observed data marked with a cross, showing that the Cecil allocated higher probability to the observed proportion.
Consequently, Cecil’s predictive accuracy was higher than David’s, which is shown in the next figure.
As a result, the posterior probability of Cecil increased (the height of the arrow is now 0.62, whereas the posterior probability of David decreased to 0.38) and the posterior distribution was updated with the data.
Sequential Analysis
The “Sequential Analysis” section presents the results of Bayesian testing sequentially, one observation at the time (available only with non-aggregated data). The following figure shows how the posterior probabilities of each hypothesis change with the additional observations.
Posterior Prediction
The “Posterior Prediction” section is designed to assess the predictions from the updated hypotheses. It offers the same types of figures as the “Inference” section with an additional option to display predictions for the future sample proportions. The following figures visualize the predictions for a future set of 10 observations based on both hypotheses simultaneously, weighted by their posterior probability (i.e., prediction using Bayesian model averaging). The bars that fall inside the 95% highest posterior interval have a darker color than those that fall outside.
About the authors
