The R code for this analysis can be found in the OSF repository: https://osf.io/ypf6c/
For each outcome, responses were analysed with a mixed two-way ANOVA, treating format as between-subjects factor and R range as a within-subjects factor. The Greenhouse-Geisser correction was applied when within-subjects factor violated the sphericity assumption (i.e., Mauchly's test p-value was significant, p <= 0.05). Significant interactions were followed up with one-way ANOVAs examining the effect of format at each level of R range, then pairwise post-hoc tests. In the case of no significant interaction, significant main effects were followed up with pairwise post hoc tests: Tukey’s post hoc tests for between factor comparisons and Bonferroni-corrected Welch’s paired t-tests for within factor comparisons (note this is a deviation from preregistered analysis plan, which specified Tukey’s post hoc tests for all pairwise comparisons). For figures, brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Experiment 1
Perceived Risk
Considering Perceived risk, there was a significant main effect of R range, F(1.71, 2507.19) = 1772.35, p < .001, η2G = 0.294, but not format, and the effect of the interaction term was non-significant.
Figure 1: Perceived riskiness of the COVID-19 situation for each of the three ranges presented. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001)
Perceived likelihood of upper limit
Figure 2: Perceived likelihood of the upper value of the range being the true value of R, when indirect uncertainty presented in each of the different formats. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Perceived likelihood Covid-19 increasing
Figure 3: Perceived likelihood that the pandemic is currently increasing, given each range for R and each format of indirect uncertainty presentation. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Perceived R certainty
Figure 4: Perceived certainty of R, by format of indirect uncertainty presentation. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Figure 5: Perceived certainty of R, by range. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Surprise if above range
Figure 6: Surprise if the true value of R turned out to be above the range presented, by format of indirect uncertainty. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Figure 7: Surprise if the true value of R turned out to be bove the range presented, by range of R presented. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Surprise if range midpoint
Figure 8: Surprise if the true value of R turned out to be exactly at the midpoint of the range presented, by range of R presented. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Surprise if below range
Figure 9: Surprise if the true value of R turned out to be below the range presented, by format of indirect uncertainty. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Figure 10: Surprise if the true value of R turned out to be below the range presented, by range of R. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Trustworthiness
Figure 11: Perceived trustworthiness of the information, by format of indirect uncertainty presented. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Effort required
Figure 12: Effort required to understand the information, by range of R. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Informedness
Considering Informedness, there was a significant main effect of format, F(4, 1464) = 17.02, p < .001, η2G = 0.036, and R range, F(1.98, 2899.83) = 23.74, p < .001, η2G = 0.003. The effect of the interaction term was non-significant.
Figure 13: How well informed participants felt about the COVID-19 situation after reading the information, by indirect uncertainty format. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Figure 14: How well informed participants felt after reading the information, by R range. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Additional results: Plots comparing means between format conditions at each level of R range
Figure 15: Summary plot of results for perceived risk from COVID-19, perceived likelihood of the true value of R falling at the upper limit of the range presented, the perceived likelihood that COVID-19 was increasing, and the perceived certainty of R by both range of R presented and indirect uncertainty format presented. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Figure 16: Summary plot showing results for the level of surprise if the true value of R fell above, at the midpoint of, and below, the range for R presented, by both range presented and indirect uncertainty format presented. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Figure 17: Summary plot showing perceived trustworthiness of the information provided, how much effort it required to understand, and how well informed it made people feel, by range of R presented and format of indirect uncertainty. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Experiment 2
In the following sections we report results for each outcome with follow-up tests and plotting of main effects or interactions, where significant.
Perceived Risk
Figure 18: Perceived risk of COVID-19 situation after viewing the graph, by range of R represented. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Perceived likelihood of upper limit
Considering Perceived likelihood of upper limit, there was a significant main effect of format, F(2, 1466) = 6.16, p = .002, η2G = 0.006, and R range, F(1, 1466) = 7.37, p = .007, η2G = 0.001. The effect of the interaction term was non-significant.
Figure 19: Perceived likelihood of the true value of R falling at exactly the upper limit of the consensus range presented, by level of indirect uncertainty (Quality of Evidence) presented. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Figure 20: Perceived likelihood of the true value of R falling on exactly the upper limit of the consensus range, by range presented (shapes show overall distribution of responses, dot is mean and lines indicate standard deviation). Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Perceived likelihood Covid-19 increasing
Considering Perceived likelihood Covid-19 increasing, there was a significant main effect of R range, F(1, 1466) = 2872.64, p < .001, η2G = 0.525, but not format, and the effect of the interaction term was non-significant.
Figure 21: Perceived likelihood that COVID-19 is increasing, given the consensus range presented. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Perceived R certainty
Figure 22: Perceived certainty of R, by the amount of indirect uncertainty (Quality of Evidence) portrayed. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Figure 23: Perceived certainty of R, given the consensus range of R illustrated. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Surprise if above range
Figure 24: Surprise if the true value of R turned out to be above the consensus range illustrated, by range of R presented and level of indirect uncertainty (Quality of Evidence) portrayed. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Surprise if range midpoint
Considering Surprise if range midpoint, there was a significant main effect of format, F(2, 1466) = 5.43, p = .004, η2G = 0.006, but not R range, and the effect of the interaction term was non-significant.
Figure 25: Surprise if the true value of R turned out to fall at the exact midpoint of the consensus range presented, by level of indirect uncertainty (Quality of Evidence) portrayed. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Surprise if below range
Considering Surprise if below range, there was a significant main effect of format, F(2, 1466) = 3.42, p = .033, η2G = 0.004, and R range, F(1, 1466) = 58.91, p < .001, η2G = 0.009. The effect of the interaction term was non-significant.
Figure 26: Surprise if the true value of R turned out to be above the consensus range presented, by the level of indirect uncertainty (Quality of Evidence) portrayed. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Figure 27: Surprise if the true value of R turned out to be below the consensus range presented, by range of R. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Trustworthiness
Figure 28: Perceived trustworthiness of the information, by the level of indirect uncertainty (Quality of Evidence) portrayed. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Figure 29: Perceived trustworthiness of the information by consensus range of R portrayed. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Effort required
Figure 30: Amount of effort needed to understand the information, by consensus range of R presented. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Informedness
Figure 31: How informed people felt after looking at the graph, by level of indirect uncertainty portrayed. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Figure 32: How informed people felt after looking at the graph, by the consensus range of R portrayed. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Additional results: Plots comparing means between format conditions at each level of R range
Figure 33: Summary plot showing the perceived riskiness of the COVID-19 situation, perceived likelihood of R falling at the upper end of the consensus range, the perceived likelihood that COVID-19 is currently increasing, and the perceived certainty of R by both the range of R presented and the level of indirect uncertainty (Quality of Evidence). Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Figure 34: Summary plot showing the level of surprise if the true value of R turned out to be above, at the midpoint, or below the consensus range for R, by the range of R presented and level of indirect uncertainty (Quality of Evidence) portrayed. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Figure 35: Summary plot showing the perceived trustworthiness, amount of effort required to understand the information, and level of informedness after seeing the graph, by consensus range of R presented and the level of indirect uncertainty (Quality of Evidence) portrayed. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
Exploratory inter-experiment comparisons
The following plots treat Experiment 1 and Experiment 2 format conditions as independent groups and compare means between different formats, at each level of R range, using Tukey’s post hoc tests. Shaded regions show overall distribution of responses, with mean (point) and 95% confidence interval (error bars). Brackets and asterixis indicate a significant pairwise difference between groups (*p < .05, **p < .01, ***p < .001).
