Fall

October 16th

3:30 PM ET

Sam Zhang
CU Boulder

Title: An illusion of predictability in scientific results: Even experts confuse inferential uncertainty and outcome variability

Abstract: Traditionally, scientists have placed more emphasis on communicating inferential uncertainty (i.e., the precision of statistical estimates) compared to outcome variability (i.e., the predictability of individual outcomes). Here, we show that this can lead to sizable misperceptions about the implications of scientific results. Specifically, we present three preregistered, randomized experiments where participants saw the same scientific findings visualized as showing only inferential uncertainty, only outcome variability, or both and answered questions about the size and importance of findings they were shown. Our results, composed of responses from medical professionals, professional data scientists, and tenure-track faculty, show that the prevalent form of visualizing only inferential uncertainty can lead to significant overestimates of treatment effects, even among highly trained experts. In contrast, we find that depicting both inferential uncertainty and outcome variability leads to more accurate perceptions of results while appearing to leave other subjective impressions of the results unchanged, on average.

November 6th

3:30 PM ET

Richard Single
UVM

December 4th

3:30 PM ET

Leonard Stefanski
North Carolina State University

Spring

February 2nd

3:45 PM ET

Rebecca Hubbard
University of Pennsylvania

March 18th

3:30 PM ET

Abigail Crockers
University of Western University