When expert predictions fail.

Unlike in controlled settings, social scientists' performance when predicting real-world societal trends is indistinguishable from that of laypeople or naive statistical methods. There is no conclusive evidence that domain-specific expertise improves predictive accuracy of societal change. Causal models in social sciences are oversimplified, confusing levels of analysis, and generally lack specificity about the magnitude or distribution of effects, limiting prediction accuracy. Engaging in counterfactual thought experiments to go beyond the most obvious factors specified in a theory, predictive accuracy in social sciences can be improved. Integrating broad foundational models with context-specific time series data, as in meteorology, can enhance the accuracy of social science predictions. Enhancing analytical training and understanding of temporal dynamics can yield more precise, riskier predictions, the practical application of which requires the fostering of intellectual humility in scientists. We examine the opportunities and challenges of expert judgment in the social sciences, scrutinizing the way social scientists make predictions. While social scientists show above-chance accuracy in predicting laboratory-based phenomena, they often struggle to predict real-world societal changes. We argue that most causal models used in social sciences are oversimplified, confuse levels of analysis to which a model applies, misalign the nature of the model with the nature of the phenomena, and fail to consider factors beyond the scientist's pet theory. Taking cues from physical sciences and meteorology, we advocate an approach that integrates broad foundational models with context-specific time series data. We call for a shift in the social sciences towards more precise, daring predictions and greater intellectual humility. [ABSTRACT FROM AUTHOR]