Meta-Analyses of the Temporal Stability and Convergent Validity of Risk Preference Measures

Alexandra Bagaïni¹, Yunrui Liu¹, Madlaina Kapoor¹, Gayoung Son², Paul-Christian Bürkner³, Loreen Tisdall¹, and Rui Mata¹

¹Center for Cognitive and Decision Sciences, University of Basel; ²Department of Psychology, University of Bern; ³Department of Statistics, TU Dortmund University

About the study

Understanding whether risk preference represents a stable, coherent trait is central to efforts aimed at explaining, predicting, and preventing risk-related behaviours. We help characterise the nature of the construct by adopting an individual participant data meta-analytic approach to summarise the temporal stability of over 350 risk preference measures (33 panels, 57 samples, >575,000 respondents). Our findings reveal significant heterogeneity across and within measure categories (propensity, frequency, behaviour),domains (e.g., investment, occupational, alcohol consumption), and sample characteristics(e.g., age). Specifically, while self-reported propensity and frequency measures of risk preference show a higher degree of stability relative to behavioural measures, these patterns are moderated by domain and age. Crucially, an analysis of convergent validity reveals a low agreement across measures, questioning the idea that they capture the same underlying phenomena. Our results raise concerns about the coherence and measurement of the risk preference construct.

A pre-print is accessible on PsyArxiv. We make the estimated test-retest correlations and inter-correlations from the primary data sources as well as all analysis scripts publicly available in an OSF repository.

About the companion website

This is the companion website for the study Meta-Analyses of the Temporal Stability and Convergent Validity of Risk Preference Measures. Here, we provide additional information on the data, analyses and results reported in the manuscript:

• Data: Information on the sources of the data included in the analyses.
• Workflow: Step-by-step instructions on how to use the files in the OSF repository to replicate the analyses described in the paper.
• Processing: Supplementary overview of the test-retest correlations and inter-correlations.
• Analysis: Supplementary information on the analyses and results.
  • Variance Decomposition: Descriptions of the effects examined for the variance decomposition analysis of test-retest correlations and inter-correlations, separately.
  • Temporal Stability: Model summaries, MCMC diagnostics and posterior predictive checks.
  • Convergent Validity: Model summaries, MCMC diagnostics and posterior predictive checks, as well as robustness checks.
• Multiverse: Summary output showing the robustness of the results of the above-mentioned analyses to variations in the processed data.