MBH RIG: Optimizing the Discrepant Results When Using, Interpreting, and Integrating Assessment Data

InCHIP Mind-Body Health RIG Talk
“Optimizing the Use of Discrepant Results When Using, Interpreting, and Integrating Assessment Data”
Andres De Los Reyes, PhD, University of Maryland
Researchers often administer multiple instruments designed to measure the same domain. In research, scores taken from multiple instruments may be used to assess their psychometric properties (e.g., convergent or discriminant, criterion-related validity) or to address substantive research aims (e.g., identifying risk factors for mental health concerns, predicting treatment outcomes, identifying evidence-based therapies). In applied settings, these same instruments may be used to make high-stakes decisions about individuals (e.g., diagnosing, treatment planning, response monitoring). Several decades of research indicate that in both applied and research settings, two instruments designed to measure the same domain commonly produce discrepant results when assessing individuals (De Los Reyes, 2024). By “discrepant results” I mean distinct estimates about the same domain in terms of its form, function, level, and relations to other domains. Yet, for nearly 70 years, researchers have made sense of these discrepant results using paradigms that assume that results that converge contain all the valid data (e.g., Converging Operations, multi-trait, multi-method matrix; Campbell & Fiske, 1959; Garner et al., 1956). This assumption translates to use of data integration strategies that misclassify discrepant results as measurement error when these results, in fact, contain valid data (Makol et al., 2025). Grounded in new training and coursework resources (https://bit.ly/3PS9jBj), this talk reveals a falsifiable approach to “diagnosing” discrepant results that is not only conceptually grounded, but also directly informs such practices as selecting data sources, constructing assessment batteries, and integrating data.