Validity, Theory and Ethics of Process Data in Large-Scale Assessments and Surveys

Georgetown University - King’s College London Symposium Series, Third Installment in the Series, "Validity, Theory, and Ethics of Process Data in Large-Scale Assessments," December 12, 2025

Organizers: Dr. Qiwei Britt He, Georgetown University; Dr. David Pepper, King’s College London

Integrating Process Data With Explanation-Focused Validity Theory in Large-Scale Assessments
Bruno D. Zumbo, Distinguished University Scholar and Professor, (Tier 1) Canada Research Chair in Psychometrics and Measurement, University of British Columbia

Assessment researchers have typically used process data in descriptive ways, such as improving prediction, refining classification, or enhancing model diagnostics. In this paper, I describe the adaptation of my explanation-focused validity framework that positions process data as evidential structures that support explanatory accounts of item responding. From this perspective, process data contribute to validity not simply by increasing model fit, reliability, or precision, but by enabling the development and testing of explanatory models that illuminate how response processes unfold and how these processes relate to the intended construct. This shift—from descriptive use to explanatory inference—moves process data to the center of contemporary validation practice and underscores their role in strengthening the interpretive meaning of test scores.

1. PRELIMINARIES AND DEFINITIONS
• Product-Process Distinction in Computer-Based Assessment
• Product Construct(s) and Process Construct(s)
2. SHOULD WE FOCUS ON PROCESS DATA OR RESPONSE PROCESSES?
• Following Cronbach and Meehl’s (1955) “explanatory turn,” contemporary scholars (e.g., Markus & Borsboom, 2024; Zumbo, 2023, 2025) emphasize that both observed and latent scores must remain distinct from constructs.
3. CONTEMPORARY NOTIONS OF CONSTRUCTS IN TERMS OF CONCEPTUAL EMBEDDING
4. SHEDDING LIGHT ON PROCESS DATA
5. Closing remarks
• Summary remarks, principal challenges and opportunities, consequences and ethics, and concluding remarks