Publication Details

Abstract

This article introduces generalizability theory (G-theory) to institutional research and assessment practitioners, and explains how it can be utilized to evaluate the reliability of assessment procedures in order to improve student learning outcomes. The fundamental concepts associated with G-theory are briefly discussed, followed by a discussion of the software needed to conduct a generalizability study (G-study) analysis. The article then presents a case study of a G-study analysis; this case study was conducted in order to evaluate the generalizability and dependability of an exam that third-year medical school students complete. The conclusion discusses several situations that institutional research and assessment practitioners are likely to encounter where G-theory can be used to evaluate and improve their assessment procedures in pursuit of improving student learning outcomes.

Keywords: generalizability theory, decision study, reliability, assessment

Additional Information

Authors

Paul W. Sturgis, University of Central Florida
Leslie Marchand, University of Central Florida
M. David Miller, University of Florida
Wei Xu, American Board of Anesthesiology
Analia Castiglioni, University of Central Florida College of Medicine

Copyright © Association for Institutional Research 2022