Outcome Measures: Metrics Beyond First-Time, Full-Time Cohort

Richard Reeves is Chief, Postsecondary Branch, National Center for Education Statistics (NCES). Prior to joining NCES, the research arm of the U.S. Department of Education, Rich worked for two universities where he conducted survey research, reported to IPEDS, and supported enrollment management activities. In his current role since 2013, Rich has overseen several initiatives to increase the accessibility, visibility, and utility of the Integrated Postsecondary Education Data Systems (IPEDS) data through the updated website; has published broad-language IPEDS brochures; and has recently launched the new Outcomes Measures (OM) and Academic Libraries survey components. Professionally, when not doing IPEDS activities, Rich is immersed in studying organizational leadership and working on quality issues related to administrative data.

rr1.jpgeAIR: Please provide some background/context for the release of Outcome Measures (OM) data.

For years, the postsecondary research community has needed a set of metrics that describe student completions beyond the first-time, full-time degree/certificate-seeking cohort. The development of OM has been in the making since the 2008 HEA reauthorization. It takes years to confer with the industry and build a new survey instrument. OM provides the public with award and transfer-out data for students that begin at an institution, transfer into the institution, and attend full-time or part-time.

eAIR: Why is the release of the OM data significant for NCES and IPEDS? For the IR community?

The IR community and researchers will have three more cohorts that allows for benchmark comparisons as well as have a completion metric that is representative of their student body. This will facilitate comparison between institutions and allow for data-driven policy/program conversations about student progress and completion. While NCES gathers data to be used, we are excited that the data can also be used for consumers, namely part-time and transfer students who are seeking completion information on similar students.

eAIR: What is important for IR professionals to know as they begin to use the data? What are the advantages and disadvantages as compared to the more commonly used federal graduation rate?

The Graduation Rates (GR) cohort is limited to the extent that, for many institutions, it represents a small fraction of their student enrollment. OM offers completion and enrollment measures for four cohorts after eight years. OM data can give institutions the opportunity to spotlight their efforts in helping their students to transfer out when the focus prior to OM has been only on the rate of students that have completed a program by time to degree. With both OM and GR cohorts and their outcomes, there is less standardization in how these data will be used in comparisons. Rather, institutions should consider how these measures can aid in telling a more complete narrative about their students.

eAIR: Do you have any examples of how the data are being used so far?

It is early yet. College Navigator has the data displayed at an institutional level. We have seen the national and local media begin generating some data analysis on the 2016-17 OM. I have yet to see research studies or policy analysis using the data. We hope to have the 2015-16 OM data released in early 2018. Then there will be two years of OM data for analysis.

eAIR: What changes or enhancements are planned for this year’s collection of OM?

For the past two years, NCES has developed OM changes that will be in effect in the 2017-18 data collection. There are four major changes: 1) All institutions will report on a full-year cohort regardless the institution’s reporting type, 2) the four cohorts each have two subcohorts of Pell Grant and Non-Pell Grant recipients, 3) there is a new status point of reporting awards at four years after entry, and 4) awards reported will be reported at the highest level and disaggregated by undergraduate award type. These changes will provide the public both more inclusive measures and better data on outcomes for the most economically disadvantaged students. While these are large changes to OM we do not anticipate any additional changes in the near future. For more information, visit the OM resource page. You can also watch the newly released Outcome Measures web conference presentation by Gigi Jones, Outcome Measures Survey Director.

eAIR: How do you see IPEDS changing over the next decade?

Over the past year we have attended Technical Review Panels (TRPs) about subbaccaluareate certificates, distance education, gender definitions, OM, and data tools. The National Postsecondary Education Corporative (NPEC), an independent advisory group that sets the IPEDS Research & Development agenda, has recently commissioned papers on dual enrollment, finance statistics, distance education, graduate students, transfer students, and student financial aid. We know that these growing areas of higher education have an impact on the IPEDS collection, and I anticipate both more TRPs on these topics and ultimately proposals to address some of these issues in upcoming proposed materials. IPEDS changes its collection through a public process whereby NCES proposes changes to the Office of Management and Budget announced in the Federal Registrar. We announce these proposals in the This Week in IPEDS email and through our Twitter feed IPEDS_NCES.

eAIR: Is there anything else you would like to add?

Yes, I would like to take a moment to thank the dedicated institutional researchers that work to provide timely and accurate data to IPEDS. It is their dedication and professionalism that makes IPEDS the quality data source it is. I would also like to thank Gigi Jones for her work on OM; her efforts have made this survey component successful. We have an excellent team of federal employees and contractors supporting the IPEDS collection. I am continually amazed at the level of industry support we receive and with IPEDS’ ability to work with the industry. This is not a new circumstance, but one that has been a successful work in progress for decades.  




To add a comment, Sign In
There are no comments.