Data And Money: Are You Tuned Into PIRS?

scale.jpg

Savvy AIR members are watching the development of a rating system that may connect IPEDS data to student financial aid.  

The U.S. Department of Education is developing the Postsecondary Institution Ratings System (PIRS) in response to the Obama Administration's call to publish a new system before the 2015-2016 academic year.

Who in institutional research is paying attention to PIRS? In a recent survey of AIR members, only 54% of those who work in the U.S. noted that they are “very” or “somewhat” aware of PIRS. Also, a large percentage of AIR members are not engaged in the topic. Only 19% have initiated discussions about the ratings system, 24% have participated in discussions initiated by others, and 22% have been asked to monitor news about PIRS.

What do you need to know about PIRS? The U.S. Department of Education explains that PIRS is part of a plan to “tie financial aid to college performance, challenge states to fund public colleges based on performance, and hold students and colleges receiving student aid responsible for making progress toward a degree.” Familiarize yourself with the details of PIRS as they evolve and watch for updates on the Department’s website.

What action do you need to take? Be known on your campus as the person knowledgeable about PIRS. Track PIRS-related developments. Join in—and lead—campus conversations on the topic. Take ownership for the accuracy of your IPEDS data.

What are AIR members asking about PIRS? IR professionals express myriad concerns about the feasibility of PIRS. Can it be effective across institution types? Who will develop the metrics? Will the media morph “ratings” into “rankings,” and will consumers understand the difference? Will it be a burden for institutions in general, and for IR in particular?

Several AIR members were invited to serve as technical experts/presenters at the PIRS Technical Symposium on February 6, 2014, four of whom shared their insights after the meeting.

Braden Hosch
Assistant Vice President for Institutional Research, Planning and Effectiveness, Stony Brook University

My large take-away from the Technical Symposium was that most technical experts agreed that designing a fair rating system that could provide consumer information and also be used for accountability would be extremely challenging with existing data. The Twittersphere had a bingo game during the Symposium for mentions of the phrases “the devil’s in the details” and “unit record system.” Because PIRS is a White House initiative, it will at some point come across the radar screen of your senior leadership; be prepared to brief them on what you know. Watch for the draft methodology for PIRS that NCES is supposed to release this spring. You will want to know what components are included and how your institution stacks up. Examining how competitor institutions fare will also likely be important. Since NCES will likely request additional comments, be prepared to offer feedback through your institution over the summer.

Christine Keller
Executive Director of the Voluntary System of Accountability (VSA) and Associate Vice President for Academic Affairs, Association of Public Land-grant Universities (APLU)

The PIRS Technical Symposium underscored the importance of accurate, complete, and meaningful data on institutions and students as the foundation of any system. The institutional research community must continue to reinforce this message as the discussions about PIRS move forward.

Hans L’Orange
Vice President for Research and Information Resources, State Higher Education Executive Officers (SHEEO)

PIRS is going to happen; it's a matter of "when" not "if." NCES has the unenviable task of addressing a very complex challenge in a short period of time, and I believe they took the comments received at the Technical Symposium to heart. It's also been an interesting sociological experience to observe and participate in the very open discussions around the proposal; the data policy community in particular has been quite involved to date. Everyone will have to be involved quite soon. 

Patrick Perry
Vice Chancellor of Technology, Research and Information Systems, California Community Colleges Chancellor’s Office
 
The PIRS Technical Symposium offered a diverse set of stakeholders an opportunity to provide input on one of the more provocative proposals higher education has faced: should and how might the federal government rate colleges for consumer and accountability purposes. There was a strong convergence around the possible pitfalls and distinctions between these uses, along with many segmental differences about what is important to student consumers. Community colleges specifically have concern about the use of existing IPEDS metrics as the basis for ratings, and a primary focal point for all parties was the creation of some adjustment or peering model that would allow a more realistic comparison amongst similar institutions. Wage outcomes of graduates and the creation of a national student unit-record system were also not discounted as solutions.

The PIRS Technical Symposium was also the topic of a recent IR in the Know contribution submitted by Christine Keller.

Join the conversation. What are your comments and questions about PIRS?

 

 

 Comments

 
To add a comment, Sign In
Total Comments: 7
 
Kimberly posted on 2/13/2014 3:45 PM
I attended as a guest and found the symposium very interesting. I would agree with Hans that the system will be developed so we should all focus attention on providing feedback that will help NCES develop an accurate system - the best system possible given the time constraints. I would also echo Christine's call to our community to "reinforce the importance of accurate, complete, and meaningful data." This is a time for our collective best thinking to ensure that whatever we end up with can truly reflect the good work being done across the higher ed spectrum.
Mike posted on 2/13/2014 4:10 PM
Of the suggestions that I've seen, I like the APLU's best. Don't even try to measure educational quality, instead look at a few basic bedrock items to identify the bad apples in education. Or if we want to call it a ratings system, design it to identify low rated schools rather than they to find a way to identify highly rated schools. USNews and others have been attempting to do that for decades and their rankings are no better now than they were decades ago -- because it's a useless quest to begin with.

I also like how the APLU stayed away from using wage data, yet acknowledged the demand for information about post-graduation activity. A combination of looking at employment and post-graduate study, though still somewhat crude, is better than the alternatives.

One thing which I would have liked to have seen mentioned more at the symposium and in the suggestions: looking at alumni only a year or even five years after graduation is too short-sighted. Five years after receiving their bachelors degrees, most future PhDs (and for that matter MDs) are going to still be in grad school with tiny incomes. I'd guess that it takes at least eight or ten years after graduation before we start seeing measures that can adequately give us an idea of the alumni outcomes.
Heather posted on 2/13/2014 8:04 PM
I was aware of changes but this article has made a compelling case for me to become knowledgable and a resource on my campus. Thank you.
Jeanne posted on 2/14/2014 9:38 AM
PIRS was not on my radar until reading this article. As with other initiatives that tie funding to performance, I am concerned about how states will be able to compare all state institutions equitably. With the short timeline for implementation, institutions needs to be more active in the discussion to make sure that the system is fair to everyone. Great article!
Katherine posted on 2/14/2014 1:56 PM
I confess to being very nervous about this initiative. It seems like it would be incredibly complicated to provide enough context for this to be fair and useful to anyone. As far as the suggestion that it be used instead as a measure of meeting minimum standards...isn't the accreditation process already supposed to do this?
Gerry posted on 2/15/2014 9:24 AM
http://www.insidehighered.com/quicktakes/2014/02/07/comments-college-ratings-plan

Where you stand seems to be a function of where you sit (not original with me). I do think that AIR should make a statement since this is one of our core areas. The web site is one of the better,
Meg posted on 2/20/2014 3:48 PM
Thanks for keeping the rest of us in AIR up-to-date. As a member of the President's Executive staff, I will be sure to share that I am the person knowledgeable about PIRS.