Out of the Basement, Into Decision Making

John Pryor is founder and principal of Pryor Education Insights and one of the leading experts in college survey research at both the national and local level.

eAIR: How did you become interested in assessment and survey research, and how did that transition into a career?

JohnPryor.jpgI wanted to make a difference. I originally started graduate school in cognitive psychology, but it was apparent to me by the time I got my MA that it was not for me. I wanted something more than spending my life writing papers that I thought only five people would ever read. After a long time pondering what I wanted to do, I ended up taking a research assistant job at UVA. I found that the very rigorous training I’d received in methodology and statistics was sorely needed in this field, and the first big project I was given was running a statewide survey of alcohol and drug use in the Virginia public university system. I was essentially doing institutional research, but it was another five years before I heard that term.

I eventually took a job at Dartmouth, evaluating alcohol prevention programs in the college health service department. Once word got out that I knew about surveys, they came to me with results from the CORE Alcohol and Other Drug Survey and asked me to help interpret the results. That led to the Dean asking me to present to the Board of Trustees, and, eventually, the Dean asked if I would consider expanding my purview into other areas of student life. At that time, the office of Student Affairs Research, Evaluation, and Planning was born – as was my career in this area.

eAIR: How did your work at Dartmouth, Higher Education Research Institute (HERI), and Gallup prepare you for starting your own higher education research business?

I’ve been privileged to work with people throughout my career who have encouraged me to grow professionally, so I have always been encouraged to present at conferences, be vocal at meetings, and take on interesting projects. For example, in the 1990s when the World Wide Web became publicly prominent, I happened to be at my desk in the basement of the Dartmouth Health Services (it’s always a basement office, right?) when a new version of the Mozilla browser came out. One of the cool new things in that version of Mozilla was forms capability. Most sites used forms to create mailing lists, and it immediately occurred to me that this was very much like taking a survey.

So I walked down the street to the Kiewit Computation Center and into the office of the associate director. I explained what I was thinking and we talked about how my idea might work. He told me he had just the right person for the job. A few days later, a sophomore with skateboard in hand walked into my office. Between the two of us, we created what I think was the first web survey ever implemented. However, it wouldn’t have happened without the support of the people around me.

Likewise, at HERI I was encouraged to have a high profile, which meant presenting at a lot at conferences, publishing, and using that national platform to have a voice in higher education research. Ironically enough, one of the pieces I got the most recognition for was defending the National Survey of Student Engagement (NSSE) against criticism at an Association for the Study of Higher Education (ASHE) Presidential panel. After that, the NSSE folks forgave me for constantly sneaking Cooperative Institutional Research Program (CIRP) material onto the tables in their booth at conferences.

These two roles gave me a great deal of visibility, which has helped in creating my own firm. Working at the local level at Dartmouth and UVA and at the national level at UCLA and Gallup also gave me experiences that not a lot of people are fortunate enough to have, so there is work out there that just naturally finds its way to me because of that.

eAIR: A quote on the Pryor Education Insights website reads as follows: “It's not enough to do great research. You also have to be able to communicate information effectively and provide the insight that makes it relevant.” What are some ways IR directors can accomplish this on their campuses?

Great question. This was the topic of a master session I did at the AIR Forum and a keynote at Texas AIR. A lot of this I learned when I did my TEDxUCLA talk, as well as trial and error throughout the years – mostly error – and you learn a lot from that.

In IR, we exist to help people use information to assist in making decisions. Not everyone making decisions in higher education wants to sit down with us and go through a 50-slide PowerPoint with pivot tables and links to 50-page reports. Most presidents, provosts, and deans don’t have time for that. They want to talk with someone they trust and get to the bottom line. If you want your hard work to pay off, and if that 50-page report and PowerPoint presentation are worth it, then you synthesize the relevant points and communicate them in a way that gives your audience that “ah ha” moment. To do that you need to get out of the basement and into the shoes of the people making decisions. What is important to them? What do they need to know? How do they process information?

One of my deans at Dartmouth frustrated me when I first started going over results. I’d be showing him something and he’d say, “uh huh, uh huh” as I went through the information. I thought at first that he was bored and wanted me to get to the end and leave. As I watched him more and had more experience, I learned that this was how he behaved when he was engaged. He wasn’t telling me he wanted me to skip to the end; he was telling me that this made sense and he was with me on it. That helped me to provide information in the way he processed it and be more effective in my job. I always encourage IR professionals to get to know the people they are there to help, and figure out how they can best communicate with them. After all, we owe it to the people who fill out our surveys to make that information come alive and be useful. The saddest thing ever is the beautifully written report that stays on the shelf.

eAIR: How can IR and assessment practitioners become part of the decision-making process by leveraging their knowledge of data analyses?

It is less about data analysis and more about communication – that is how you become part of the decision-making process. You have to have very good data analysis skills, of course, as the information has to be totally credible. But it is how you communicate the information that determines if you become part of the decision-making process. We have to be integral to that process, or we will end up sitting in the basement wondering why we weren’t asked to be in the meeting upstairs.