The Institutional Factbook

Kallina_Wendy.jpgBy Wendy Kallina

Nearly two decades ago, as a faculty member coordinating a major, I was referred to the factbook when looking for metrics for a five-year program review. Trying to understand the graduation rate and retention rate that I was seeing, I remember being absolutely floored when the assistant registrar confirmed that I should only count freshman students who started their first term in my program. Less than 5% of my students were in that group – how could that be right? My scope was limited to the metrics I was told to include in a program review, my frame of reference was programmatic, and I am not sure that I even thought about why those definitions would matter beyond my department.

Nearly two decades later, I was invited to a multi-departmental meeting to specifically talk about institutional metrics. I opened the conversation with a review of the data requests from the chairs and assistant deans. During the discussion of rates and trends with this very engaged group, I recognized the puzzlement of being a faculty member trying to understand data without the larger context. I asked for some time to talk about definitions, data sources, and IPEDS and they agreed. Within minutes, the conversation changed as they understood the source of the definitions for many of the metrics they were being held accountable for, how they could access additional institutional and comparison data related to those metrics, and how to find data in the factbook.

Since that meeting, I have been thinking about the institutional factbook. Technology has allowed us to move away from expensive printed factbooks with their glossy covers and beautiful pictures. Some institutions offer the factbook in PDF format, which has allowed even more historical information, alumni biographies, student work, or fun facts to be included without the accompanying printing costs. Other institutions are offering a more interactive format and users can print selected information, tables, or charts. There also appears to be a group of institutions that have abandoned the title of factbook altogether in favor of an offering of interactive tables on the IR website. Overall, despite the changes in format, the actual data offerings tend to remain the same.

All of this to say that we have a history of investing time and effort into our factbooks but less time into explaining their contents and promoting their use. We can all point to our data super users, but are we assuming a level of understanding about institutional metrics that doesn’t universally exist in our faculty and staff? As I was reviewing multiple factbooks for inspiration, I couldn’t help but wonder if we create factbooks for other institutional researchers. Is it enough for our average consumer to add a note to our graduation rate chart that we use a first-time, full-time freshman cohort?

What is first-time? Ever in college or at that institution? How many classes count as full-time? What is a cohort, anyway?

We try to avoid the use of academic jargon with students, especially first-generation students, because we know it is a barrier to communication. Are we, as institutional researchers, using our own jargon? I will confess to having to backtrack and provide a definition more than once when talking to a broader academic audience.

For most of the college community, the factbook is the primary means of interaction with the institutional research office. Is there a way to leverage the factbook to facilitate the growth of data literacy across the institution? How can we provide additional information or context to enable the reader to actually make sense of the data? Put another way, how can we explain what the data are for an audience who doesn’t work with the data on a regular basis? Or how do we reach the audience who hasn’t fully realized the potential uses for the data?

The form and structure of factbooks has changed and, in some cases and for some users, these changes have likely been dramatic. Perhaps it is time to turn our attention to the function of our factbooks. If the function is to report and our improvements are focused on making reports easier to access and customize, then there has been substantial movement in the right direction. If function includes some interpretation or assistance in understanding and use of the data, then there hasn’t been much movement at all.

I am not sure how this movement would happen. An institution-wide training is impractical for many reasons, but maybe some combination of resources, embedded in the factbook or easily accessible through the IR office could be considered. It is becoming increasingly clear that access to the data is not enough, people need to know how to use the data. Our institutional factbooks could serve not only as a source of data but as invitations to explore the data.

 

 Comments

 
To add a comment, Sign In
Total Comments: 1
 
Leslie posted on 12/20/2018 3:41 PM
Wendy,

On our campus, I have realized that many staff and faculty do not know how to interpret data and therefore do not use data because it is confusing to them. We do not have a factbook for our campus (our system of colleges produces one), but we do have a Fall Academic Profile that is released each February.

I am a one person IR office and I began having IR presentations this year. The first one was well attended and I have been asked to present this same information at our next in-service. That presentation explained the hows and whys of our federal and state mandated reporting and many did not know why our college has to do this reporting. I have decided to hold a couple presentations each year and cover different topics, and I have had suggestions to hold a presentation on how to interpret data and how to read charts.

I think of this as the IR office internal marketing strategy. It is an effort to begin to make others aware of all the IR does and to offer help to those that may be confused about data and the various definitions, so they can begin to understand data and how to use it more effectively.