Comparing the Student Experience Over Time

Using Multi-Layered “Cat-Scans”

 By Jeffrey Cornett
Director of Institutional Research
Ivy Tech Community College – Central Indiana Region
 

CAT-scans track Cohorts Across Time. They illustrate the full range of student experiences over cohort outcome years. Unlike performance metrics that focus on a single outcome type at a single point in time, CAT-scans provide much more information to reconcile all outcomes. Most importantly, examining the full range of student experiences provides useful insight into why and when students leave before graduating.

The simplest CAT-scan looks at just four outcome layers – graduates, retained, transfers-out, and dropouts. The mix of these four outcomes varies widely as you track students over longer periods of time.

VDD1.JPG

Each of these layers can be disaggregated to provide greater insight into the student experience and why it varies. Our 20-layer CAT-scan graphs track 20 different outcomes over nine successive cohort outcome years. The examples shown combine starting student cohorts from fall of 2004-2012.

  • Degree graduates from our college are shown in green in the bottom four layers, and further differentiate to show those who then transferred to another college to continue their education.

  • The two layers in blue show students still enrolled at our college, and are split between those who have successfully completed 30 hours with us or not.

  • The next six layers up in yellow are transfers out who did not graduate with us. Many will graduate elsewhere, or are still in school. Others eventually drop out before graduating. Transfers are further sub-divided based on whether or not students were academically successful at our school before transferring.

  • Other leavers are too often lumped together as dropouts and regarded as retention failures. However, further disaggregation reveals important lessons. Some students leave before earning any credits (black). Other students fail academically (red) with low GPA's. Other layers (light and dark grey) show academically successful students who stopped out or dropped out, and also differentiate between those who left before or after earning 30 credits with us.

VDD2.JPG
CAT-scans lend themselves well to dashboard comparisons, and in so doing, allow users to see in what ways and at what points in time, selected subsets of students differ in how they experience college. For example, the student experience varies widely between college-ready and college-prep students.

Visual Analytic Features:

  • CAT-scans are produced in Excel using pivot-table selections from a custom data mart. This data mart is developed by combining internal data with the transfer history available from the National Student Clearinghouse.

  • The charts are stacked 100 percent area charts; 100 percent of student outcomes are illustrated for each cohort outcome year.

  • Colors for graph layers are carefully selected to encourage grouping similar outcomes together.

  • Colors and boxes are also deliberately formatted in the legend. Numbering the legend labels makes it easier to refer to a specific layer by number, rather than by the detailed description.

  • All starting cohorts of students are combined over the past decade. The early cohort years will have more students. Therefore, the X-axis labels are concatenated to include the student counts for each year.

  • Headings and footings are important. They need to make it easy to identify student selections when comparing dashboard style across diverse cohorts. The second row of the heading is a concatenation of the various fields used in the pivot table selection criteria.

  • CAT-scans are published as interactive PDFs as part of a much larger collection of information for our region in Ivy Tech. Our interactive Knowledge Bank uses dashboards to allow for visual comparisons across graphs. Clicking on an item in a dashboard zooms in on this item as a form of drill-down navigation. Dashboards are accompanied by a page of explanation of the key insights that can be learned by studying and comparing the dashboard graphs.

  View Larger, Interactive Full Version (PDF opens in new tab). 

 

 

 Comments

 
To add a comment, Sign In
Total Comments: 1
 
Jeff posted on 4/15/2016 10:59 AM
The CAT-scan graph examples above are a small fraction of the many uses for this display technique. Our interactive Knowledge Bank uses CAT-scans extensively to help tell the story of Success Rates, Retention, Transfers, and Leavers. To see more of these examples, go to the AIR 50th Anniversary Digital Time Capsule page at:
https://www.airweb.org/AboutUs/History/Pages/Digital-Time-Capsule.aspx

Our core 600-page Knowledge Bank is found about a third of the way down the page. Download the PDF and learn to surf the document. We actually have 1000 more pages of interactive content delivered as separately linked files, including many CAT-scan studies not uploaded to the e-AIR site. Nevertheless, in spite of the broken links, our core file will be enough to illustrate the range of uses.