Over 80% of IR Dashboards Not Publically Accessible

Communicating data to decision makers is a large part of institutional research—one that has changed a great deal with the availability of web-based tools, such as data dashboards. A recent AIR member survey explored the extent to which institutional researchers use and maintain dashboards to communicate data.  

Over half of respondents (54%) indicated that their offices manage one or more dashboard. While the IR office has primary responsibility for maintaining dashboards (82%), demonstrating that many IR offices have the staff talent needed to do this work, a small proportion (11%) depend on IT staff to manage dashboards.  

IR dashboards have multiple audiences, with campus and institution administrators being the most common (92%). Additional audiences include faculty, Boards of Trustees, and the general public. However, most IR dashboards are not open to the public (85%) and nearly one-half (45%) of these dashboards are limited to select campus administrators. As such, the limited and targeted audience form of dashboard is the most common. 

Survey respondents were invited to provide URL links to dashboards they find particularly useful. A review of these examples revealed a wide interpretation of what defines a data dashboard. While some examples include dynamic charts that allow for user manipulation over a variety of variables for multiple institutions and state systems, others linked to PDFs or online images of static charts.   

These publically available examples represent a fraction of IR dashboards, but raise important questions related to dashboard use. What is your dashboard’s audience, and is the availability to manipulate data (or lack thereof) your goal? Is your dashboard’s investment adequate? Should the number of users served be considered in evaluating the cost of developing and maintaining a dashboard? If aiming at multiple audiences, is the cost to build in flexibility justified? It is likely that multiple audiences add to the complexity of designing and managing effective dashboards. In many cases, they are available to a variety of stakeholders and must effectively communicate information to a broad range of users. 

When asked what dashboard qualities or features members value most, a number of characteristics were reported. Members prefer clear, visual dashboards that are interactive and flexible. The word cloud identifies the most common responses.  

Finally, the AIR survey asked about the types of follow-up that IR offices undertake as part of managing their dashboards. One-quarter (24%) have reviewed the number of users who visited their dashboards, 14% have evaluated user response to the dashboards, and 36% have invited users to provide feedback. It is good practice to have on-going evaluation and feedback from visitors to dashboard sites, allowing IR offices to conduct research on their own activities. Even placing a feedback button on the page can make it easy to learn what visitors to the dashboard think about it and offers them an opportunity to suggest improvements. 

Interested in learning more about dashboards? Register for AIR’s upcoming webinar, Excel Dashboards: From Theory to Practice, October 22, 2013. You can also see the complete survey results on the AIR website.   

 

 Comments

 
To add a comment, Sign In
Total Comments: 24
 
Heather posted on 10/9/2013 7:46 PM
This is a nice, brief summary of survey results. Interesting findings. I could see additional surveys based on some of this information being done.
George posted on 10/9/2013 8:25 PM
This is indeed an interesting study. I working with dashboards as a consultant in the for-profit world, I think true dashboards have two principal requirements. First is that they reflect real-time, or near real-time data. The second is that it should have limited interactivity (think of an executive summary, but with graphics).

I am interested in considering a dashboard type interface for specific audiences at my institution. Does anyone have any recommendations?
Julia posted on 10/10/2013 12:03 AM
Additionally, it would be interesting to find out what platform other institutions are using for their dashboards. For instance, is Microsoft SharePoint being used as a solution?
Lynn posted on 10/10/2013 8:33 AM
I appreciate the thoughts on justifying cost. Much of the data we deal with is cyclical, by year or term, so a high-tech commercial dashboard may not be the best use of resources.
Meg posted on 10/10/2013 8:48 AM
Very well written. Thank you. While I feel that our institution is using the IR dashboard to reach the desired audience, I realize that maybe stating that purpose/goal more clearly may be needed.
William posted on 10/10/2013 9:01 AM
This is a very interesting article that poses a number of questions. I think the most useful portion of this is the fifth paragraph that poses the questions of what is the purpose and audience for a dashboard. I am neither surprised nor do I think it is a bad thing that 80% of them are not publiclly accessible. In many cases dashboards are designed for management information and strategic planing, not transparency and accountability. The issues summarized in dashboards are often quite complex and can't effectivelt be boiled down to a bar chart that members of the genral public will understand effectively. We also live in a world where the media scans web sites and reports on what is found there with little to no explanation of context.

Bill Knight
Danielle posted on 10/10/2013 9:11 AM
This is a great summary of the survey results. Perhaps a call for best practices in dashboards could help some of those members that are looking to implement or improve their institutional dashboards.
Mary Ann posted on 10/10/2013 9:15 AM
Excellent summary! I like the link to the dashboard examples, the questions the author raises towards the end, & the word cloud.

I completed the AIR dashboards survey, so I'm familiar with the questions on it, & this author did an excellent job synthesizing both the quantitative & qualitative data from it.
Ghenet posted on 10/10/2013 9:25 AM
The summary is well organized. It would have been nice to mention the total number of participants at the beginning of the narrative.
Veena posted on 10/10/2013 9:48 AM
This is a very interesting study. I will definitely register for the webinar on Oct 22nd. We recently created an IR dashboard using excel spreadsheet. But it’s not based on the live data, rather it’s based on term to term static data and we update it only twice a year. I am interested in knowing what indicators other institutions are showing on their dashboards.
Angela posted on 10/10/2013 9:52 AM
Very interesting. I'm curious as to which software is used by most institutions to construct dashboards. The survey didn't seem to ask this, but I wonder if most IR offices are using Excel or if they have purchased dashboard/data visualization software.
Eric posted on 10/10/2013 10:41 AM
This is a very timely feature for me! About two weeks ago I updated my institution's enrollment dashboard (using Excel with static data) which is shared internally with the administrative team. What I found interesting since I created it last year is the lack of feedback from administrators. They have expressed their interest and support for it, but when I ask for specific feedback they have nothing to suggest. I'm not sure whether that reflects on the usability of the dashboard or the administrators' non-use of it. This year I semester I will expand access to it for a larger internal audience.

Thank you for sharing the results of the AIR survey!
Tim posted on 10/10/2013 10:42 AM
A good introduction to the webinar occuring on this topic.
Lucien posted on 10/10/2013 10:48 AM
Frankly, I did not think the percentage was so high (information being limited publically). I find that the goal from my three previous and current institutions was/is to filer all information through one banner system, but roadblocks along the way were present, espeically the financing of a new system. Adidtionally, having a lare percentage of individuals with access can create vulnerability for the institution. Granted, most of the information is for public use; however, institutions prefer to control how the information is portrayed to the public...for image purposes.

Great article...
Mary-Lou posted on 10/10/2013 11:06 AM
The article is timely and relevant and provides good advice. I will surmise that most of us live in the world of dashboards, score cards, and/or Key Performance Indicators. In fact, I recently gave a presentation on a scorecard our office developed for the College's strategic planning efforts.
Marlene posted on 10/10/2013 12:08 PM
This is a very timely and informative article. There are many considerations for dashboards. At my institution, I even just attended a training that would enable users to build their own dashboard pages in the main dashboard tool. I think that it is very valuable to have that kind of flexibility and interactivity.
Ken posted on 10/10/2013 12:30 PM
One of the key elements of the article/findings is the statement that "they are available to a variety of stakeholders and must effectively communicate information to a broad range of users." I think that the article captured the essence of this in the study and the outcomes. While the study indicates that the effort to address numerous users of the information, the baseline of outcome should address users in terms of needs and this article raises that important approach. K. Scott
Jon posted on 10/10/2013 2:36 PM
Great article. We have worked with a number of IR offices on dashboards and found the Administration/Board dashboards is by far the most popular use. Less than 5% of IR offices that use iDashboards publish them externally. From the public facing initiative, here are some different examples:

http://nesccbi.northeaststate.edu:8081/idashboards/?guestuser=guest&dashID=145
http://166.122.30.183:8080/idashboards/viewer/?guestuser=guest&dashID=35
http://166.122.30.183:8080/idashboards/viewer/?guestuser=guest&dashID=29
http://dashboard.llcc.edu:8080/idashboards/?guestuser=guest
http://idashboards.daltonstate.edu:8080/idashboards/?guestuser=guest
https://dashboard.csusb.edu/idashboards/?guestuser=idashguest&dashID=35

It seems that the feedback has been the public initiatives are not high on the priority list.
Barb posted on 10/10/2013 4:05 PM
Thanks for this timely and informative write-up of the survey results! It too am interested in the platforms people are using for dashboard creation.

I did take Craig's Excel Dashboards workshop (check it out, folks! great stuff!), and we're currently building out some dashboards in Tableau. Although the data we connect to is not updated frequently, we do not have to manually update the data and it allows dashboard users to "drill down" to unpack interesting patterns behind big-picture numbers. We can also use it for presentations and to answer questions on the fly.
Eric posted on 10/10/2013 4:26 PM
I'd be interested to see a more feedback regarding state governing boards and/or system offices. In Mississippi we receive a large number of requests from the media, other institutions, and the legislature. We found that making our dashboard reporting tools public has reduced the amount of time we spend on common ad hoc requests and extremely useful when needing to access a wide range of data in a short period of time since they are accessible on a mobile device. We utilize Tableau and these reports can be found here: http://www.mississippi.edu/research/idp.html
Felice posted on 10/11/2013 1:53 PM
I find this topic extremely relevant and useful for practitioners at all types of institutions; I used to maintain a comprehensive internal site for the research & planning office and we experimented with a variety of 'dashboards'. I found that each audience had specific needs and timetables and I started to customize those dashboards for each audience, but it took an extraordinary amount of time to keep up with the work. Since I was essentially a one-person IR shop, I had to revert back to a more institution-wide indicator report and use meetings or presentations to customize the information.
Garnett posted on 10/16/2013 4:30 PM
I think that training in dashboard development and presentation is critical. I took the AIR Excel Dashboard class a few years ago and it and the templates that came with the class were a hugh boost in getting me started. Since then, I have become proficient in Tableau and Q1 Macros. Training is the key.
Sheila posted on 10/24/2013 10:37 AM
I think this is a great article, and the timing couldn't be better! As we begin a new academic year, it is important that IR personal know the importance of data. Dashboards are a great way to tell an institions story, but if not shared with the right people, they are in vain.
Lisa posted on 11/7/2013 1:29 PM
Great article, interesting results. If a follow-up survey is conducted, I would be very interested in knowing how decision makers use dashboards in their decision making. To what extent do dashboards impact the decisions they make in the short- and long-term? What is the return on investment considering the amount of time and effort it takes (Felice's comments).

Eric makes a very valuable point - how dashboards cut down on ad hoc data requests and questions. Thanks Jon for additional examples.