Avoiding Data Warehouse Pitfalls

Are you involved in developing or implementing a data warehouse? If so, there are some things you should probably know.

When asked about data warehouses, AIR members report that they’re very prevalent; of the one-third of campus and system IR respondents who say their institution doesn’t use one currently, 60% say that there are plans to implement one.

What is surprising is that concerns voiced by that group are markedly different from the experiences of those who have implemented data warehouses. Of note, none of those planning a data warehouse foresee any struggles regarding the costs of funding and potential staffing shortfalls.

However, those who have been involved in data warehouse implementation have much to say regarding those particular pitfalls. Nearly half of respondents whose IR office is responsible for a querying data for reports say that the size of their IR staff needed to be increased to compensate for the larger workload, and an equal number of those whose IR office is responsible for managing data warehouse hardware and software say their budget increased to offset that expense.

Here is some of their advice:

Sufficient Resources:

  • Identify a knowledgeable and objective Executive Sponsor who will lead the planning effort, and who will recommend a budget that is aligned with the long-term goals for the project.
  • Budget appropriately or it will not happen or will be a very slow process. Dedicate staff to nothing but this project for one year while it is being built.

Project Support:

  • Getting all key staff “on board” is crucial because implementation is a long process.
  • Everyone needs to commit to it 100%. The college must commit to funding it in the future. IT must commit to designing and maintaining it. Users must commit to using it.

IR Involvement:

  • Get involved. Implement standard operating policies that guide data definitions, when to use census vs. nightly refreshed data, who has access, and who will train other offices that might want access.
Cl​ear Definitions:
  • Important considerations for implementing a data warehouse include development of data definitions and documentation for consistency.

Bill Knight, Assistant Provost of Institutional Effectiveness at Ball State University (and author of Leadership and Management in Institutional Research: Enhancing Personal and Professional Effectiveness), offers this advice:

Talk with other institutions and have a realistic plan for time, cost, and communication. We have been in the implementation phase for nearly two years with little tangible results, primarily (as I see it) due to lack of agreement of who has responsibility for what. Data warehouses are not “plug and play”; they require substantial support from IT, IR, and others across campus.

Have you helped implement a data warehouse at your institution? What advice would you offer to those who are looking to do so in the near future? Share your thoughts and comments below.



To add a comment, Sign In
Total Comments: 2
Michael posted on 5/15/2015 4:41 PM
I agree with Bill. Data warehouses are not plug and play. Indeed, if one is purchasing a warehouse solution from a vendor, best to think of that as only the first step in the development of a warehouse (strategy is often called "buy, then build"). While purchasing the frame can help you get started, very likely it will need to be modified significantly over time. And that will take both time and cooperation from many offices on campus (at first,especially with IT and Enrollment Management).

It is important that IR should not be a consumer of a warehouse, it needs to play a major part in its development. When you get right down to it, many IR offices already have data warehouses, in that they take information from institutional transactional systems and transform them into tables (sometimes relational) that make them more conducive for reporting. And they have procedures to clean the data (data quality). What is missing most times is the nightly feeds required for many operational reporting, and a delivery system for operational reports supporting to large number of institutional constituents.

In the end it is not about technology but about people and institutional culture. I like to define our data warehousing efforts as creating a campus wide reporting environment. As such it is not a project with a defined end point, but an evolutionary process that transforms how the campus makes decision by providing better contextual information about itself and the environment in which it resides.

Michael Dillon PhD
Associate Vice Provost
Institutional Research, Analysis and Decision Support (IRADS)
University of Maryland, Baltimore County (UMBC)
Bob posted on 12/10/2015 1:41 PM
My advice is not to 'start' by building a datawarehouse. The issues raised about ownership, definitions, expectations, transparancy, etc are real. If you attack a full data warehouse head on these issues will dig in.
Instead don't wait for a data warehouse to get results - make your own, small and manageable, data warehouse in Excel with a simple extrat. Use it to do some small trend analysis, or drill-down rankings, show some others the value without forcing them to buy in.
The best thing to do is to do some useful analysis, and data governance along the way, using tools you already have. When you're ready to up the ante and aggregate into a full data warehouse, it will be an easy transition.