for Management Research, Policy Analysis, and Planning
The Functions of
Joe L. Saupe
Association for Institutional Research
About the Author
Joe L. Saupe is a professor in the Department of Higher and Adult Education and Foundations in the College of Education at the University of Missouri-Columbia and serves as director of the Office of Research and Development of the College.
Before assuming his current position, he served as university director of institutional research for the University of Missouri system (1968 to 1988) and held positions at Michigan State University (1956 to 1968), his last position there being that of associate director of institutional research. He received his graduate degrees from the University of Illinois.
The author is a charter member and past president of the Association for Institutional Research (AIR). He has been an active participant in committee work, has chaired and consistently participated in the annual AIR Forum, and has provided leadership in professional development activities. He authored the first edition of the present publication and two earlier monographs intended to assist individuals and institutions new to institutional research.
The Association for Institutional Research
314 Stone Building, Florida State University
Tallahassee, Florida 32306-3038 (904)644-4470
Copyright© 1990 by the Association for Institutional Research
for Management Research, Policy Analysis, and Planning
The Functions of Institutional Research 2nd Edition
Joe L. Saupe
Association for Institutional Research
This monograph is a revision of a document published in 1981. While the basic nature and functions of institutional research have not changed during the intervening years,developments in the dynamic domain of institutional research have led to this revision.
This publication and its predecessor were sponsored by the Publications Board of the Association for Institutional Research. Suggestions for the revision were provided by members of the Publications Board, by members of the Executive Committee and by other members of the Association; these contributions are gratefully acknowledged. A draft of the revision was reviewed and very helpful suggestions were provided by Trudy Bers, Peter Ewell, Mary Kinnick and Patrick Terenzini on behalf of the Publications Board.
The purposes of the Association for Institutional Research, which has individual and organizational members from all types of postsecondary institutions, from system, state and federal agencies and from nearly thirty countries, are to benefit, assist and advance research leading to improved understanding, planning and operation of institutions of postsecondary education. The Association offers an annual forum on institutional research, publications and other services to members and the larger postsecondary education community. In addition to the international association there are a number of affiliated state and regional groups which also hold annual meetings and provide other member services. Additional information about the Association may be obtained from its executive office, the address for which is listed on the inside front cover.
The purpose of this monograph is to provide in a general way a description of the nature of institutional research, of its role in institutional governance and of the contributions it can make to the functioning of postsecondary institutions. It is intended to be useful to persons beginning work in institutional research, to executive officers and others who are considering the role of the activity in their institutions and to students of postsecondary education.
The first section of the monograph is a discussion of the nature and purposes of institutional research. Several forms of the activity are described and institutional research is compared and contrasted with other types of research. The second section describes the various ways in which the function of institutional research is included in the organizational structures of colleges and universities. Whether or not there are one or more organizational units specifically responsible for institutional research, the activity occurs in many or all units of the institution.
Offices of institutional research typically are assigned responsibilities other than research; these responsibilities are discussed in the next section. Characteristics of sound institutional research and basic tools of the activity are then described. Comments on communication of the findings of institutional research and descriptions of several special forms and methodologies of the activity are included. The final section is an outline of the potential of institutional research for contributing to planning, decision making and policy formulation in a wide variety of areas of institutional governance.
Joe L. Saupe
University of Missouri-Columbia
The Nature and Purpose of Institutional Research
Institutional research is research conducted within an institution of higher education to provide information which supports institutional planning, policy formation and decision making. The institution may be a single campus, a multi-campus system, a state or provincial system or an even larger grouping of colleges and universities. Although the activity of institutional research is commonly associated with the individual campus, it also is carried out within higher education systems to serve the governance responsibilities which reside there.
Institutional research can be distinguished from research on postsecondary education which has as its purpose the advancement of knowledge about and practice in postsecondary education generally. The subject of institutional research is the individual college, university, or system. While institutional research can involve data and analyses which contribute to wider knowledge about how colleges and individuals function, this type of result generally is not sought for its own sake.
Activities of institutional research are frequently undertaken in association with specific planning, policy, or decision situations. Information to answer specific questions is desired. How many sections of a specific course should be offered? By what amount should tuition rates be increased to produce a target amount of tuition income? Is attrition a problem at our institution? Are our faculty salaries competitive with those paid by peer institutions? Are the outcomes of our degree programs what the stated purposes of the programs suggest they should be? Institutional research designed to answer such questions is a form of applied research.
The assembling of the quantitative and qualitative information for use in periodic or ad hoc reviews of programs or organizational units illustrates the form of institutional research having characteristics of evaluation. Information on cost and productivity underlies judgements about efficiency. Information on other characteristics of programs and units and on outcomes leads to judgements about effectiveness or quality. Information on program purposes, on programs offered by other institutions, on the labor market and on potential demand produce judgements about the need for academic programs. Judgements of these types lead to decisions about program initiation, continuation, and improvement.
Occasionally institutional research leads to general information about the college or university and its environment and provides a comprehensive view of the institution which may inform planning, policy formulation, and decision making of a variety of types. Information of this type may arise as a by-product of institutional research on some specific question. For example, a study of student flow undertaken to guide enrollment projections may reveal that large numbers of juniors and seniors entered the institution as transfer students. This incidental finding may have a variety of implications. Similarly, institutional research may be undertaken on a general topic and not be guided by a specific problem or question. The expectation is that the findings will be generally informative. For example, surveys of alumni and of members of the community in which the institution is located may reveal attitudes and impressions about the institution. These forms of institutional research have characteristics of basic research.
Some activities of institutional research may, intentionally or incidentally, identify situations within the institution which are causes for concern. A by-product of the routine tabulation of enrollment data by program or a special analysis of enrollment data designed to isolate patterns of retention and attrition may reveal that attrition appears to be a special problem for selected programs. Some form of administrative or academic attention, perhaps accompanied by additional study, may be suggested by such results. Thus, problem identification may be a result of institutional research.
Some projects of institutional research may be called action research, because the researcher and client work closely throughout the problem definition, research design, data collection, analysis, interpretation and implementation phases of the activity. The institutional researcher who is asked to serve on or as a resource to a task force charged with studying some matter and with making recommendations thereon can work closely with the task force and may then be able to follow up after the task force has reported to insure the recommendations of the task force are correctly understood and acted upon. Such an arrangement provides an excellent opportunity for institutional research to have impact.
Policy analysis is another phrase which applies to some forms of institutional research. Deliberations on matters of policy are often, or often should be, accompanied by analysis and that analysis is institutional research. Deliberations on admissions policies may require analysis of the impacts of policy changes on different segments of the potential student population as well as on the size and composition of the student body.
Clearly, the term research, as used here, has a broad meaning. Information about the college or university results from analyses of quantitative data and qualitative assessments. Tabular displays of counts of fall-term students and of annual expenditures illustrate the simplest form of analysis. Comparisons of current with prior-year totals of such data provide a similar, still simple, form of analysis which conveys information. Cost analyses, space utilization analyses and teaching-load analyses are conventional types of institutional research. Statistical techniques ranging from the calculations of averages and percentages to the applications of complex multivariate procedures are included. A variety of prior-year, current and projected institutional data, along with quantitative representations of policy variables and assumptions, may be used to simulate institutional functioning in future years using techniques of mathematical modeling. Simulation is a form of analysis by which implications of alternative courses of action are assessed. The measurement of student outcomes and the examination of the measures in the context of student and institutional goals and quantitative and qualitative characteristics of academic programs is another form of analysis. The performance of administrative and support programs may be similarly analyzed.
Institutional research, like other types of research, should be objective, systematic and thorough. The outcomes of the research should be as free as possible from the influence of personal philosophy, political considerations or desired results. The information provided by institutional research is combined with academic and professional judgement in planning and other decision-making processes. Almost never is the final decision based solely on the findings of the research, nor should it be. Considerations of institutional
philosophy and tradition, of priorities and of the environment in which the institution exists may be as important as the findings of the research in determining the course of action to be followed. For example, a study may suggest a technique of recruiting students which has considerable potential for increasing enrollment. The factors of judgement may lead to a rejection of the proposal that the techniques be used or to a major modification in it. Nonetheless, the research has served its purpose by bringing information to the decision-making arena and stimulating reflection about recruitment and the various factors involved in this institutional activity. The research might not have served this purpose if it had been of questionable validity or had been guided by some preconceived notion of what result was desired or expected.
This is not to say that institutional research should be undertaken or carried out in ignorance of the nature of the institution and the forces which guide its operation. An investigation of forms of "hard-sell" recruiting, for example, would not be useful in colleges and universities where this type of activity would be inconsistent with strongly held institutional values. The design and the presentation and interpretation of the findings of institutional research can be guided by the nature of the institution and its environment and the usefulness of the results thereby enhanced.
Institutional research, then, is an essential ingredient of sound college or university governance. It should occur throughout the institution wherever any sort of planning occurs, any type of policy issue is considered and any decision about some aspect of the institution is proposed. Institutional research has been described as an attitude of commitment to the institution's purpose in society and to the value of critical appraisal and careful investigation. Institutional governance is informed and rational to the degree that such an attitude pervades the institution.
The Place of Institutional Research in the Organization
Institutional Research is carried out in the individual academic and administrative units of the college or university in support of the information needs, planning and decision-making responsibilities which reside at the unit level. At some colleges and universities, a conscious decision had been made that each unit in central administration will be responsible for institutional research relating to the activities of that unit. There may be no formally identified offices for institutional research in these institutions, but this does not mean that the activity is absent. Rather, it is dispersed and carried out by persons who may have principal responsibilities other than institutional research. Economy and in-depth knowledge of the matters studied may be points in favor of such arrangements. Problems of dispersed institutional research are duplication, a lack of research expertise in some or many institutional domains, an inability to deal with issues which cross organizational boundaries, and an absence of an institution-wide view in the research activities.
At other colleges and universities, offices of institutional resarch have been established in recognition of the fact that the activity requires specialized expertise and full-time attention. In some cases, the title Office of Institutional Research is given to an organizational unit which supports functions such as planning and
budgeting wherever these occur within the institution. Elsewhere, the connection of institutional research with the activities of planning or budgeting has resulted in organizational units titled Office of Institutional Research and Planning, Office of Institutional Research and Budgetintg or Office of Institutional Research, Planning and Budgeting. Names such as Office of Institutional Analysis and Office of Institutional Studies also are used.
Various administrative units may be charged with some responsibility for institutional research. The placement of the unit within the administrative organization determines the nature of the unit's responsibilities or type of institutional research it undertakes. In some colleges and universities, institutional research reports to the chief executive officer. The specific charges to such offices vary widely, but this organizational arrangement recognizes the function as one of central importance and so broad that to be effective it must be placed near the top. Placement at the top may indicate that institutional research on academic, administrative, financial and auxiliary matters are all important and cannot effectively be carried out in isolation from one another. This arrangement also recognizes that institutional research supports planning and resource allocation which crosses organizational boundaries.
Another model is that of an office of institutional research and planning which is responsible to a vice president for planning. Such an organization recognizes that information developed from research underlies institutional planning. Although the name of the activity may not include the designation "institutional research" because the principal responsibility is planning, the information development phase of the activity is institutional research nonetheless.
The office of institutional research may be a responsibility of the institution's chief academic officer in which case research and information on academic purposes, programs, program outcomes, policy and personnel will support academic planning, budgeting for academic units and other responsibilities of that officer. Such a unit may be responsible for the institution's program evaluation and assessment activities. In this regard it may conduct surveys of graduates and former students. It may be responsible for needs assessment studies designed to guide the development of new programs. The unit also will respond to requests for assistance from deans, chairpersons, and faculty committees.
In many colleges and universities a unit charged with leading efforts to improve instruction and academic programs has been established. Institutional research supports the activities of such a unit. Research on teaching methods and instructional media may be carried out there. Programs of student ratings of instruction are often housed in such units and are based upon research. Questions about testing and grading lead to research on these topics. Courses and curricula, and the interaction of students therewith, are analyzed and evaluated. Data underlying periodic reviews of programs and academic units are assembled. Evaluations of special services (for example, learning centers created to serve marginal or disadvantaged students) are carried out.
Responsibilities for program assessment may be assigned to such a unit by virtue of its
experience and expertise with regard to instructional programs and testing.
Interests in institutional and program assessment for purposes of accountability and program involvement, often stimulated by authorities external to the institution, have in some institutions led to the designation of a person or office responsible for leadership and staff support for assessment activities. Assessment of institutional and program effectiveness is a form of institutional research.
An office of student research may exist within the institution's student affairs organization and be responsible for research on campus climates and subcultures, on the characteristics of various services provided students, on residence hall life, and on factors involved in retention and attrition. The student affairs' research office may investigate the development of student attitudes and values. Market research, designed to enhance the congruence between the offerings of the college or university and the needs and expectations of its clientele, may be carried out by this office or by units for institutional research located elsewhere.
An office of financial analysis, analytical studies or administrative research may be located within the organization of the institution's chief officer for business and finance. Studies of business operations and budget and cost analysis are likely to be principal responsibilities of such a unit, and financial planning may be based largely upon the analysis of this institutional research activity.
It is important to recognize that data descriptive of the operation of the college or university are by-products of various institutional operating processes. Data about students are acquired as a result of admissions, registration and associated processes. Budget building and financial transactions result in budget and accounting data. Personnel transactions generate data about employees. The translation of these various items of data into information useful to planners and decision makers requires analysis and, indeeed, is a major activity of many offices of institutional research. The products of this translation and analysis have been referred to as management information.
In colleges and universities of all sizes, computers are used to facilitate the several operating processes and to manage the data involved in these processes. What were at one time called administrative data processing systems are now called management information systems. The applicability of the current designation depends upon the degree to which the computer system, in fact, not only processes the data for operational purposes but also includes capabilities for translating the data into information. These capabilities cannot exist without careful planning and management of the data. Data administration is the function that attempts to insure that the data captured from the operational data systems are meaningful. The knowledge and perspectives of persons responsible for institutional research can make important contributions to the function of data administration.
The relationship between administrative data processing systems and management information and a recognition of the importance of data administration has resulted at some colleges and universities in a close association between the administrative computing organization and the office of institutional research.
In these organizational arrangements, the activity of institutional research includes participation in the preparation of specifications for information systems as well as analysis and dissemination of the information handled by the systems.
Finally, in considering the various ways in which the college organization may incorporate institutional research, the fact that the activity often occurs within the standard organizational units bears emphasis. Information about students may be prepared in the office of admissions where research on admissions criteria may also be conducted. The office of registration and records may prepare reports and analyses of enrollment statistics and trends and may be responsible for enrollment projections. The accounting office prepares the annual report of income and expenditures which is important information for planning and decision making and may undertake analyses of trends and projections of financial variables. The physical plant or the space assignment office may be responsible for maintaining information on buildings and rooms, for analyses of the utilization thereof, for data and studies on facilities maintenance, for research on budding and room accessibility for handicapped persons, for studies related to energy conservation efforts and for other activities which support campus and facility planning. Personnel offices study employee classification systems, salary scales and compensation policies. Any component of the college or university may have a responsibility for institutional research.
The dispersal of institutional research activities which has been multiplied by the widespread use of personal computers and personal computer networks and access to central computer data bases has introduced problems of data comparability. The data analysis of the dean of arts and science may conflict with that of the vice president for academic affairs simply because of differences in definitions and sources of basic data. Some offices of institutional research work with data administrators to develop official definitions and sources of data in order that decision makers can focus attention on the issues and the relevant management information rather than upon conflicting data analyses.
A benefit of an office of institutional research, based upon its in-depth comprehension of institutional data systems, should be its capacity to combine, analyze and interpret data resulting from the several operational activities of the college or university. The office need not be responsible for all varieties of institutional research, but it can serve as a reliable source for comprehensive and authoritative information about the institution.
Associated Responsibilities of an Institutional Research Office
By virtue of its responsibilities for data and information about the college or university, an office of institutional research typically will be assigned responsibilities which need not be considered research on the institution. The following are illustrative.
The office of institutional research is likely to have some resonsibility for the institution's responses to national statistical surveys such as the Integrated Postsecondary Education Data System (IPEDS) of the National Center for Education Statistics in the United States or the surveys for Statistics Canada
publications in that country. Similarly, the data forms which must be completed for the state or provincial agency with responsibility for higher education may be assigned the institutional research office. The nature of such responsibilities varies. At one extreme, the office simply serves as the point of coordination, receiving the packets of forms, distributing them to other offices where they are completed, then collecting and returning them. At the other extreme, many or all of the data forms may be completed in the office of institutional research itself, drawing upon whatever data files are necessary. Even in the former case, staff in the office are likely to be--and because of their expertise, should be--called upon for assistance in interpreting the standard definitions and instructions of such surveys. They may also be asked to assist in determining how the data in the institution's files should be processed in order to produce the required figures, and in general to insure that the institutional data provided are consistent and accurate.
There are two points to be made about reporting data to state, provincial and federal agencies. First, while there are few immediate rewards for filling out forms, the responsibility should be taken seriously. The data are collected for what may be presumed to be good reasons and should be as sound as the data assembled to serve purposes internal to the institution. The completed forms are products of the institution and, as with any institutional product, should be of high quality. Further, the accuracy of the data may affect perceptions of the credibility of the institution and thereby influence policy decisions important to it.
Secondly, the data reported to the agencies are descriptive of the college or university and the potential value to the institution of the data should not be overlooked. A caution in this regard, however, is that the data are reported in standard categories which may detract from their utility to the institution. Counts of degrees granted for standard subject field categories are less informative to the institution than are counts arrayed by specific institutional academic units and programs. Current fund expenditure amounts displayed in standard expenditure categories are less meaningful internally than are data displayed by the cost centers of the institution. Thus, in capitalizing on the availability of data produced for the external forms, attention needs to be given to displaying the data in categories that are meaningful to the institution.
A related responsibility often assigned to the office of institutional research is that of responding to questionnaires and other non-routine requests for data or information. Almost daily, a college or university receives some request for information from an agency of government, from the publisher of a higher education directory, from a doctoral student or from some other source. Some selectivity with regard to which inquiries merit responses must be exercised and, if it is decided that a response will be given, the response should be prepared carefully for the reasons previously cited. By virtue of its general responsibility for and understanding of data or information on the institution and because of its commitment to consistent data of high quality, it is natural that many or all such inquiries or questionnaires be referred to the office of institutional research. It is at least embarrassing when reports of apparently conflicting information about the college or university, made by different representatives of the institution, are found.
The dispersal of activities of institutional research within the college or university has led to a need for sharing information about the activity. The office of institutional research, by virtue of its responsibilities for and expertise on institutional data and research, may provide the leadership in orienting others to the nature and sources of institutional data and their use. One purpose of the orientation is to encourage consistent use of the official institutional data in order to avoid conflicting data analyses. The other purpose is simply to provide education on the techniques of institutional research and, as appropriate, on research methodology generally.
The data and information managed by the office of institutional research may be used elsewhere in the college or university for purposes other than planning, policy formulation and decision making. For example, the office of public information may seek data to include in press releases or publications about the institution. Similarly, general or specific data may be required to support proposals for external funding for research or other purposes, and the office of institutional research may be asked to serve as the principal source of such data and information.
The office of institutional research often provides the continuing point of contact for the state or provincial agency for higher education on matters relating to institutional data. Institutional research staff may be asked to serve on agency committees where a central concern is institutional data. Persons from institutions provide advice on the development and refinement of state- or provincial-level information systems, on funding formula and the data requirements for them, on studies of special issues in higher education, as well as on the information required for state- or provincial-level strategic or long-range planning. It is appropriate that the expertise of the person assigned to institutional research be drawn upon in such endeavors.
Quite often, by virtue of background and interest, the institutional researcher will keep abreast of the journals and books on postsecondary education and, particularly, the literature on research on postsecondary education. If this person has an academic background or is so inclined for some other reason, he or she may, on occasion, contribute to this literature. While the purposes of institutional research and research on higher education differ, the two forms of research contribute one to the other. The problems, methodology, and results of the general research can be applied and particularized in institutional research, and the findings of institutional research may merit generalization through broader studies. The interest of the institutional researcher in the liberature on higher education often leads to the development of a library of publications maintained by the office of institutional research but available to others, particularly administrators.
Finally, the person responsible for institutional research may be called upon to provide advice on planning, policy and other issues facing the college or university. In one sense, this function is a natural consequence of the institutional research activity. The products of research need to be interpreted and their implications explained. The consequences of alternative courses of action, based upon the research, need to be described and qualified. The person who has done the research should be well qualified to describe its results and implications and to answer questions about it. When the director of institutional research participates in planning, policy formulation and decision-making deliberations at the stage in which considerations other than those raised by the research are brought to bear, it is important to
recognize that participation is based upon the director's status as an expert on the institution and higher education rather than upon the director's responsibility as a researcher. The distinction suggested here may be difficult to identify in specific situations. The point is that the values and perspectives of the researcher are not identical to those of the decision maker. There is considerable merit in distinguishing the two roles. Just as the results of the research will seldom be the sole determinant of the decision, so the desired decision cannot be allowed to bias the outcomes of the research.
Characteristics of Effective Institutional Research
It is not within the scope of this monograph to describe or review all the methods and tools of research which are employed in institutional research. Numerous books describe applicable topics, such as descriptive statistics, sampling and statistical inference, questionnaire construction and survey research, experimental and quasi-experimental design, principles of operations research, procedures for program evaluation and methods of qualitative research. Not only is it beyond the range of competencies of any single individual or, even, office of institutional research, to possess expertise in the full array of these methodologies, such expertise is often unnecessary since at many colleges and universities there are qualified researchers on the faculty who can be called upon to advise on or to carry out selected projects.
Some of the fundamental requirements of good research merit mention as they apply to institutional research. The first is that of purpose. Each activity or project of institutional research should be guided by a purpose or set of purposes stated as specifically as possible. Normally the resources available for institutional research are inadequate to justify undertaking projects because "it would be interesting to know ...." or "it might be useful to know ...." (To some degree, this is regrettable because what might be useful to know could turn out to be, in fact, very useful to know.) In view of the applied nature of most institutional research, the guidance given to the research effort by a purpose can be enhanced by including consideration of what actions or decisions might be made on the basis of the results of the research. For example, the initial purpose of a project might be to determine attrition rates for various types of students. This can be done, but when the question becomes "What actions might be taken as a result of knowing the rates?" the nature of the project may change. The fundamental question turns out to be "What are the characteristics of the college or university which lead to attrition and what might be done to change these characteristics?" Similarly, if the question is "What is the faculty-salary cost per student credit hour for the undergraduate courses of each department?" an analysis may provide the answer. However, the data collecetd and their analysis will differ depending upon whether the resulting unit costs will be used to project faculty-salary costs on the basis of projected student-credit-hour data or will be drawn upon in making budget decisions. If it is the later, a more complete analysis may be needed. The linkage between institutional research and planning, policy formulation or decision making is provided by the purposes given to the former by the requirements of the latter.
Assumptions are involved in institutional research. If, in the previous example, a unit cost is multiplied by a
projected count of student credit hours to produce a future year cost estimate, an assumption is that the unit cost is not a function of department size (or of any other variable which may change between the current and projected year). There are assumptions about the meaning and accuracy--validity and reliability--of the basic data, and the researcher needs to recognize the assumptions and to point them out to the consumers of the research. It is often useful to review the assumptions with the consumer before the project is begun. The interpretation and implications of the results of a project are typically influenced by the assumptions involved. An understanding of, and selection from among, alternative assumptions before the project is begun can help insure that the eventual results are interpreted properly and that implications are derived appropriately.
Communication of the findings of institutional research takes a variety of forms. Oral reports, in person or on the phone, are made to answer generally straightforward questions. A table or two or a display of data in a chart or graph may constitute a sufficient report. Letters or memoranda are used as brief reports which do not merit or require general distribution. Written reports, including tables and, perhaps, graphs and charts, are produced for many projects. The audiences for reports of institutional research are typically administrators who often are served best by a concise report--an executive summary--which emphasizes the results and, perhaps, implications of the project. However, because faculty members are often consumers of institutional research, and because administrators have faculty backgrounds and perspectives, it is well to have a complete report in the file for use in answering questions of detail which may be raised. The more complete report includes relationships to previous research and a description of the methodology in sufficient detail that the study could be replicated. The more complete technical report is written as though intended for publication to serve an audience of other researchers.
Personal computers, local area networks, electronic mail, desk-top publishing, and advanced graphics capabilities have added options for the communication of results of institutional research. For example, a table of data or, even, a complete "fact book" may be stored in a computer data base to be retrieved by anyone with need for the data. Complete reports may be treated similarly. Data and reports stored electronically or on microfiche can save much space in file cabinets and be readily available. Questions about results may be asked and answered, and dialogues on the topic of the research may be carried out, by electronic mail. The existence of sophisticated graphics capabilities has drawn attention to this form of communication.
A consideration of the content of an institutional research report may illuminate the nature of the activity and the manner in which it aids planning, policy formulation and decision making. As already noted, institutional research consists of analysis which results in information. Data analysis yields results, generally quantitative, which constitute what may be considered an initial level of information. Results are, of course, included in the report. A next level of information may be added by means of an analyses of the results in the context of the purpose of the research, the assumptions used, and other factors. This analysis constitutes the interpretation of the research, may produce generalizations and adds meaning to the results. The analysis of the interpretations in the context of the institutional environment may produce implications. A final stage of analysis may result in recommendations. The
extent to which this sequence of analysis applies to individual projects, of course, varies. The point is that the sequence represents a movement from analyses which are clearly institutional research toward analyses which are characteristic of planning, policy formulation and decision making. At each step, additional considerations are brought to bear. As the new considerations begin to depart from those of fact and evidence, the analysis loses characteristics of research. Certainly the researcher should illuminate the subject of the research to whatever degree the evidence and the researcher's experience and expertise permit. At the same time, the distinction between the roles of the researcher and the decision maker needs to be maintained.
While written reports are important products of institutional research, it is often desirable to attempt to ensure that research undertaken is relevant and research completed is useful. Some institutional research offices draw upon advisory committees or upon networks of users of research to ensure that the projects undertaken are relevant to the issues faced by the college or university. After a report is written and distributed, the contribution it makes often can be increased by some follow-up techniques designed to ensure that the research results are understood correctly and interpreted appropriately.
Basic Tools of Institutional Research
There are several techniques or tools of institutional research which deserve special attention because, even though they are not unique to institutional research, they are used frequently.
First, the conversion of data in the institution's operational data processing systems into management information frequently is a responsibility of the office of institutional research. The activity undertaken to fulfill this responsibility is not guided by a specific research purpose in the sense of a plan to be prepared, a policy to be established or a decision to be made. Rather, the purpose is to develop a set of data which provides a meaningful picture of the institution and its operation and which is comprehensive and flexible enough to provide answers to unanticipated questions. One product may be a set of summary reports which is prepared regularly and consistently and which, consequently, reveals trends in key institutional characteristics. Another product is a data base from which ad hoc analyses may be produced on demand with relative ease.
Computer technology has enabled the development of such management information systems. This technology allows not only the accumulation of consistent subtotals (by department) and totals (for the institution) of various types of data describing students, courses, employees, finances, and facilities, but by combining the various types of data, it also allows reports of, for example, student credit hours per full-time-equivalent faculty member, expenditures per student, and square feet of space per student contact hour. By taking advantage of the capability of the computer to store, retrieve, and manipulate data, a variety of types of information descriptive of the institution and its functioning can be produced. Offices of institutional research are involved, sometimes centrally, in the development and operation of computer information systems, because these offices are expected to possess expertise in all types of institutional data and in their uses for planning, policy formulation and decision making.
It might appear that the task of converting data from the institution's operating data systems into management information requires only computer programming. Usually this is not the case. Inconsistencies between data systems sometimes seem to be the rule and not the exception. It is difficult to prepare a meaningful report of student credit hours per full-time-equivalent faculty by department when the student information system and the personnel or budget systems make use of differing sets of departments. The function of data administration, mentioned previously, exists because of the recognition of such difficulties. This function requires in-depth knowledge not only of the several operating data systems, but also of the nature of the information needed for planning, policy formulation, and decision making.
The decision support system (DSS) constitutes another method of carrying out institutional research. Fundamentally, a DSS is a data base and computer software for using the data base which are made available to the decision maker as an aid in making decisions. The central feature of the DSS methodology is that it enables the decision maker to do the analysis, rather than making use of analyses carried out by someone else. The decision maker decides what analysis to carry out and understands the implications and limitations of the information produced by the analysis. A decision support system is developed by a technical person, often a member of the institutional research staff, working closely with the decision maker. The development of a decision support system typically is a continuous process; as the system is used, features of it are refined and new features are added. The relationship between the institutional researcher and the decision maker is a close and continuing one.
A relative of the decision support system is the executive support system (ESS), also referred to as the executive information system (EIS). (Some authorities distinguish between executive support and executive information systems.) An ESS can be defined as the use of computer technology to serve (a portion of) the information needs of an executive officer, whose responsibilities are more general than those of the decision maker served by a DSS. An ESS may include the capabilities of a DSS, but can be expected to have such additional features as electronic mail, electronic conferencing, and powerful graphics. The executive may use the system principally to acquire information about the college or university, rather than as an aid in making specific decisions. Advanced computer technology which makes the system very easy to use is often involved. The ESS is developed in much the same manner as the DSS; the technician with institutional research expertise or support works with the executive in a mode which accommodates the limited amount of time the executive has available for the activity.
Modeling is also employed in institutional analysis. It involves the specification of mathematical relationships among variables of institutional operation. Parameters of the mathematical relationships are derived from historical data and are used to project variables of the model for future years. By varying values of model parameters, which reflect assumptions about future relationships, answers to "what-if" questions may be derived. Enrollment projections are developed using enrollment and student-flow models. Faculty-flow models are used to project numbers of faculty in various categories, particularly numbers on tenure, and elaborate cost and budget projection models are used. It is not unusual to find that the most difficult step in
applying the simulation-model techniques is the development of the required (consistently defined and developed) historical or base-line data. Models often are included in the software components of decision support systems and executive support systems.
Another technique of institutional research is comparative analysis. Quantitative descriptions are given meaning by comparison. Several bases of comparison may be used. One basis is the comparable prior year value or the trend for a series of prior years. For example, meaning is added to an average class size of 36 for the history department by comparing this average with the averages for past years. A second basis is comparison with similar units. For example, how does the average class size for the history department compare with the averages for the sociology and economics departments.
The basis of comparison may be some predetermined standard. Using such considerations as student demand for history courses, the educationally desirable size of classes for different courses, and teaching-load policy, it may have been established that the average class size of the introductory courses should be 50 and of advance courses 20, for an overall average size of 32. The actual size of 36, then, will exceed the standard.
Other institutions provide yet another basis of comparison. How does the figure 36 compare with the average class sizes (calculated the same way!) for the history departments of other (comparable) colleges or universities?
Data exchange is a practice of institutional research which provides the data required to make comparison with other colleges or universities. The normal procedure is for a group of peer institutions to agree upon sets of data to be exchanged and the schedule for the exchange activities. Several approaches are used. At one extreme, fully analyzed data (e.g., unit cost or student-credit-hour-per-faculty-appointment ratios) are exchanged. At the other extreme, >more basic data are exchanged in formats which permit each participating institution to conduct analyses in a manner tailored to local conventions, analytical procedures, and needs. Data exchanges which lead to peer institutional comparisons are frequently undertaken by public colleges and universities in order to generate information for use in supporting requests for governmental appropriations. A second purpose is to provide information to enlighten institutional planning, policy formulation and decision making.
Colleges and universities exchange information on policies and procedures as well as data. Deliberations on a policy issue often lead to the question "What is the policy at our peer institutions?" The data exchange mechanism can be used to answer this question. Data exchanges often take place electronically. Data are exchanged on floppy disks or over telephone lines. Comparisons of policies and procedures are facilitated by electronic mail.
A concern with data definition underlies almost all varieties of institutional research because the information resulting from analyzed data of any type can be only as meaningful as the definitions underlying the original data and the degree to which the definitions are observed in assembling the data. The definitions must lead to data that are meaningful to those who use the data and that are relevant to planning, policy and other decision issues of the college or university. The involvement of persons responsible for institutional research in data administration occurs, at least in part, because of the importance of the definitions underlying the
information. The importance of the involvement of the ultimate user, the decision maker, in determining the definitions is recognized by the processes used in developing a decision support system. Agreement on, and use of, data definitions is a central concern in data exchange efforts and in all other forms of comparative analysis. The term "comparison" implies careful attention to data compatibility and comparability which are assured only by sound and complete definitions.
Contributions of Institutional Research to Planning, Decision Making, and Policy Formulation
The range and variety of problems, questions and issues which arise in higher education and for which institutional research is relevant defy categorization or enumeration. Any administrator and any committee may seek institutional esearch to inform plans, decisions and actions. In many cases the researcher can aid in specifying the information to be brought to bear on a problem or issue and, for this reason, should be included at an early stage. The breadth of the potential applicability of institutional research is indicated by the following illustrations.
Institutional research can aid in determining how the institution's several publics perceive its missions and goals and in specifying new or altered missions, goals and objectives. It can assist in relating performance to goals by assessing institutional outcomes and accomplishments, can point to areas in which performance does not appear to meet expectations and can suggest strategies for improvement. Institutional research can facilitate institutional self-study and accreditation processes and can contribute evidence that the college or university is accountable for its use of resources and performance.
Institutional research can contribute to program planning and development by means of market research and needs assessment. It can support intensive reviews of programs or departments by providing relevant factual evidence and by summarizing qualitative information. It can illuminate reviews and revisions of curricula by producing information on students' course-selection behavior. Institutional research can provide information relevant to questions about the grade-giving behavior of faculty and the grade-earning behavior of students; such questions may arise from concerns about standards or about equity with students.
Institutional research can study the culture of the college or university, investigating the extent to which various values and norms are present among the faculty, students and administrators and the extent to which the culture is shared or in conflict. Information from such investigations can inform the direction of planning or policy and can provide an understanding of potential obstacles to moving in new directions.
Institutional research underlies the improvement of instruction. Procedures and specific instruments used in the evaluation of instruction, such as student rating-of-instruction forms, are selected or developed by means of research. The evaluation of instructional methods and media is a process designed to ead to improvement and is guided by evidence from research.
Institutional research can assist in identifying inefficiencies in instructional activities and in the allocation of resources. Data on class sizes, teaching loads and student-credit-hour productivity and data
on the incidence of small classes and on the frequency of offering of individual courses are made available to academic administrators.
The admissions program can benefit from institutional research. Criteria for admissions can derive from relationships between measures of student ability and success in programs. Data on sources of students and the "yields" of alternative strategies of admissions officers and others can assist in tailoring the admissions program to the mission and goals of the college or university and of specific programs.
Institutional research not only can provide enrollment projections but also can provide analyses of enrollment trends and relationships which guide enrollment policy and suggest assumptions and strategies for enrollment planning. Data describing the student body can be related to enrollment goals. Data on retention and attrition can reveal problems. Institutional research on causes of attrition and on strategies for increasing retention can contribute to maximizing society's investment in education.
Institutional research can support efforts to provide education to special types of students by assessing their preferences, predispositions and academic behavior. In what regards do part-time students, minority-group students, women students, highly talented students, handicapped students, older students and others differ from the traditional student in ways which have implications for the achievement of the educational goals of such students and of the college or university? Students' program, course and scheduling behavior can be summarized, and attempts to achieve student and institutional goals can be evaluated.
Institutional research can assist with initiatives intended to foster access to the educational opportunities offered by the college or university and can contribute to attempts to ensure that the applicant's choice of the institution is an informed one. Institutional research can assist in developing the consumer information which should be available to prospective students. The financial affairs of students can be determined and used as consumer information as well as referents for the determination of financial aid programs and policies. The effectiveness of the program of financial aid in achieving the goals set for this program can be evaluated and the evaluation may lead to improved use of financial aid resources.
Equal opportunity and affirmative action goals are established with the aid of information, and data are used to assess progress toward the goals. The establishment of salary and compensation goals, policies and guidelines can be informed by institutional research. Investigations of equity in salaries for faculty, administrators and support staff draw upon a wide variety of variables. Institutional research offices may also become involved in litigation on issues of affirmative action and salary discrimination by serving as the source of "official" data and analysis.
Institutional research can be applied in the evaluation and improvement of such programs as academic advising, counseling, career planning, placement, intercollegiate athletics, health services and housing.
Questions about faculty workload and considerations of policy pertaining to it can be illuminated by institutional research. Current workload patterns can be measured. Faculty preferences regarding workload patterns can be determined.
Questions about the size, composition and quality of the faculty can be subjects for institutional research. Promotion and tenure practices and rates can be displayed and analyzed, and a faculty-flow model can be used to project the effects of alternate assumptions or policies on numbers and characteristics of the faculty at points in the future. Characteristics and preferences of faculty members, described by institutional research, can be useful in planning programs of faculty development. Issues involved in the appraisal of faculty performance--whether arising from goals of faculty development or from questions about promotion, tenure and salary policies and procedures--can be subjects for institutional research. Information which is the subject of collective bargaining, where it exists, is assembled by institutional research.
The processes involved in resource acquisition and allocation rely on institutional research. Budgets are analyzed in the contexts of goals, priorities, workload and performance. Income and expenditure projections are made to guide budget planning. Costs analyses are carried out in support of various responsibilities of governance.
Institutional research can contribute to the institution's development program. It can provide and assist in organizing information about the institution used in proposals for external funding of specific projects; it can assist in building case statements for fund-raising campaigns; and it can contribute to designing information-based strategies for seeking donations from foundations, corporations, and individuals.
Institutional research can aid in the formulation of policies, structures and rates for student tuition and fees. Enrollment projections can be translated into projections of tuition and fee income.
Facilities planning, allocation and management are guided by institutional research. The inventory of buildings and rooms is maintained. Utilization of classrooms and other types of space is measured and compared with standards to guide reallocation decisions. Assessments of the condition, suitability and utilization of existing facilities combine with the requirements of programs to produce plans for maintenance, rehabilitation, remodeling and new building. Energy use and conservation are areas of study which have obvious applicability to problems facing colleges and universities.
Institutional research alone cannot lead to sound plans, appropriate policies, or correct decisions for the college or university. The wisdom, integrity, and courage possessed by those who share the responsibilities of governance are the principal determinants of the soundness of plans, the appropriateness of policies, and the correctness of decisions. Institutional research can, however, provide data and information which contribute to and, in some instances, are essential for maintaining the quality of governance expected of an institution whose existence is based upon principles of rationality, wisdom and truth.
For additional information on the functions and methods of institutional research, the reader is referred to publications of or sponsored by the Association for Institutional Research and to selected other sources which, despite their dates of publication, are comprehensive, remain timely and can be highly recommended.
Publications of or Sponsored by the Association For Institutional Research
(Some issues of several of these publications are now out of print.)
A Declaration on Institutional Research, S. Suslow. Esso Education Foundation and the Association for Institutional Research, 1972.
A Primer on Institutional Research, J.A. Muffo & G.W. McLaughlin eds. Association for Institutional Research, 1987.
Annual Forum Proceedings. Association for Institutional Research. Published annually 1963-1984; titles vary.
Annual Proceedings, European AIR Forum. EAIR Secretariate, University of Twente, The Netherlands (contact AIR for specifics).
Bibliography of Reference Sources, W.P. Fenstemacher & D.R. Coleman, eds. Association for Institutional Research, 1984.
General Session Presentations. Association for Institutional Research. Published annually since 1985; titles vary.
Higher Education: Handbook of Theory and Research, J.C. Smart, ed. New York: Agathon Press Inc. Published annually since 1985.
New Directions for Institutional Research. San Francisco: Jossey-Bass Inc. Published quarterly since 1974; titles vary.
Research in Higher Education. New York: Human Sciences Press, Inc. Published since 1973 (by APS Publications from 1973 to 1979 and by Agathon Press from 1979 to 1988); four issues a year, 1973-75; eight issues a year, 1976-1988; six issues a year since 1989.
The AIR Professional File. Association for Institutional Research. Published since 1978; up to four issues a year.
The Design, Production and Use of Computer Graphics: A Tutorial and Resource Guide, R.S. Mims. The Association for Institutional Research, 1987.
Improving Academic Management: A Handbook of Planning and Institutional Research, P. Jedamus, M.W. Peterson & Associates. San Francisco: Jossey-Bass Inc., 1980.
Institutional Research in the University: A Handbook, P.L. Dressel & Associates. San Francisco: Jossey-Bass Inc., 1971.
State-Wide Planning in Higher Education, D.K. Halstead. Washington, D.C.: U.S. Government Printing Office, 1974.