Using Quality Benchmarks for Assessing and Developing Undergraduate Programs

Using Quality Benchmarks for Assessing and Developing Undergraduate Programs
 
Jossey-Bass (2010)
 
By Dana S. Dunn, Maureen A. McCarthy, Suzanne C. Baker, and Jane S. Halonen
 
Reviewed by Marv Noltze
 
The authors of Using Quality Benchmarks for Assessing and Developing Undergraduate Programs have extensive experience and bring considerable authority to bear on the topic of program development. The range and depth of their experience is evident in the book’s tone and in the specificity provided.
 
Undergraduate program quality—within and across institutions—spans a wide range. Consequently, there is a need for a common set of benchmarks to focus discussions about evaluation. The authors of this book highlight a pattern of issues related to program health, which they categorize into eight areas or domains: program climate, accountability and assessment matters, student learning outcomes, student development, curriculum, faculty characteristics, program resources, and administrative support.
 
They employ a developmental approach across these areas to establish a framework for program evaluation. The domains are examined chapter by chapter in terms of importance, scale, management, and implementation. Rubrics for a multitude of criteria are provided for each domain, and a continuum that ranges from undeveloped to developed, effective, and distinguished is used; these matrixes function as benchmarks of program quality. Each chapter concludes with guiding questions that prompt the reader to consider how aspects of each benchmark can be fully incorporated into the overall pattern.
 
This is a holistic reference that can guide academic leaders in program improvement in an organic fashion.
After the domains are discussed, the book addresses special contexts in which benchmarks may have to be adapted or modified due to the unique nature of a discipline, such as the assessment of interdisciplinary programs, the arts, and natural sciences. They advise the reader to consult discipline associations and to introduce innovations where possible. Recommendations are provided for organizing and conducting self-studies in addition to employing the benchmarks to guide external reviews. The book includes an appendix that contains a range of potential internal data sources to document each of the domains. It concludes with a list of discipline-related accrediting organizations that are useful for obtaining answers to specific program concerns.
 
The authors acknowledge that their approach is not a cookbook formula; rather, it is flexible and can be adapted to the specific needs of many institutions and settings. Based primarily on observations from four-year public universities, the benchmarks and recommendations are deployable in most colleges and universities. This is a holistic reference that can guide academic leaders in program improvement in an organic fashion.
 
The authors’ proposal, if implemented and documented as advised, builds a solid base for accreditation purposes and fits with existing literature on assessment and creation of cultures of demonstrated student learning. It is consistent with the tenets of the Association of American Colleges and Universities (AAC&U) Liberal Education and America’s Promise (LEAP) initiative, including incorporation of high-impact practices, authentic assessments, and essential learning outcomes. Moreover, benchmarking in these areas creates what Kuh, Kinzie, Schuh, and Whitt (2005) called “an improvement-oriented ethos” that is “a perpetual learning mode” which seeks “to maintain momentum toward positive change” (p. 133).
 
This book is a useful reference in shaping discussions on alignment of student learning outcomes (SLO) between two-year and four-year institutions that participate in the AAC&U Quality Collaboratives program. The book’s authors note that each SLO has multiple layers, including university, program, and course (p. 75); each SLO also has layers between different degree levels, and this reality has fostered discussion about the means by which two-year and four-year universities can align and build upon learning outcomes for transfer students.
 
While the authors cover a wide breadth of subjects that contribute to a program’s health in the eight domains, some topics would have benefitted from further discussion. An area that could be termed “environmental integration” was touched on in the program climate and faculty characteristics chapters, but did not receive the thorough scrutiny it deserves.
 
 
Much of the discussion of undergraduate education improvement tends to address solitary topics…In contrast, this approach calls for broad, comprehensive examination of many factors that contribute to a discipline’s quality.
Also, higher education is increasingly held accountable by employers, the marketplace, and the general public for graduates’ skills and relevance. Perhaps an additional criterion could be added to assess the relationship between an academic department and the marketplace it serves. It is common for community colleges to consult employers and community leaders when shaping programs, and similar practices in the four-year campus environment are becoming more prevalent. The Lumina Foundation’s Degree Qualifications Profile (DQP) project was informed, in part, by employer input regarding expectations of graduates’ demonstrated skills and competencies for entering the 21st century workplace. It logically follows that program evaluation should include consideration of the depth of relationships between academic programs and employers as well as how programs and their graduates are perceived in the greater marketplace. 
 
In addition, it would have been helpful if the authors had included specific examples of practices that lead to distinguished status for benchmarks of prime interest, such as student development and curriculum evaluation. While it is not feasible to provide an exhaustive list for each domain, resources could be cited for reference and clarity.
 
The authors’ approach adds to the widely discussed program evaluation topic in several important ways. The developmental structure of this method requires participants to make ongoing commitments to each domain rather than focus their attention only when their units are due for cyclical reviews. Much of the discussion of undergraduate education improvement tends to address solitary topics, such as access or student learning assessment. In contrast, this approach calls for broad, comprehensive examination of many factors that contribute to a discipline’s quality. In essence, it asks the question: What factors contribute to or hinder this program’s ability to thrive? The body of evidence generated along the way will create continuity within a unit and also facilitate formal program review.
 
This book is a valuable asset to a wide audience, including institutional academic leaders, department chairs, faculty members, and others concerned with improving undergraduate education. It provides a sound template with specific exemplars for examining undergraduate programs along many dimensions. It will prompt readers to gauge their programs based on the benchmarks and to consider other sources of evidence and activities to document and refine outcomes. This is a resource readers will refer to for a range of purposes, including institutional and program accreditation, internal studies, and other assessment endeavors.
 
References
 
Adelman, C.; Ewell, P.; Gaston, P.; & Schneider, C. G. (2011). The degree qualifications profile. Indianapolis, Ind.: Lumina Foundation. Retrieved from http://www.luminafoundation.org/publications/The_Degree_Qualifications_Profile.pdf

Association of American Colleges & Universities. (n.d.). Liberal education and America’s promise [Web site]. Washington, D.C.: Author. Retrieved from http://www.aacu.org/leap/index.cfm

Association of American Colleges & Universities. (n.d.). Quality collaboratives: Assessing and reporting degree qualifications profile competencies in the context of transfer [Web site]. Washington, D.C.: Author. Retrieved from http://www.aacu.org/qc/index.cfm

Dunn, D. S; McCarthy, M. A; Baker S. C.; & Halonen, J. S. (2010). Using quality benchmarks for assessing and developing undergraduate programs. San Francisco, Calif.: Jossey-Bass. 

Kuh, G. D.; Kinzie, J.; Schuh, J. H.; & Whitt, E. J. (2005). Student success in college: Creating conditions that matter. San Francisco, Calif.: Jossey-Bass 

Lumina Foundation. (n.d.). Degree qualifications profile [Web site]. Indianapolis, Ind.: Author. Retrieved from http://www.luminafoundation.org/tag/dqp/

Marv Noltze is an Information Manager at the University of Wisconsin Oshkosh, a partner institution in the AAC&U Quality Collaboratives initiative.

 

 

 Comments:

 
To add a comment, Sign In
There are no comments.