Building Communities for Assessment of Learning

Shari Ellertson is Director of Institutional Research at Boise State University. She presented sessions on building institutional capacity to conduct assessment and teaching others to write effective outcomes at the 2015 Forum in Denver this past May.

Interview by Lisa Gwaltney

eAIR: How did you get started in assessment and how did that evolve into a career in IR?

ShariE.jpgIn graduate school, I didn’t actually know what assessment of student learning looked like, but I wanted to learn. I was fortunate to be surrounded by people who were known in the assessment field and I learned through observing, being on committees, and talking with faculty about their programs. My initial work in assessment really happened organically out of my own curiosity about how people learn.

That work in assessment translated into me helping others understand IR work involving research papers, complex analyses, and even facilitating communities of people interested in data.

eAIR: What is your involvement with other campus departments as they write learning outcomes?

One of the things that has changed in IR is that our job responsibilities have expanded to include the coordination and completion of assessment activities. To be effective, IR professionals find themselves in the role of teaching others how to do assessment (write outcomes, develop plans, collect and analyze data)—it is no longer just us doing the work.

I am regularly involved in individual consultations, workshops, and discussion-group settings where several different units come together for active work sessions. It’s been important to me to walk people through the process and create a feedback/peer group to assess the effectiveness of learning outcomes developed during these sessions.

For a learning outcome to be effective, it must be clear, meaningful, and measurable. I often ask people to answer questions such as: Can I explain it to others? Do others know what I mean? Is this useful and meaningful? Does it matter? Can we measure or observe this?

eAIR: How do assessment and IR professionals develop the skills to train and teach others to write meaningful learning outcomes?

By building networks, working within faculty/staff governance, developing key partners, being on campus assessment committees, etc. It’s not just about delivering workshops or content, but helping to develop a shared language around outcomes by involving others.

It’s also important to provide examples of learning outcomes to gain additional perspectives and a richer experience. As IR and assessment practitioners, we need to acknowledge that a plan is not going to be perfect the first time around. It is a process, not a destination, and that’s why community is so important. We need to learn from what others have done—both the successes and the failures. It really is a cultural shift.

eAIR: What strategies do you use in transferring knowledge to others about effective assessment?

There are a variety of ways to get stakeholders engaged, such as modeling behaviors, practicing, and providing adaptable strategies they can use to help develop assessment plans. My goal of inclusivity means that I strive to involve others and provide tips and support so they can work with colleagues in their departments or units.

  • I often provide templates for writing learning outcomes. For example: “participants/students will (action verb) [specific action/skill they will be able to do].”  The learning outcomes need to focus on what students will know (knowledge), be able to do (skill), and/or value or appreciate (disposition) as a result of the learning experience and not on what we are doing for the participant/student. I also provide a list of action verbs (e.g., define, explain, analyze) aligned to Bloom’s Taxonomy, which gives people opportunities to explore levels of learning.

  • Another strategy is to define what success looks like. I previously mentioned clarity, utility, and measurability as key components of effective outcomes. For example, to get others to operationalize the concepts, ask them to put themselves in the shoes of an external reviewer for a program just like theirs and then have them describe how they would evaluate outcomes. What would differentiate an effective outcome from one that is not? Brainstorming and creating rubrics are a few of the successful ways to get people talking about criteria for evaluating assessment plans, learning outcomes, and assessment strategies.

  • An example of an adaptable strategy is the use of visioning activities and scenarios to facilitate brainstorming. For example, if working with student affairs, you might have them imagine that they are meeting with top employers in the region to convince these companies to recruit their students. They will have to explain the skills and abilities these students have to persuade employers to hire them. I have found these types of activities to be great for brainstorming and conversation starters with members of a department/unit.

Ultimately, I have found that our mindset and how we engage people is extremely important. Rather than being prescriptive we need to be inviting. We must start from the perspective that we are all educators; we are all involved in this process.

My emphasis in both IR and assessment is around “meaning making.” It doesn’t happen in a vacuum. Assessment of learning calls us to be more collaborative and to be more involved in community building. Initially this may seem difficult, as we all have reports that are due and deadlines to meet. However, we must ultimately become bridge builders between and among units within our institutions.