IR Projects: Sources & Uses of Qualitative Data

Ask eAIR invites questions from AIR members about the work of institutional research, careers in the field, and other broad topics that resonate with a large cross-section of readers. If you are interested in writing an eAIR article, or have an interesting topic, please contact eAIR@airweb.org.  

This month’s question is answered by Felice Billups, Professor, Educational Leadership Doctoral Program and Research Fellow, Center for Research & Evaluation, Johnson & Wales University.

The ideas, opinions, and perspectives expressed are those of the authors, and not necessarily of AIR. Subscribers are invited to join the discussion by commenting at the end of the article.

Dear Felice: I have been asked to conduct a qualitative study regarding student perceptions of our campus services. I am familiar with 1 to 1 interviews as well as focus groups, but are there other ways to capture qualitative perceptions, experiences, and stories?

When asked to conduct a qualitative project, or to add a qualitative component to a quantitative investigation, many institutional researchers consider interviews or focus group sessions as their best – and perhaps their only – options for capturing qualitative data from campus stakeholders. While these strategies are valuable in many ways, there are many other options, although some of them may be less well known or understood.

Qualitative research is defined as a “situated activity that locates the observer in the world and…consists of a set of interpretive, material practices that make the world visible” (Denzin & Lincoln, 2011, p. 3). Based on this definition, one way to describe qualitative data is to define it as data that you can hear, intuit, touch, or see (Patton, 2015). To expand on that definition, I would like to offer a varied list of the sources of qualitative data, many of which can be effectively integrated into an institutional research project:

  • Words, conversations, narratives (1-1 interviews, conversational analysis)            

  • Synergistic discussions (focus group sessions, dyadic/triadic interviews)

  • Observed realities

  • Documents, artifacts, archival materials

  • Reflective data (reflective questionnaires, journals, diaries)

Each of these data can be collected and applied to a study in various ways. This article briefly describes the nature of each data source and provides a few examples of how you might incorporate each one into your own projects.

Words, Conversations, and Narratives  

Most researchers are familiar with depth interviews and use them in many studies where personal perspectives are warranted. Guided by a facilitator, people verbally describe their experiences in 1-1 conversations, either in person, by phone, or virtually. Many studies begin with the administration of a self-administered survey questionnaire and are followed with interviews involving a subset of the same population. These supplemental conversations are designed to clarify, enhance, and/or support quantitative findings.

Interviews are one of the most common forms of qualitative data collection, but few researchers capitalize on the extensive array of types and formats. The one-size-fits-all label should not be universally applied to all qualitative interviewing. Semi-structured interviews (conversations guided in a partially directed manner) are the most common but the unstructured interview format, which is more conversational in its flow, offers many benefits. Additionally, there is the phenomenological interview (lived experience, three-phase, or storytelling formats), the ethnographic interview (field-based cultural interviews, part of cultural studies which could just as easily be applied to the study of groups on a college campus), and the narrative interview, which comprise several different formats. The narrative life story interview differs from the narrative thematic/dialogic interview, where a researcher attempts to capture an individual’s story through specific elements of the storytelling (actors, climax, storyline, timeline, structure of words and tone of voice, and coda/epilogue). In all of these cases, although they may seem more complicated than a semi-structured interview format, they each offer a unique entry into a participant’s experience and a way to substantiate a research project. Interviewing students who are transitioning to college for the first time could constitute a phenomenological interview project, while interviewing new faculty members might comprise an ethnographic interview series to study the acculturation process of faculty members. In both instances, the value of these findings may yield meaningful implications for practice and policy.

Similarly, the practice of participating in and analyzing the conversations-in-process is known as conversational analysis, the practice of studying the structure and use of speech. This strategy is rooted in ethnographic field work traditions and reflects how people make sense of their worlds through ‘talk’. This approach can be applied when a researcher attends a campus-wide meeting where activities such as institutional self-study or strategic planning are underway. Similarly, a researcher can ask to join students who are meeting about a campus issue, as a way to better understand how students view their role in the campus community. These strategies are often connected to climate or culture instruments in the analysis of organizational functioning, or in the quest of understanding the roots of organizational behavior.

Synergistic Discussions

Another common data collection strategy consists of the focus group. Focus groups, compared with depth interviews, capitalize on the synergy of group interactions. They can serve as supplemental, exploratory activities or as stand-alone strategies; for instance, focus groups can be used to solicit feedback on a new dining services program or plans to change access to campus buildings. These data can assist researchers with the design of a survey questionnaire that can be distributed to a larger number of students or the findings can serve as an initial exploration of a program.

Like depth interviewing, there are varieties of focus groups that can be used in a college setting. The single purpose session is the most common strategy but there are other very interesting options. The multiple purpose design coordinates several group sessions to compare findings across those groups. The two-way group design allows one group to observe another group and then discuss their observations. The dual moderator and dueling moderator groups both generate rich discussions by using moderators in different, provocative ways. The brainstorming, program evaluation, and envisioning/planning groups are also very useful in a college setting, particularly when campus-wide processes such as strategic planning, program review, assessment, and institutional self-study are in play (Barbour, 2007; Krueger & Casey, 2015: Liamputtong, 2011)

Less commonly used but also very useful are dyadic interviews (facilitator-guided 2-person interviews), which comprise another type of synergistic discussion. The dyadic interview, is a depth interview that maximizes the synergy of two participants responding and reacting to one another (Morgan 2016). Dyads allow for a more personalized conversation between smaller groups of individuals, beneficial for sensitive or confidential topics. Dyadic moderators obtain rich, detailed information which provides insights that could be missed in a larger focus group session. In fact, the dyad bridges the 1-1 interview session and the focus group in many ways. These synergistic formats are easily adapted for students, faculty, employees, or any other campus stakeholders.

Similarly, there are also triadic interview formats (3-person interviews guided by a moderator) which are used by researchers in special cases. Both formats offer numerous advantages to IR professionals who wish to follow-up with other study phases, but who wish to interact with just a few individuals in a close, depth-based conversation. It is important to note, however, that once a group expands to 3-4 or more people, they may qualify as an academic focus group instead (Krueger & Casey, 2015, p. 151).

Observed Realities

Observing others may either seem like a straightforward practice, or a covert one; neither impression captures the essence of field-based observation. Lincoln and Denzin (2008) define observation as “going into a social situation and looking…as a way of gathering materials about that social world” (p. 48). Regardless, a researcher must appreciate what they are seeing, as well as what they are not seeing, with an eye for detail, nuance, and levels of interaction. Observed realities, therefore, can supplement many studies in creative and productive ways. Similar to the researcher’s involvement in a conversational analysis format, the observer-researcher may sit in a classroom or meeting space to observe the verbal and non-verbal interactions of a group. Observations can support classroom teaching strategies, or can explain or extend survey findings about a campus process, such student interactions during orientation programs or athletic events when the focus of a study is centered on sense of belonging, community, or transitioning.

Documents, Artifacts, and Archival materials

The value of gathering, reviewing, and interpreting documents, artifacts, objects, material culture items, and archival materials cannot be understated. Nearly every type of qualitative study is enhanced by including these data. In a campus environment, these materials are plentiful and often very accessible, such as websites for the institution or specific departments, the campus’s architecture, signage, symbols, and integration with existing neighborhoods; meeting minutes and records, newsletters, newspapers, catalogs, annual reports, marketing documents, and other similar items are all considered documentation. Artifacts, objects, or material culture items are typically tangible, tactile objects such as art work or public displays on bulletin board posts or electronic signage or wayfinding, and items indigenous to campus traditions such as commencement or convocation ceremonies. These are all considered data, which can support, supplement, explain, and corroborate data with other phases of a study.

For instance, if you have administered a large-scale survey about student perceptions of a new campus master plan, you might find that there are anomalies in some of the data. A combined strategy of follow-up focus groups and an analysis of student letters to the editor in the campus newspaper, blog or social media posts, and/or articles they write in support of or in rejection of aspects of the plan are all viable data that adds to your understanding of the ‘story’.

Reflective Data

What do we mean by reflective? What constitutes a reflective practice in a research environment? Finlay (2008) defines reflective practice as the “process of learning through and from one’s experience towards gaining new insights … and by being self-aware and critically evaluative” (p. 1). The most common tools for gathering reflective data comprise field note logs, journals, diaries, and reflective questionnaires. For the purposes of an IR project, reflective questionnaires and journals are two strategies which can enhance and supplement a study.

Participant reflections can be captured through open-ended reflective questionnaires that follow an earlier data collection phase. While many researchers consider a few open-ended questions at the end of a survey questionnaire to constitute reflective data, the true reflection should be holistic and emergent, rather than merely attached to a quantitative instrument as an add-on. One of the best ways to allow for these emergent reflections is to invite participants to answer one or two open-ended questions about a topic. This thoughtful processing, or even debriefing about a specific topic, allows the participant to generate feelings, perspectives, and insights that effectively corroborate with other data. Reflective questionnaires can involve a single open-ended question to follow-up with an earlier research phase, a free word association question to elicit new insights, or a scenario-based question, which asks the participant to weigh in on a proposed idea, plan, or concept.

Journals are another vehicle for capturing ongoing or interval-based reflections of an event, phenomenon, or circumstance. For example, a researcher could study the ways in which faculty and students form a sense of community in a living-learning residence hall community. Participants might be asked to maintain a journal of how the community emerged, how they felt about the interactions among group members, and how they felt about their own growth and response to the development of relationships. By using journal data to supplement a campus-wide survey administration, researchers can juxtapose the quantitative data with the journal entries to better understand a program’s strengths and weaknesses and contribute to their ongoing refinement.

These various data types and sources stress the very nature of qualitative research, which is to explore people’s lives, behaviors, emotions, perspectives, and experiences. In a research environment that is often quantitatively-centric, qualitative research offers a valuable perspective. Therefore, including a variety of qualitative data in an institutional research project can provide the institutional research professional with new ways to accomplish their work.

References

  • Barbour, R. S. (2007). Doing focus groups. Los Angeles, CA: Sage.
  • Denzin, N. K., & Lincoln, Y. S. (2011). Introduction: The discipline and practice of qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage Handbook of Qualitative Research (4th ed.), pp. 1-9. Thousand Oaks, CA: Sage.
  • Finlay, L. (2008). Reflecting on ‘reflective practice’. Practice-based Professional Learning Centre, Paper 52, 1-24.
  • Krueger, R. A., & Casey, M. A. (2015). Focus groups: A practical guide for applied research (5th ed). Los Angeles, CA: Sage.
  • Liamputtong, P. (2011). Focus group methodology: Principles and practice. Thousand Oaks, CA: Sage.
  • Lincoln, Y. S., & Denzin, N. K. (Eds.) (2008). Collecting and interpreting qualitative materials, (3rd ed.). Thousand Oaks, CA: Sage.
  • Morgan, D. L. (2016). Essentials of dyadic interviewing. Walnut Creek, CA: Left Coast Press.
  • Patton, M. Q. (2015). Qualitative research & evaluation methods: Integrating theory and practice (4th ed.). Thousand Oaks, CA: Sage.

 

 Comments

 
To add a comment, Sign In
There are no comments.