allblogs

May 30 2024

Crafting Compelling Narratives: A Guide to Presenting Themes from Qualitative Data

This article is rated as:

In our previous article, Interpreting themes from qualitative data: thematic analysis, we introduced the concept of thematic analysis, provided you with a 5-step process to complete your analysis, and highlighted some common challenges with thematic analysis.

In this follow-up article, we explore how to present your themes effectively. Whether you’re developing a comprehensive final report, a concise summary report, an eye-catching infographic, or a presentation, how you present your themes can significantly affect the clarity and impact of your findings.

Here are our 8 top tips to master the art of presenting themes from thematic analysis. Don’t forget to download our free infographic here!

Step 1: Understand Your Audience and Purpose

The first and arguably most important step in presenting your themes is understanding your audience and purpose. Before delving into the specifics of presenting your themes, it’s essential to consider who will be consuming this information and why.

Consider Your Audience:

Familiarity: Assess the understanding and experience of your audience with evaluation and qualitative analysis techniques. Do they possess a strong understanding of these concepts, or do they have differing levels of experience?
Audience Engagement and Relevance: Consider the diverse interests, engagement levels, and specific needs of your audience. You will need to tailor your presentation of themes and content to both captivate and resonate with their interests while ensuring that your analysis directly addresses their concerns and contributes meaningfully to their decision-making processes.

Clarify the Purpose:

Desired Outcome: Determine what you aim to achieve with your thematic analysis. Are you seeking to inform, persuade, inspire action, or spark discussion?
Fulfilling Evaluation Questions and Objectives: Align your analysis with the broader objectives of your evaluation. How do your themes contribute to answering your evaluation questions or addressing your evaluation aims?

By taking the time to understand your audience and purpose, you can tailor your presentation of themes to effectively communicate your findings and achieve your desired outcomes. This foundational step sets the stage for the rest of your theme summary, guiding subsequent decisions regarding content, format, and delivery methods.

Now it’s time to move forward with writing and presenting the themes!

Step 2: Provide Context

Providing context is essential for ensuring that your audience fully comprehends the significance and relevance of the themes you present.

Provide an Overview of the Evaluation Questions:

Clearly State your Evaluation Questions: Articulate your evaluation questions or objectives. This establishes the foundation for your thematic analysis and provides clarity to your audience regarding the purpose of your evaluation.
Highlight the Significance: Explain the importance of your evaluation question and the broader implications of addressing it. This contextualizes your themes within the project or program being evaluated, aiding your audience in understanding the relevance and significance of your findings.

Provide an Overview of the Methodology:

Outline Your Methodological Approach: Describe the methodology employed for conducting the thematic analysis, including the qualitative data collection methods utilized, such as interviews, focus groups, or document analysis.
Explain Your Analytical Process: Provide an overview of the steps undertaken to analyze the data and derive themes. Highlight any specific analytical techniques, whether inductive or deductive, and mention any software utilized for analysis.
Address Rigor and Trustworthiness: Discuss the measures taken to ensure the rigor and trustworthiness of the thematic analysis, such as inter-coder reliability checks, member checking, or reflexivity.
Discuss Data Saturation: If applicable, mention whether data saturation was achieved and its impact on the thematic analysis process.

Provide Supplementary Materials:

Include Appendices: Consider including supplementary materials, such as a copy of your codebook or a list of related themes, in an appendix of your report. This allows interested readers to delve deeper into the analytical details of your thematic analysis and enhances transparency and accessibility.

Offering a thorough overview of your evaluation question, methodology, and data collection process establishes a strong foundation for presenting your themes. This context helps your audience know the origin and importance of the identified themes which can enhance their understanding of your evaluation findings.

Step 3: Organize Your Themes

Themes within a thematic analysis are often presented in a narrative format. Organizing these themes is crucial to ensure a smooth flow in your narrative, making it easy for your audience to follow. This step is important in guiding your audience through the logical progression of your analysis, resulting in a coherent and understandable presentation of your findings.

Group Related Themes:

Identify Common Threads: Review your themes to identify any commonalities or relationships. Group themes that share similar characteristics or are conceptually related.
Create Theme Clusters: Consider clustering related themes together to form overarching categories or sub-themes. This structured approach provides clarity and coherence to your presentation.

Establish a Logical Flow:

Introduction and Overview: Begin by introducing the overarching themes and providing an overview of what will be covered. This sets the stage for a deeper exploration of each theme.

I see three main ways to structure your themes and analysis:

1) Alignment with Evaluation Questions: This method focuses on organizing themes according to the specific evaluation questions they address, ensuring direct alignment with the objectives of the evaluation. It provides a targeted approach to presenting themes that directly respond to the research questions or outcomes.

2) Sequential Presentation: This method involves organizing themes in a sequence that mirrors the progression of the evaluation or analytical journey. It presents themes in the order they were identified or developed during the analysis, providing a logical flow that reflects the process of exploration and discovery. Sequential presentation focuses on the order of identification or development as well as the weight of the themes and their prominence i.e., those that were the most prominent are discussed first.

3) Chronological Order: This method entails presenting themes in chronological order to reflect the sequence of events captured in the data. It highlights temporal patterns or changes over time, offering insights into the evolution of themes within the context of the evaluated phenomenon. Chronological order emphasizes the temporal sequence of events.

When structuring themes and analysis for evaluation, it’s essential to consider factors such as the nature of the data, evaluation objectives, and audience preferences. For advisory groups or those deeply involved in the evaluation, a detailed approach focused on the alignment with the evaluation questions may be the most appropriate. While decision-makers may favour upfront, concise insights outlined through sequential presentation. Operational teams may benefit from practical, actionable recommendations tailored to their daily tasks as presented in chronological order. By aligning the presentation format with audience needs, clarity and relevance are maximized, enhancing the impact of evaluation findings.

Transition between Themes: Make sure your themes flow smoothly by using clear signals or phrases to transition between them. This keeps your analysis easy to follow and helps your audience stay engaged.

Logically organizing your themes helps people understand and stay interested. When your presentation is well-organized, it’s easier for your audience to absorb your analysis and get a deeper grasp of your findings.

Step 4: Use Clear and Descriptive Titles

Crafting clear and descriptive titles for your themes is essential for effectively communicating the focus and essence of each theme to your audience.

Craft Effective Theme Titles:

Capture the Main Idea: Ensure that each theme title succinctly captures the main idea or concept it represents. Aim to condense the essence of the theme into a few words or a brief phrase. Clear and descriptive titles serve as signposts that orient your audience and provide insight into the content that follows.
Grab their Attention: Create captivating titles that grab your audience’s attention and make them curious. Well-chosen titles can draw people in and encourage them to learn more about the themes discussed in your presentation.
Avoid Ambiguity and Complexity: Choose words and phrases that are clear, straightforward, and easily understood by your audience. Avoid jargon, technical terminology, or unclear language that may obscure the meaning of the theme. Keep it simple to make sure your message is clear.

By using clear and descriptive titles for your themes, you make your analysis easier to understand and more impactful. These titles act as a roadmap, helping your audience navigate through your thematic analysis and ensuring that your main points are communicated clearly. For example: “Understanding Engagement Levels: In-person programming supports higher engagement compared to online”.

Step 5: Share Example Quotes

Using real quotes from your qualitative data makes your themes more vivid and enhances how you present your findings. As we discussed in Interpreting themes from qualitative data: thematic analysis, you can set up a special “example quote” feature in your data analysis software to quickly highlight quotes that represent each theme, enabling easy retrieval while you write up your narrative. This can help to save time, streamline the quote selection process, and eliminate the need to re-read data excerpts later on.

Bring Themes to Life:

Select Meaningful Quotes: Pick quotes that clearly show what each theme is about. Choose quotes that capture the main ideas, emotions, or experiences related to the theme. These quotes should connect with your audience and help explain your analysis clearly.

Represent Diversity:

Ensure Diverse Perspectives: Ensure that the quotes you select represent a diverse range of perspectives, voices, and experiences within each theme. This inclusivity adds richness and depth to your narrative.
Use a Variety of Sources: Draw quotes from various sources within your data, such as different interviewees, focus group participants, or document excerpts. This variety showcases the breadth of your analysis and reinforces the credibility of your findings.
Ensure Ethical Processes: Ensure that any ethical processes, such as removing identifiable features from quotes, have been followed. In projects with small populations, consider removing any unique turns of phrase or colloquialisms that could potentially identify the speaker.

Using quotes that illustrate each theme enriches your presentation of qualitative findings and helps your audience better understand your analysis. These carefully chosen quotes act as strong evidence, bringing your themes to life, and making your analysis more compelling.

Step 6: Interpret and Analyze Your Findings

Providing a thorough interpretation and analysis for each theme is crucial for enhancing the depth and impact of your thematic analysis.

Explain the Significance of the Theme:

Highlight Relevance: Emphasize how the theme contributes to the overall narrative of the findings. Discuss how each theme sheds light on key aspects of the program or intervention under investigation.

Explore Implications:

Discuss Practical Applications: Explain how insights from the theme can inform program directions or decision-making processes.

Demonstrate Analytical Rigor:

Demonstrate Depth of Analysis: Demonstrate the depth of your analytical engagement with the theme by going beyond surface-level description. Provide nuanced insights and interpretations that reflect your deep understanding of the data.
Reflect Critically: Critically reflect on the themes you’ve presented, recognizing any limitations or biases in your analysis. Discuss different ways to interpret the data or conflicting viewpoints, showing your dedication to thorough analysis and self-awareness.

Tell a Coherent Story:

Ensure Your Story Flows Smoothly: Make sure that your interpretation and analysis fit together smoothly to tell a clear story with your data. Blend themes, interpretations, and evidence seamlessly to create a persuasive and engaging narrative.

Connect to Existing Literature:

Integrate Existing Literature: If applicable, situate the theme within the existing body of literature on the topic.
Align with Relevant Theoretical Frameworks: Consider how the theme aligns with theoretical frameworks or conceptual models underpinning your evaluation if applicable.

By offering thoughtful interpretation and analysis, you enrich the presentation of your themes, adding depth and complexity to your findings and demonstrating analytical rigor.

Step 7: Visualize Your Themes

Visualizing your themes through charts, graphs, or thematic maps can significantly enhance the clarity and impact of your presentation of themes. Visualizations provide an alternative mode of understanding that complements textual descriptions. Incorporating visual elements can increase engagement and retention of information. Take a look at our article: 3 Easy Ways to Quantify Your Qualitative Data.

Choose Appropriate Visual Formats:

Select Visual Aids: Choose visual formats that are suitable for representing the nature of your qualitative data and the relationships between themes. Options include bar charts, line graphs, pie charts, scatterplots, or thematic maps (see our previous article for an example).
Match Complexity: Match the complexity of your data and analysis with the appropriate level of detail in your visualizations. Simple visualizations may suffice for straightforward themes, while more complex themes may require more elaborate representations.

Convey Complex Relationships:

Highlight Patterns: Use visualizations to highlight patterns, trends, or relationships between themes that may not be immediately apparent from textual descriptions alone. Visual representations can help reveal underlying structures or dynamics within the data.
Compare and Contrast: Employ side-by-side comparisons or juxtapositions of themes to facilitate comparisons and contrasts. This allows your audience to discern similarities, differences, or variations across themes more easily.

Customize Visualizations:

Tailor to Audience Needs: Customize your visualizations to match your audience’s preferences and needs, considering factors like their familiarity with visual data and cultural backgrounds.

Integrate with Narrative:

Visual Integration for Impact: Incorporate visualizations seamlessly into the narrative to support and strengthen key points and themes. Ensure they enhance your narrative rather than detract from it.
Narrative Alignment: Ensure that the visualizations you choose align with the narrative structure and contribute meaningfully to the overall coherence and flow of your argument.
Add Contextual Information: Include explanations, labels, and notes with your visualizations to help your audience understand the meaning and importance of the theme.

Based on my experience, infographics are generally enhanced by visuals like icons, images, and pie charts, while detailed visuals such as scatterplots and thematic maps are better suited for reports. By turning your themes into visual representations, you make ideas easier to grasp and more engaging. Thoughtfully selected visuals add depth and interest to your themes, making them more dynamic and captivating for your audience.

Step 8: Conclude with Key Insights

Wrapping up your presentation of themes with a synthesis of key insights is important for solidifying understanding and emphasizing the significance of your findings.

Highlight Main Findings: Summarize the main themes and findings that emerged from your thematic analysis. Provide a concise overview of the key insights gleaned from your data and their relevance in addressing your evaluation questions and objectives.
Explore Practical Applications: Consider practical applications of your findings and how they can inform decision-making, policy development, or future programming.
Identify potential areas for future evaluation: Discuss unresolved issues, areas needing further investigation, or opportunities for methodological improvement based on the insights from your thematic analysis.

Summarizing key insights from your thematic analysis wraps up your narrative with a clear understanding of your findings and their broader impact.

Conclusion:

Presenting themes from thematic analysis is not just about summarizing your findings but also about effectively communicating the richness and complexity of your data. By understanding your audience, organizing your themes thoughtfully, and providing context, interpretation, and visualizations, you can deliver a compelling presentation that showcases the depth of your research insights.

May 30 2024

Is A Learning Log Right For You?

By Rebecca Perlmutter and Cory Georgopoulos, Innovation Network Senior Associates

As evaluators, we’re always seeking to incorporate methods that help us collect data in comprehensive and equitable ways. In recent years, we have found learning logs to be critical tools for enhancing our evaluation work. Based on Emergent Learning principles, learning logs are used to capture insights and events in real-time by recording an initiative’s challenges, experiments, and successes (including factors of success). Learning logs allow teams to identify themes across a particular initiative or scope of work, and to better understand that project’s full trajectory. Teams can also revisit their learning log as they reflect on their work, grounding their conversations in the concrete stories and data they’ve gathered.

In October 2023, we presented our experience using learning logs for long-term feedback and short-cycle reflection at the American Evaluation Association’s annual conference. Our presentation focused on our recent evaluation work with two of our clients, whom we’ll refer to here as Project A and Project B. In both projects, we employed learning logs as an experiment and as an opportunity to collect rich qualitative data. We’ve distilled some of our key takeaways into this blog to shed more light on learning logs as a useful evaluation tool, and to share important considerations for when and how to incorporate them into your own work.

Project A

Our Project A client is a community of practice of advocates who share policy goals, build capacity, and develop a mutual commitment towards advancing racial equity within their field. Innovation Network partnered with Project A to help improve its cohort facilitation, identify how it could build shared power among its organizations, and capture the learning interests of its participants. To do so, we conducted five monthly after-action reflection meetings with Project A’s facilitation team and used a learning log to collect data from these meetings. Prior to each session, Project A’s three facilitators populated their own learning log (using the digital collaboration tool Mural) with the successes, challenges, and experiments they had experienced since the previous month. Our team would reference their Mural to calibrate our own facilitation and follow-up questions. We discussed the same set of questions with the facilitation team every month and took notes. We then pulled out significant insights and events from our notes and added them to our learning log. We were also able to use the learning log for rapid reflection with Project A’s facilitators. After each session, we summarized and distilled our learning log entries and shared them with the facilitation team. They would then revisit this summary before our next session.

Key takeaways

Our role as external evaluators added an extra layer to this project. Since we maintained the learning log and only met with the Project A facilitators monthly, there was often a lag with the data we collected. Sometimes a lot would happen between our calls — and sometimes not much would, which was a different challenge! Our process was time-intensive for Project A’s small group of facilitators, which made us realize that this approach works best with fewer people in general. Their facilitation team didn’t always have the time to pre-populate their learning log or review previous entries, and at times, it would also take our team a while to fill out the log. Additionally, because the data we collected was filtered through Project A’s facilitators for the sake of logistical ease, it was still one step removed from participants in the community of practice.

Despite these challenges, Project A’s facilitators enjoyed having these conversations because they provided a dedicated opportunity to step back and reflect on their work. We weren’t just asking them to recount their recent actions — we also wanted to know their thought processes and the conditions that influenced their decisions. Over time, we got better at interpreting what they meant, and we captured different perspectives depending on the proximity of our sessions to one of their events. Having the team’s thoughts ahead of time via their Mural board was also very useful for facilitating our sessions and allowing them time to process. By conducting these regular sessions together, we were able to build a strong relationship with Project A’s facilitators and ultimately collect comprehensive data.

Project B

Our Project B client is a national nonprofit that works with policy advocates at the state level. Project B invited Innovation Network to conduct an evaluation on their efforts to align their internal processes with their new theory of change (TOC), an effort to shift their role in the advocacy space from focusing on policy wins to creating environments that allow for larger, transformational change. We conducted our evaluation in two phases: The first focused on Project B’s experience operationalizing their new TOC, and the second (which is still ongoing) is evaluating the outcomes of this transition. We met regularly with the Project B team in “reflection moment” conversations, which were designed to capture their insights and assumptions about aligning to the TOC. Similar to our work with Project A, we had Project B staff populate a Mural with responses to our reflection questions, which we used to inform each session. Within the sessions, Project B staff reflected on how they were adapting, what they hoped to achieve, and what was and wasn’t working as they experimented with changes to become more equitable and aligned to their TOC. We captured their reflections in a learning log after each session, and shared these learnings back with their staff in summarized versions.

Key takeaways

The amount of time required to populate the learning log was a challenge in our work with Project B, just as it was with Project A. We gathered a lot of insights during our reflections, so we had to decide what was “important” to include. We also spent quite a bit of time synthesizing our findings from the learning log to create summaries for the Project B team, although they did not always have time to review these summarized versions prior to the next session.

Ultimately, the Project B team said they learned a lot through our reflection conversations, and they were able to apply their learnings to their immediate work aligning to the TOC. We also used the analysis of our learning log from Phase 1 to map the learning areas that the Project B team is interested in focusing on during Phase 2 of our evaluation, as well as to document hypotheses about the outcomes the Project B team expects to see as a result of their changes around the TOC.

Is a learning log right for you?

We hope the above examples from our work have given you better insight into the practical application of learning logs in evaluations. If you’re considering employing this tool yourself, we have some recommendations for how to effectively incorporate a learning log into your evaluation work:

Establish a clear vision

Due to the sheer amount of data you’ll collect within your evaluation, it’s important to know what you want to get out of your learning log before you start making entries. Consider asking yourself: What is the question I’m trying to answer? Determining the purpose of your learning log will help keep you on track during your sessions and inform your facilitation.

Narrow your focus

Our work with Project A demonstrated that a learning log works well for gathering a lot of detailed information on one topic. If you’re covering numerous topics — as we did in our evaluation for Project B — we recommend keeping your analysis focused on the specific areas of work. These areas will inform your reflection questions and help you facilitate conversations that speak to the question you’re trying to answer.

Allow yourself enough time

As evaluators, we recognize time is a precious resource. However, in order to elicit useful data from a learning log, it’s important to allot enough time — for both yourself and your participants. Avoid employing a learning log if you’re pressed for time within your own schedule, or if it’s difficult to establish regular meetings with your participants. In addition to contributing to your learning log, you’ll want ample time to synthesize emergent themes and share your findings with your participants along the way. Your participants will also benefit from having enough time to reflect on your questions in advance, contribute to their own learning log, and digest the learnings you share with them.

Be consistent

One of our key takeaways across both projects was understanding the importance of consistent facilitators. Assigning the same team members to maintain the learning logs and facilitate reflections meant that we were better able to keep track of salient insights, identify big-picture themes, and use our past knowledge to shape the direction of future sessions. While you may not be able to keep your reflection participants consistent, maintaining the consistency of your own evaluation team will ensure you understand the full breadth of your learning log findings.

In our work with Project B, our participants did often change from session to session. As a result, we’re curious how maintaining consistent questions across all of our reflection sessions — as we did with Project A — would have impacted the kind of data we received. It’s worth considering whether you should strive for consistency within your facilitation questions if you know your participants will change throughout the course of your evaluation.

Be adaptable

While collecting this data in your learning log will allow for significant insights, it may also be overwhelming for your participants. Be prepared to adjust your approach to accommodate their needs and maintain their buy-in throughout the process. Throughout the course of your evaluation, you may also realize that the data that emerges from your learning log is different from the assumptions you made at the beginning of the project. For example, we pre-coded the data within our Project B learning log based on our ideas about which topics would rise to the top of our conversations. Along the way, we realized these codes weren’t necessarily accurate or relevant, and we had to re-do our coding system to ensure a more thorough analysis. While it’s important to have a system in mind for how you’ll distill and summarize your learning log data, be mindful that your specific approaches may need to change based on what you learn.

Incorporate checks and balances

Using a learning log as an evaluation tool often means that, as the evaluator, you will be the one interpreting all the data you’ve collected. Throughout the process of your evaluation, consider incorporating opportunities to garner your participants’ feedback on your findings. Do they agree with your interpretation? Why or why not? Incorporating regular checks and balances to your interpretations will not only ensure your participants feel heard, it will also help your findings be as thorough and accurate as possible.

Final thoughts

Ultimately, using learning logs within our work at Innovation Network has enabled us to garner extensive data for our evaluations. By facilitating and capturing reflective discussions with our clients, we created touchpoints that allowed them to reflect on their work and stay involved in our evaluations. In this way, we were also able to establish stronger relationships with our clients, and as a result, to identify more useful themes and learnings they could then apply to their work. Learning logs may require a commitment to maintain, but the payoff is often worth it.

Have you used a learning log in your evaluations before? We’d love to learn about your experience in the comments below as we continue to incorporate this tool into our own evaluation practices. Interested in experimenting with a learning log for yourself? Download our sample template to get started.

Is A Learning Log Right For You? was originally published in InnovationNetwork on Medium, where people are continuing the conversation by highlighting and responding to this story.

May 29 2024

Evaluaciones de Sistemas: Transformando Políticas y Decisiones

Las evaluaciones de sistemas desempeñan un papel crucial en la configuración de soluciones políticas y en impulsar cambios transformacionales. Desglosemos su importancia:

Comprendiendo Sistemas Complejos:
- Las evaluaciones de sistemas nos ayudan a comprender sistemas intrincados, ya sean sociales, económicos o ambientales.
- Al analizar interconexiones, bucles de retroalimentación y comportamientos emergentes, obtenemos perspectivas sobre cómo diferentes componentes interactúan e influyen en los resultados.
Identificando Puntos de Apalancamiento:
- Las evaluaciones revelan puntos de apalancamiento: áreas donde intervenciones pequeñas pueden llevar a un impacto significativo.
- Estos puntos guían a los formuladores de políticas hacia estrategias efectivas para el cambio.
Toma de Decisiones Basada en Evidencia:
- Las evaluaciones rigurosas proporcionan evidencia empírica.
- Los formuladores de políticas pueden usar esta evidencia para diseñar e implementar políticas efectivas.
- Por ejemplo, evaluar el impacto de las reformas educativas ayuda a refinar métodos de enseñanza y currículo.
Abogacía y Responsabilidad:
- Las evaluaciones sirven como herramientas de abogacía.
- Destacan éxitos y fracasos, responsabilizando a los formuladores de políticas.
- Los defensores pueden usar los hallazgos de las evaluaciones para presionar por reformas necesarias.
Aprendizaje y Adaptación:
- Las evaluaciones fomentan una cultura del aprendizaje.
- Los formuladores de políticas aprenden tanto de resultados positivos como negativos.
- Las políticas adaptativas evolucionan basadas en perspectivas de evaluación.
Abordando Desafíos Sistémicos:
- El cambio transformacional a menudo requiere cambios sistémicos.
- Las evaluaciones identifican cuellos de botella y oportunidades sistémicas.
- Por ejemplo, evaluar sistemas de atención médica puede llevar a un acceso equitativo y mejores resultados de salud.

En resumen, las evaluaciones de sistemas empoderan a los formuladores de políticas, informan decisiones basadas en evidencia y promueven cambios transformadores en las políticas.

May 28 2024

How to Fix Dense Maps (with Small Multiples)

A good rule of thumb: If your map feels too dense… try small multiples!

Before, we started with a single map… with 2 variables. Counts and rates. A bubble map on top of a heat map.

After, we’ll use small multiples: 2 separate maps, 1 for each variable. The headings, color-coding, takeaway sentences, bolding, and annotations will help to explain our patterns, too.

What’s Inside

0:00 Welcome to Dataviz On The Go!

0:16 Before: Map with 2 Variables (Circles AND Shading)

1:07 After: Small Multiples!

1:24 Call-Out Boxes (aka “Annotations”)

1:46 The Before-After Makeover

2:12 Your Questions??