• Skip to main content
  • Skip to footer
  • Home

The May 13 Group

the next day for evaluation

  • Get Involved
  • Our Work
  • About Us
You are here: Home / Archives for allblogs / evalacademy

evalacademy

Jul 14 2022

What is Theory of Change in evaluation?

We get it, Theory of Change (ToC) doesn’t really sound like the most fun, interesting, or appealing evaluation tool. I certainly wasn’t buzzing with excitement when I first heard the phrase ToC! 

If this also sounds like you, then you’ve come to the right place. This article aims to introduce you to ToC and is aimed at those who may have heard of the tool before but aren’t quite sure as to what it is and how it can be useful for evaluation. After putting my preconceptions of the tool to one side and learning more about its uses, I have found that ToC has been one of the most valuable tools to me so far as an evaluator. 

This article is the first part of our discussion on ToC and answers the question of “what is theory of change in evaluation?”

The second part of our discussion will focus on how to develop and use ToC for evaluation. 


So, what is Theory of Change (ToC)? 

Theory of Change (ToC) explains how a set of activities will solve a problem. It does this by clearly describing the problem and the change you (and the program) intend to create.

Most importantly, ToC depicts change that is not linear. Processes of change actually have multiple feedback loops that need to be understood and described.

In short, ToC is a diagram to describe “if you do this, then these are the intended results.” When we break it down a little more we get … 

Although ToC has similar components to other evaluation tools such as logic models that mainly focus on the what, ToC goes into more detail about the why.

It does this by identifying processes called “causal logic” or “causal mechanisms” which are represented in the diagram using arrows. We will go into further detail about causal logic below.

While logic models also highlight contextual factors and assumptions, they are often focused at a high-level and don’t look at each specific step within the change process (e.g., between each outcome box). This is something that is key to a ToC. 

ToC allows us to map each step towards a long-term goal which provides an explicit and testable diagram of how and why a change is expected to happen in a particular context.

It is not just a list of activities with arrows linking them to their intended outcomes like a logic model, but it explains how these changes will happen and how the program will contribute to this change at each little step in the change process.

You guessed it, ToC creates its own program theory. It also looks at what other external factors outside of the program will affect the desired change. 

ToC can be developed for any level of an intervention – an event, a project, a program, a policy, a strategy, or an organization.

This makes it both useful for program planning and in monitoring and evaluation where it is often captured through a one-page diagram that contains multiple boxes and arrows. This diagram often has a supporting written narrative that explains each step of the theory in more detail to walk the reader through the diagram.

A ToC is a living document; it is not set in stone and should be revisited and revised as the program progresses to make the most of opportunities and address challenges that arise. Depending on the lifespan of the program, a 6-monthly or yearly review of the ToC diagram and narrative might be appropriate. 


What are the main components of a Theory of Change (ToC)? 

There are several main components of a ToC. Some of these components are included in the ToC diagram itself, while others should be captured in the supporting narrative or an alternate table. This has been explained in further detail below. 

  • Activities are the actions that will be taken by the program that are expected to contribute to the change. For example, this could include interviews with patients, training of clinicians, etc. 

  • Outputs are the products or the services that the program will create through the activities. For example, certain pieces of new knowledge (e.g., patients’ opinions) or new processes (e.g., a new clinic workflow). 

  • Outcomes are the changes in knowledge (e.g., clinicians have greater understanding of a condition), attitudes (e.g., patients change their perspectives about exercise), or skills (e.g., clinicians learn how to use new software) of the key actors that are expected to lead to a change in their behaviour (e.g., clinicians use their knowledge to support patients, patients exercise 3 times a week, MOA’s use TNA for tracking appointments). Outcomes are sometimes broken down into intermediate, end of program, and high-level depending on when they are expected to be realized in the program life cycle. However, there is no right number of outcomes which entirely depends on the program itself and the change process they are describing.  

  • Impacts are the top-level changes that result wholly or in part from the outcomes (e.g., the change process) to which the program contributed to. 

  • Purpose is the overall goal/aim the program aims to contribute to; it explains the program’s reason for existence. Unlike a Vision Statement which is more high-level and focuses on the what, the program’s purpose focuses on the why. For example, why is your program on the journey it’s on? 

  • Impact Pathways are the pathways through which the program is expected to contribute to change. These are often actor specific to make them more tangible (e.g., Clinician Capacity Building Pathway, Clinic Workflow Pathway) 

  • Causal Logic describes the how and why between each step in a ToC. In the ToC diagram it is often presented using arrows. More detail about the causal logic is captured within the supporting narrative. 

Example ToC diagram structure

Main components that need to be captured outside of the TOC diagram: 

  • Key Actors are the people who are crucial to the program. Those developing a ToC should be aware of the key actors’ roles within the context of the program, and how the program will work with these key actors to create change. Key actors can be identified using the framing of “who is doing what differently and why as a result of the program?” and captured using a Stakeholder Matrix (our template can be found here).  

  • Causal Logic describes the how and why between each step in a ToC. Detail about the causal logic which links each step in the ToC diagram is captured using the descriptive narrative that accompanies the diagram. The narrative walks the reader step-by-step through the diagram. 

  • Indicators are a marker of accomplishment that can be used to measure the success of a program (e.g., did the program contribute to a certain outcome?). These play a key role in the evaluation plan and can be captured and listed using the suggested ToC tracking table below.  

  • Assumptions are the factors outside of the program’s control that are necessary to ensure the program’s success. This is one of the most valuable parts of the ToC process as stakeholders get to hear each other’s understanding of the goals, challenges, and what is needed in the program’s context for it to succeed. Like indicators, it is very important to list and understand these key assumptions as they relate to each individual outcome. 

Example of a Theory of Change (ToC) tracking table


Why should I use a Theory of Change (ToC)? 

ToC can help you to understand why change is expected to occur by altering the way you think about your program from what you are doing to what you want to achieve. The tool can also help programs respond to changes and guide stakeholder engagement, communication, and support decision-making.

By defining long-term goals and then mapping backwards to identify the necessary preconditions, ToC can provide the basis for arguing that a program is making a difference whilst identifying weaknesses in the argument and providing the opportunity to make changes.

It is well suited to complex programs that are influenced by multiple systems and actors due to its fluid, yet comprehensive approach. 


Why is ToC useful for evaluation? 

  • ToC essentially provides a diagram that can be tested by analyzing relevant indicators to see if the program actually contributed to the intended change 

  • ToC builds a solid foundation for an impact evaluation by providing a reporting framework and identifying what data needs to be gathered to test the theory 

  • A well-developed ToC can help to create better key evaluation questions, identify indicators for monitoring change, and identify gaps in available data 

  • This can help to focus new data collection on areas where there are gaps leading to more effective use of evaluation resources 

  • By identifying the why and how of a program, ToC can help to reflect on what has worked or not worked to understand the past, identify opportunities for learning, and plan for the future 

However, developing a ToC can be challenging as it involves facilitating collaboration with all key stakeholders, synthesizing a range of views and information sources, as well as obtaining agreement and buy-in from stakeholders. The collaborative aspect of developing a ToC is crucial and will be further discussed in part 2 of our series on ToC. 


How has Theory of Change (ToC) helped me in my evaluations? 

ToC has played a BIG part in my role as an evaluator so far in helping me to deliver meaningful results to stakeholders. It has also helped me to build an understanding of change processes and increase the capacities of key stakeholders to do the same. 

Collaboratively developing a ToC before the start of a program has supported my clients in identifying how and why the program is expected to lead to the desired change. It has helped my clients to become clearer in their own understanding, highlight gaps they may not have considered, and document their own underlying assumptions. ToC helped my clients to better understand risk and opportunity within their program’s context to ensure success.  

Collaboratively developing a ToC after a program has been launched for evaluation purposes has also benefitted my clients by helping them to understand why they made certain decisions and why they collaborated with certain actors. As well as providing me with a clear evaluation framework, ToC has helped me to provide actionable recommendations and results. Using ToC as an evaluation tool has also helped my clients to learn more about the benefits of having a clear ToC from the program’s start to help them make decisions and guide their program through challenges. 


Have you worked with ToC before, or have questions? Comment on this article or connect with us on LinkedIn or Twitter! 


Sign up for our newsletter

We’ll let you know about our new content, and curate the best new evaluation resources from around the web!


We respect your privacy.

Thank you!


 

Written by cplysy · Categorized: evalacademy

Jun 10 2022

Differences between Theory of Change, Log Frames, Results Frameworks and Logic Models – what are they and when to use them

In some of our previous articles, we’ve introduced and explained how to use different evaluation planning tools for identifying and assessing outcomes. You might have noticed how there seem to be quite a few tools doing very similar things. It’s no surprise that these tools can sometimes be confused, and the lines can be blurred around when and how to use them.

To help you on your way to deciding which tool is best suited to your evaluation, we’ve selected a few of our favourites to compare and contrast. These tools include: 

  • Theory of Change (ToC) 

  • Log Frames 

  • Results Frameworks 

  • Logic Models 

There are some common points across all of the available tools: 

  • All tools provide transparency and a visual explanation for why your program is expected to contribute to change 

  • All tools can help to track progress towards a specific objective 

  • All tools can be used at both the planning and evaluation stages of a program 

  • All tools are living documents and should be reviewed throughout the program lifespan 

  • All tools are time-consuming to develop, but to differing extents; they all require some reflexivity and strategic thinking to develop 


Theory of Change (ToC)

What 

  • Explains how a set of activities will solve a problem through a diagram often made up of boxes and arrows 

  • Goes into more detail by explaining the why also known as the “casual logic;” i.e., why one step is expected to lead to the next

Why 

  • Explanatory and best suited to complex programs that are influenced by multiple systems 

  • By defining long-term goals and then mapping backwards to identify necessary preconditions, ToC can provide the basis for arguing that a program is making a difference whilst identifying weaknesses in the argument and providing the opportunity to make changes 

When 

  • Can be used to both design and evaluate programs 

  • Can be developed at any stage of an intervention 

  • ToCs are living documents and should be flexible to the program’s needs and any changes happening on the ground

Strengths 

  • ToCs capture unintended and unexpected results 

  • Provides a reporting framework and identifies what data need to be gathered to test the theory 

Weaknesses 

  • Can be challenging and time-consuming as it involves facilitating collaboration with all key stakeholders, synthesizing a range of views and information sources, as well as obtaining agreement and buy-in from stakeholders

Log Frames

What 

  • Focused on how you will get to your program’s goal 

  • Usually presented as a matrix which structures the main activities in a program, highlights the logical connections between them, and identifies what these activities are expected to achieve 

Why 

  • Descriptive and better placed for small to medium sized projects

  • Log Frames help you to think about the relationships between available resources, planned activities, and the desired changes or results

When 

  • Most Log Frames are developed during program design and are updated throughout the program’s life span 

  • Like ToCs, Log Frames are not set in stone and should be flexible to the program’s needs

Strengths 

  • It ensures objectives are clear and measurable 

  • It ensures concrete evidence for a program’s achievement is collected 

  • Because risks and assumptions are made explicit, problems can be analyzed systematically 

Weaknesses 

  • It is a “one size fits all approach” which does not always capture the complexity and context of a program 

  • Don’t easily capture the how and why in the same way a ToC does

Results Frameworks

What 

  • Often in the form of a matrix that links activities with outcomes and results that directly relate to the objectives

  • Captures the essential steps of the logical and expected cause and effect relationship within a program

Why 

  • Focus on explaining the program’s results

  • It helps achieve strategic objectives i.e., the ultimate driver of the program by showing where resources could be best leveraged

When 

  • Useful as part of a strategic planning process

  • Is a living management document to support consensus, guide course correction, and serve as an accountability framework for evaluation

Strengths 

  • Helps identify and focus on specific, high leverage outcomes

  • Helps establish an evidence-based approach to monitoring and evaluation

  • Helps measure progress towards strategic objectives

Weaknesses 

  • The effects of interventions can be difficult to fully measure as unintended consequences and external influences are not captured. This can lead to a risk of tunnel vision

Logic Models

What 

  • Usually presented in a flow chart (not a matrix)

  • Logic models visually summarise how a program is expected to work by listing: what resources will be used, what activities will be completed, and how the activities will lead to outcomes

Why 

  • Logic models reveal intention, assumptions, and rationale behind a program

  • Logic models are useful to support stakeholders to think through and understand why a program is expected to lead to change

When 

  • In the planning phase, logic models can help to shape program strategies, set priorities, and illustrate approaches to stakeholders

  • During program implementation, logic models can support accountability

Strengths 

  • Builds a common understanding of goals, processes, and expectations for resources 

  • Can help to explain the need for a program to the community, organization, or funder 

  • Known for their easy-to-use format 

Weaknesses 

  • Don’t capture unintended or unexpected results

  • Don’t capture causality

  • While some logic models capture contextual factors and assumptions, they are often high-level and don’t look at each specific step within the change process


Theory of Change (ToC)

Main Components:

  • Activities 

  • Outputs 

  • Outcomes 

  • Impacts 

  • Purpose statement 

  • Impact pathways 

  • Description of the causal logic 

  • Description of the key actors 

  • Description of the indicators 

  • Assumptions (the factors outside of the program’s control that are necessary to ensure the program’s success) 

Description of alternative explanations and external factors (i.e., the different ways that may lead to change that are not related to your program) are often included in the narrative. 

Log Frames

Main Components:  

  • Main Goal  

  • Outputs 

  • Outcomes  

  • Activities  

Log Frames may also include indicators of how you will measure change and risks or assumptions underlying the change 

Results Frameworks

Main Components:  

  • Activities 

  • Outcomes (intermediate and longer term) 

  • Impacts (longer term) 

  • Outputs 

  • Indicators 

  • Critical assumptions and risks that must be in place for the intervention to be successful 

Results frameworks are often accompanied by a monitoring plan which includes baseline values and targets for expected outcomes and specifies the measures of achievement 

Logic Models

Main Components:  

  • Goals 

  • Inputs 

  • Activities 

  • Audience 

  • Outputs 

  • Outcomes 


Have you worked with any of these models before, or do you notice one that’s missing from our list? Comment on this article or connect with us on LinkedIn or Twitter! 


Sign up for our newsletter

We’ll let you know about our new content, and curate the best new evaluation resources from around the web!


We respect your privacy.

Thank you!


Sources: 

  • Belcher, B., Davel, R., & Claus, R. (2020) “A refined method for theory-based evaluation of the societal impacts of research”.  MethodsX. 7(1). Available from: https://www.sciencedirect.com/science/article/pii/S221501612030008X  

  • Hearn, S. (2009) “Outcome Mapping: Monitoring and Evaluating Policy Influencing” Presentation for Overseas Development Institute. Available from: https://www.slideshare.net/sihearn/om-for-policy-influencing  

  • Mitroff, I., & Bonoma, T. V. (1978). Psychological assumptions, experimentation, and real-world problems: A critique and an alternate approach to evaluation. Evaluation Quarterly, 2(2), 235–260. Available from: https://doi.org/10.1177/0193841X7800200204  

  • Nkwake, A.M. (2012) Working with Assumptions in International Development Program Evaluation: With a Foreword by Michael Bamberger. Springer Science & Business Media: Berlin, Germany.  

  • Stein, D., & Valters, C. (2012) “Understanding Theory of Change in International Development”. Justice and Security Research Programme at the London School of Economics and Political Science. Available from: http://eprints.lse.ac.uk/56359/1/JSRP_Paper1_Understanding_theory_of_change_in_international_development_Stein_Valters_2012.pdf  

  • Jones, N. D., Azzam, T., Wanzer, D. L., Skousen, D., Knight, C., & Sabarre, N. (2019). Enhancing the Effectiveness of Logic Models. American Journal of Evaluation, 1098214018824417. https://doi.org/10.1177/1098214018824417 

 

Written by cplysy · Categorized: evalacademy

Jun 10 2022

What program managers need to share with their new evaluators

Evaluators – this one’s not for you, but maybe you can share it with some clients! 

As a program manager or project lead, you may find yourself needing an evaluator. How do you get that evaluator oriented to the project? What information do they need? Let’s walk through it. 

Hopefully your new evaluator has a few questions to kick things off. Usually things like “What is the purpose for needing an evaluation? How will the information be used? What do you need answers to?” You might even sit down for an engagement meeting to answer most of those questions. This is my job. I do this on repeat. And yet no matter how I try to harvest all the important information from the key stakeholder group, I still find that I’m often missing core pieces of information when I set out to develop my evaluation plan. Key information tends to trickle out in bits and pieces over the initial weeks or even months. So, I thought it might be helpful to compile a list of what makes the new evaluator orientation and planning process as efficient and effective as possible. 

Here’s a checklist of things every program manager should share with (or tell) their evaluator:


  1. Describe the program.

    This sounds obvious, but you’d be surprised how often the people that hire me don’t start with a clear overview of what the program is! Be prepared to describe, from start to finish, what it is you do or what service you offer and who you serve. 

    TIP: Share any documentation you have, including project proposals, grant applications, or work plans. Include a description of who’s who in the zoo: what are the key roles.

  2. Has the program been evaluated before?

    If yes, share previous: 

    • Evaluation plans 

    • Evaluation reports 

    • Data (if you want year-to-year comparison, for example) 

    • Logic models or Theories of Change 

    TIP: It can be helpful to share what you liked or didn’t like about your previous evaluations and reports – what missed the mark? What was really helpful? What do you want this new evaluator to do differently or the same?

  3. A description of current data collection processes.

    Ideally this would be documented but I’ve yet to find a program that has this (and have found myself creating process maps myself for the complicated programs). In the very least, be prepared to share a verbal description of how things work. For example: 

    “When a participant expresses interest in our program, they sign a consent form (here’s a copy) and fill out a baseline survey (here it is). Then they attend a weekly workshop for 2 months. We keep an attendance list (like this). Then there is a post workshop session survey (here it is).” 

    Note that sharing the current data collection tools is critical. 

    TIP: It can be helpful to share what’s working or not working about data collection. For example: “We really struggle to get a decent response rate on our post workshop survey.” Or “We collect those attendance sheets, but we have no use for them, they just get recycled!” Even detailed insights can help “On our survey, the age ranges we offer aren’t a great reflection of our clients.” or high-level reflections “We have no ability to match data between two very important data sets”. 

  4. Funder and reporting requirements and timelines.

    Do you have mandatory reports due? Even if you don’t intend to have the evaluator write them for you, be clear about who is contributing to what and what the timelines are. Often the evaluator can time data collection to be ready for your annual, quarterly or interim reports. They can certainly help you out by ensuring that key metrics or key success factors are captured and ready for you. So even if writing the report(s) is beyond the scope of the role you want the evaluator to play, it can be helpful to share these anyway. 

    TIP: Documenting who is responsible for what in mandatory reporting is very helpful. You could ask your evaluator to contribute to certain sections, contribute data, or plan to use the evaluation report as an appendix to the required reporting template.

  5. Who are the decision-makers?

    Is there a program steering committee or advisory group? Perhaps there is even an evaluation advisory committee that is disbanded or defunct? Share these details with your evaluator so (a) they can ensure their reporting is prepared for the appropriate audience(s) and (b) they know who to go to for important decisions – like signing off on the evaluation plan.

    TIP: I’ve found it to be effective when a program has a core group of no more than 5 people who provide input into an evaluation. Five seems to be able capture enough diverse perspectives, but with more than this it gets hard to convene for key discussions (and timelines get pushed out).

BONUS TIP: Be prepared to answer key evaluator questions. 

I get that it can be difficult to come up with great insights on-the-spot – so don’t! If you have an evaluation meeting coming up, try to carve out even 15 – 30 minutes in your schedule to prepare for it.

Be prepared to describe why you want this evaluation, what questions you have about your program and how you plan to use the information.

Consider reviewing your program material – work plans, proposals, previous evaluation reports, even meeting minutes.  Were there key questions posed by program stakeholders or partners? Take a few minutes to reflect: If I could know anything, what would I want to know about this program? It could be about the way it’s run, about the impact it has, or about the effectiveness or efficiency of it. 


Evaluators want to provide you with information that is valuable, relevant, and actionable! Setting them up with the right information makes it more likely that you will find value in the role. 

Do you need help with an evaluation? Reach out to one of our Evaluation Coaches to get started. Or perhaps you’re looking to commission an evaluation. We’ve got some great tools to help you there, too! Take a look at our recommendations for what to include in your Evaluation RFP and a checklist to make sure you’ve got it all covered. 


Sign up for our newsletter

We’ll let you know about our new content, and curate the best new evaluation resources from around the web!


We respect your privacy.

Thank you!

 

Written by cplysy · Categorized: evalacademy

Apr 27 2022

What’s the difference between goal and objective? The most confusing evaluation jargon

I’ve had many experiences collaborating with other evaluators, colleagues, or program managers starting a new evaluation. Questions fly like “Which evaluation framework will we use?”, “What will our approach be?”, “What is the evaluation plan?”. And often, down in the weeds, we question things like “What are the goals of this project?”, “Is the project achieving objectives?”, “Are there project aims?”. 

That’s a lot of jargon! What do all these terms mean? 

My personal solution to this problem is to be very clear upfront: I am terminology agnostic. I hold no allegiance to any specific term. 


Some people will argue to the death that a goal is a broader concept than an aim, but I guarantee you will meet someone who believes equally as strongly that an aim is overarching with goals nested underneath. 

You will meet people who believe adamantly that an evaluation framework is a validated and tested methodology that guides your approach to a specific evaluation. But you’ll meet others who call their own evaluation plans a framework. Likely in reading this you’ve already identified what side of these debates you fall on.  

For me, no one is wrong. I chose long ago not to have etched-in-stone definitions, but to be fluid to the clients I work with and other evaluators. 

Being agnostic does not mean these definitions don’t matter. They do! It just means I take time to ensure I am on the same page as everyone else in the room so that when we say “goal” we all know what that means and when we say “plan” we all know what that is (and what document to open!).  

Being agnostic also doesn’t mean I don’t have my own preferences, and if given the opportunity I will certainly use the language that makes the most sense to me.

Here are my back-pocket definitions for some of this confusing evaluation language. 


Framework. Plan. Approach. Model.

  • Evaluation Framework

    • This is usually a published methodology – things like RE-AIM, Kirkpatrick, or PRECEDE-PROCEED. 

  • Evaluation Plan

    • This is the document I create, that plans out the evaluation for a given project or program. Check out our template for what goes into creating the plan. 

  • Evaluation approach

    • This is more around guiding principles. For me, I’ll describe approaches as “being utilization-focused”, or “participatory”. You might include things like “summative” or “formative”. 

  • Model

    • For me, most of the language I use is covered with framework, plan, and approach so I don’t tend to use model, but I would align it most closely with Framework. In fact, PRECEDE-PROCEED is actually called a “model”. 

So, when a program manager asks you to “create an evaluation framework”, the first step is confirming if what they mean is for you to create and document a plan that may use a specific evaluative framework. 

When asked which model you may apply to evaluation, you may want to describe both an evaluation approach (e.g., developmental, utilization-focused) and which framework (if any) will be used and confirm that this answers the question. 

The key is ensuring everyone is on the same page. Confirmation without assumption is critical. 

Goal. Aim. Objective. Intended impact. Outcome. Target. Benchmark.

A second grey area, and perhaps an even more common one, is differentiating goals from aims from objectives, etc. For me, these terms are much more interchangeable than the Framework/Plan discussion. Again, the key here is being clear about what terms you use, what they mean and, if relevant, how they link together. 

You may agree that there is some grey area here, but that all the terms I listed are not synonymous. I agree. I do think that targets and benchmarks stand a little apart.

Targets and benchmarks have quantitative metrics associated with them: e.g., “We aim to serve 700 clients in this fiscal year.” Still valid to evaluate, but perhaps not the same as an outcome statement: e.g., “Participants will have increased confidence in accessing support services.”

You could also likely group some of these terms: goal/aim/objective and impact/outcome, for example. The common thread is that in some way each of these terms describes what the program does. 

When I start evaluation planning, I’ll look for all of these terms hiding or disguised in: 

  • Previous evaluation reports and recommendations 

  • Strategic plans 

  • Operational plans 

  • Mission and vision statements 

  • Core values 

  • Guiding principles 

  • Funder requirements/mandates 

I try to align my language to what the program stakeholders use. My goal is to determine what the program is trying to achieve so that we can ask key evaluation questions that drive toward those goals (or outcomes, or objectives, or….). 


What are your preferred terms and what do they mean to you?

At Eval Academy we’re working hard on an evaluation dictionary to help add some clarity to confusing evaluation jargon.

What terms have we missed? Comment on this article and let me know!


Sign up for our newsletter

We’ll let you know about our new content, and curate the best new evaluation resources from around the web!


We respect your privacy.

Thank you!

 

Written by cplysy · Categorized: evalacademy

Apr 27 2022

3 Easy Ways to Quantify Your Qualitative Data

You’ve completed your qualitative data collection and you’re writing up your report. You step back and look at All. The. Text.  If only you had some quantitative data to include in a chart, or some numbers to report!

We’ve previously written about how to quantify qualitative data and offered some definitions for things like “few” or “some” or “many”. In this article, we’ll give you even more tips for quantifying qualitative data.


  1. Frequencies

    Maybe this is obvious but there’s no rule against counting codes! I will say, though, that this approach may lend itself to data that are addressing a very specific question. “Describe a time you experienced discriminatory behaviour” may be more difficult to quantify than “What is one improvement you’d make to your workplace?”

    I recently coded some data where participants were asked “What was the most helpful part of the program?”

    There were thousands of typed-in responses. I was able to read through it and quickly come up with a list. I could then report that, for example, 40% of respondents thought that flexible scheduling was most helpful, while 30% thought support from the staff was the most helpful.

    Usually there will be an “other” category and that’s okay – just describe what the “other” means: “5% of respondents reported other things, for example, they liked the website, or they liked the snacks at reception.”

    Now you have some numbers to use your data viz skills on!

  2. Demographic descriptions

    You’ve completed all those interviews and are reporting the findings through themes and quotes, but have you explored if there are differences between who said what?

    Maybe there are gender differences in how questions were answered, or role/title differences?

    You could include charts or numbers by reporting something like: “80% of frontline staff but only 20% of managers described a time when they had encountered workplace conflict” or “70% of respondents who lived in a specific geographic region reported more stories of difficulty accessing care compared to only 10% who lived in another region.”

    As always, be cautious about drawing any causation or firm correlations.

  3. Play with data presentation

    Ok, this one may not be quantifying the data per se, but it can break up those text-heavy report pages.

    Word clouds seem to be an oft-used example for visualizing qualitative data. I’ll go out on a limb and say I’m not a fan. I can’t think of a time when I’d gained any meaningful insight from a world cloud.

    Some other options include:

Tables

  • Create a two-column table with your core theme on the left and example quotes on the right.

Journey maps

  • are your participants describing an experience or a narrative? Perhaps you can map it out in stages and include descriptions or quotes at each stage. I’ve found journey maps to be very impactful in reports. You have the option of using one participant as a case study or describing a “typical” experience.

(example maps are free slide templates from SketchBubble)

Images

  • Sometimes something as simple as adding an image, icon or photo along with a quote can make it stand out more and break up the text. Maybe even try a stand-out border.

Icon array

  • If you’ve been able to do any frequencies or counts of your qualitative data, you can certainly include any number of charts to depict your data. An icon array may be particularly impactful to give a visual to how many participants said whatever it is you’re highlighting.


What tips have we missed? Share them by commenting on this article!


Sign up for our newsletter

We’ll let you know about our new content, and curate the best new evaluation resources from around the web!


We respect your privacy.

Thank you!

 

Written by cplysy · Categorized: evalacademy

  • « Go to Previous Page
  • Go to page 1
  • Interim pages omitted …
  • Go to page 20
  • Go to page 21
  • Go to page 22
  • Go to page 23
  • Go to page 24
  • Interim pages omitted …
  • Go to page 43
  • Go to Next Page »

Footer

Follow our Work

The easiest way to stay connected to our work is to join our newsletter. You’ll get updates on projects, learn about new events, and hear stories from those evaluators whom the field continues to actively exclude and erase.

Get Updates

Want to take further action or join a pod? Click here to learn more.

Copyright © 2026 · The May 13 Group · Log in

en English
af Afrikaanssq Shqipam አማርኛar العربيةhy Հայերենaz Azərbaycan dilieu Euskarabe Беларуская моваbn বাংলাbs Bosanskibg Българскиca Catalàceb Cebuanony Chichewazh-CN 简体中文zh-TW 繁體中文co Corsuhr Hrvatskics Čeština‎da Dansknl Nederlandsen Englisheo Esperantoet Eestitl Filipinofi Suomifr Françaisfy Fryskgl Galegoka ქართულიde Deutschel Ελληνικάgu ગુજરાતીht Kreyol ayisyenha Harshen Hausahaw Ōlelo Hawaiʻiiw עִבְרִיתhi हिन्दीhmn Hmonghu Magyaris Íslenskaig Igboid Bahasa Indonesiaga Gaeilgeit Italianoja 日本語jw Basa Jawakn ಕನ್ನಡkk Қазақ тіліkm ភាសាខ្មែរko 한국어ku كوردی‎ky Кыргызчаlo ພາສາລາວla Latinlv Latviešu valodalt Lietuvių kalbalb Lëtzebuergeschmk Македонски јазикmg Malagasyms Bahasa Melayuml മലയാളംmt Maltesemi Te Reo Māorimr मराठीmn Монголmy ဗမာစာne नेपालीno Norsk bokmålps پښتوfa فارسیpl Polskipt Portuguêspa ਪੰਜਾਬੀro Românăru Русскийsm Samoangd Gàidhligsr Српски језикst Sesothosn Shonasd سنڌيsi සිංහලsk Slovenčinasl Slovenščinaso Afsoomaalies Españolsu Basa Sundasw Kiswahilisv Svenskatg Тоҷикӣta தமிழ்te తెలుగుth ไทยtr Türkçeuk Українськаur اردوuz O‘zbekchavi Tiếng Việtcy Cymraegxh isiXhosayi יידישyo Yorùbázu Zulu