• Skip to main content
  • Skip to footer
  • Home

The May 13 Group

the next day for evaluation

  • Get Involved
  • Our Work
  • About Us
You are here: Home / Archives for allblogs

allblogs

Feb 29 2024

Try This: Get Your Ideal Community Partners On Board

Try this activity and let me know how it goes for you. Engaging the right community partners to join your partnership is crucial to facilitating a process that’s mutually beneficial. As you plan your introductory partnership meeting, you need to figure out who to invite, and why. You see the value in working with these […]

The post Try This: Get Your Ideal Community Partners On Board appeared first on Nicole Clark Consulting.

Written by cplysy · Categorized: nicoleclark

Feb 27 2024

Evaluation Ethics Fails, 7 illustrations

I’m sick this week, so instead of writing something new, I thought I republish something valuable from 11 years ago. This cartoon was inspired through a conversation with my good friend Ann K. Emery.

This was back when Ann was a full time evaluator and before the launch of her super successful training academy and professional development business Depict Data Studio [affiliate link].

All of the comics shared in this post are based on true stories…which is probably why they hold up so well!

Low Response Rate

Evaluator: “Here are the survey results for your program. The results suggest that _____, but here’s a caveat – there was a low response rate, so we need to take these results with a grain of salt.”

Client: “No problem. Send me the link to the survey. I’ll take the survey a dozen times, and then the response rate will be higher.”

Evaluator: “Sorry, that’s not how it works.”

Client: “Then give me a paper copy of the survey and I’ll make photocopies until we have enough responses.”

Impossible response rate

Formatting Issue

Evaluator: “Here are the results from your program.”

Client: “Uh oh, the results don’t look good. The graphs aren’t going up. Can you re-format the graph to make sure all the bars are going upwards over time?”

Don't look at the axis

Defining Outlier

Evaluators: “Here are the results from your program.”

Client: “Those results aren’t accurate.”

Evaluators: “How so?”

Client: “The bad results are obviously outliers. You need to remove those people from the sample.”

Evaluators: “We define an outlier as (1.5 x the interquartile range) below quartile 1 or above quartile 3. Other evaluators define outliers as 3 standard deviations above or below the mean. We checked, and those people are not outliers. In fact, their experience in the program was pretty typical.”

Client: “Sorry, I didn’t realize it was such a hassle to fix. I didn’t mean to create more work for you. Just send me the Word version of your report and I’ll delete that section myself.”

Outlier is someone who doesn't like our program

Whoops Typo

Evaluator: “Here are the results.”

Client: “I know our program only had a 38% success rate, but can you type 83% in the report to our funders? 83% sounds better than 38%. If anybody notices, just say you accidentally made a typo.”

Report written on backwards day

Staying Funded

One more cartoon, just to put the fails in context…

You better show great results

Additional Cartoons

I’m curious, what ethics fails have you witnessed? Anything you can share or would you need the blurry picture and garbled voice treatment too?

Update 1: Cherry-Picked Sample

Thanks to Maria Gajewski for the comment that inspired this cartoon!

My favorite is the cherry-picked sample. I was involved in a project where directors only wanted to survey students whose families were still involved in the program. It’s not too difficult to figure out families who had a bad experience would not remain with the program, but the directors just couldn’t seem to grasp this when I pointed it out!

Next time, send this cartoon to the directors 🙂

Cherry Picking Sample

 

Update 2: Consequences

Ok, here is another comment inspired cartoon (thanks bridgetjones52).  And a big thank you to everyone else for all the comments and shares!

When presenting results of evaluation, very very senior manager ‘informed’ us that there would be consequences!!!

wpid-Photo-Jun-26-2013-131-PM.jpg

Written by cplysy · Categorized: freshspectrum

Feb 26 2024

What Reparations Taught Me as an Evaluator

Over the past year, I’ve been diving deep into concepts around reparations — in a course I took about reparations for slavery in the US (check out my 5 Key Takeaways!), as well as in my dissertation work where I’m looking at reparations as a new way to frame international development and humanitarian aid.

In my work as an evaluator and researcher, I’ve been asking myself — what can we learn from reparations and anti-racism work that can shift our evaluation practice towards equity and justice? I’ve come up with 3 things we need to do better:

  1. Name and talk (and write) about race and coloniality. Too often, our work as evaluators does not meaningfully examine histories and contemporary contexts of slavery, White supremacy, and colonialism/imperialism. If we are going to contribute meaningfully to social change, we need to explore how these issues affect opportunities and outcomes for people of marginalized identities. We need to include this analysis in our presentation of findings. We need to frame our work as seeking to shift the Racial Contract.
  2. Critically examine the cultural, political, and economic contexts of our work. The current neoliberal capitalist system in the US and globally was built on the conquest and exploitation of Black and Brown folks. And it relies on continued injustice to survive. If we are to contribute to social change, as evaluators we need to interrogate the unjust systems — the power hierarchies, mental models, and relationships — in which the policies and practices we aim to shift are embedded.
  3. Problematize our own role in maintaining the status quo. We all live in a White supremacist, imperialist, capitalist society. Even if we actively work to dismantle it, our mere existence and participation in it upholds it. We are what Michael Rothberg calls ‘implicated subjects’.As evaluators, we need to continuously examine how our practices in our organizations, our work, and in relationship with communities may be contributing to injustice. We need to practice reflexivity to move towards a constructive complicity.
I illustrated 5 key takeaways from the course on reparations for slavery. Click here to read the full comic!

There are many other ways evaluators are examining their complicity in White supremacist systems. Leave a comment to share your thoughts, or follow our blog for more insights in how this shapes our evaluation work!

Kayla Boisvert is an artist and researcher for advocacy initiatives that promote social justice at Innovation Network. She lives in Western Massachusetts with her husband and silly dog-monster. She is often found running trails and climbing mountains where she lives and around the world.


What Reparations Taught Me as an Evaluator was originally published in InnovationNetwork on Medium, where people are continuing the conversation by highlighting and responding to this story.

Written by cplysy · Categorized: innovationnet

Feb 15 2024

Building Your Creativity Toolkit

“Creativity is not a talent. It is a way of operating”

These are words that came from comedy icon, John Cleese. Head to YouTube if you want to watch one of his talks on creativity, and the source of this quote.

The quote itself has really stuck with me over the years as I have attempted to bring more creativity to the research and evaluation worlds I work within. Especially when I work with academically trained individuals, who are brilliant but also lack the necessary creative training and experience required to thrive in a modern digital world.

Creativity is not one of those “you either have it or you don’t” kinds of things.

It’s a process.

And as a process, it’s something that can be taught, learned, and practiced.

Announcing my upcoming mini-workshop series.

Throughout 2024 I am going to hold a series of single session live virtual workshops (at least 6). Each of these little workshops is going to focus on a single practical method that once learned, can be added to your own personal creativity toolkit.

Here are the specific topics I have planned (for the first 6 sessions).

  • Single Panel Comics
  • Icon Arrays & Pictograms
  • Simple Maps
  • Timelines
  • Annotated Charts
  • Before & Afters

These sessions are not yet scheduled, but will be announced individually in the coming months.

There is only one way to enroll right now.

Want to be pre-registered for all 6 live sessions? Which will include lifetime recording access.

It’s simple. Register for next week’s Everyday Visual Reports workshop.

The most effective types of evaluation and research reports are short, clean, and visual (not long and complicated).

In this highly practical live course you’ll learn:

  • How to design effective short visual reports for use as fact sheets, executive summaries, case studies, method briefs, and more.
  • How to apply fundamental graphic design principles to give your reports that modern professional feel.
  • A process you can use that will benefit both the writing and design of your report.

Registration in this course includes:

  • Two live 90 minute sessions (on February 20 & 22 at 10 AM Eastern).
  • Forever access to the recordings.
  • Optional-to-attend after session Q&As (I’ll hang around for questions up to 30 minutes after each session).
  • One year of office hours with Chris Lysy (a $250 value).

*Bonus* Creativity Toolkit Series

  • Entry to 6 live single session mini-workshops to be offered throughout 2024. Dates/times TBD.
  • Topics Include: Single Panel Comics, Icon Arrays & Pictograms, Simple Maps, Timelines, Annotated Charts, Before & Afters.
  • Forever access to the recordings.

Written by cplysy · Categorized: freshspectrum

Feb 14 2024

Inspiration Week Highlights

Written by Rebecca Perlmutter, Innovation Network Senior Associate

Inspired by our friends at CEI, who begin every January with two weeks of independent study, we decided to pilot a similar week for the first of week of January. The purpose of the week was to seek inspiration and insight, and we intentionally did not create any rules or guidelines about how to use our time.

What did we do?

We’d like to share some of the resources related to learning, evaluation, and social change that resonated most with us during our week. We hope you also find some inspiration in these resources, or are motivated to look further into similar topics!

Two of us read books on emergent strategy by adrienne maree brown: Emergent Strategy and Holding Change.

  • Emergent Strategy: Shaping Change, Changing Worlds. This book is an introduction to emergent strategy, which is about “how we shape and generate complex systems and patterns through our own relatively simple interactions.” We found in it a helpful reminder that we are trying new things in order to create a better world, and trying new things requires that we slow down as we feel the change and learn what works for us.
  • Holding Change: The Way of Emergent Strategy Facilitation and Mediation. This book provides ideas for incorporating emergent strategy practices and principles in your facilitation and mediation practice and also includes essays from Black feminist facilitators. We are always seeking to deepen our wisdom as facilitators, and this book helped remind us of the importance of using facilitation to build relationships, help groups surface answers, and center each person’s humanity.

Two colleagues explored different ways to tell visual stories. One took a creative data visualization course with Gabrielle Merite (Gabrielle also offers some free resources on her website) and one took a course on non-fiction comics for writers.

  • The data viz course combines digital illustration and collage techniques to help you create engaging data visualizations.
  • The comics course demonstrates how to transform research, interviews, and articles into graphic storytelling.

We found the book Advocacy and Policy Change Evaluation by Annette L. Gardner and Claire D. Brindis to be a great way to refresh our understanding of the different approaches to the complex world of advocacy evaluation.

  • This book provides an overview of the concepts, designs, methods, and tools for conducting advocacy and policy change evaluations. It is informed by a large-scale survey of evaluators working in the field and includes case studies that provide concrete examples of advocacy and policy change evaluations.

So what did we learn?

This was a successful experiment for us! Beyond learning about the different topics in the books we read or courses we took, we had some broader reflections about what an inspiration week meant for us individually and as an organization. These reflections may be useful if you are considering holding your own inspiration week.

  • The openness of the week was empowering and enabled each of us to best choose what we needed for personal development and to prepare for the year ahead.
  • It was important to be intentional about how we used our time. We all tried to use the time for things we may not have space for during busier periods of the year. For a lot of us, this meant spending time with longer materials, like an entire book. At the same time, having a mix of learning options was key, which for us included videos and courses in addition to books.
  • Doing this right after our end of year break was the perfect way to ease into the new year, before client meetings and other commitments crowded our calendars.
  • Four days were both too slow and too fast! We realized it was important to create space to process what we were learning, rather than just moving on to consuming the next product.

Have you ever had an independent study or inspiration week at your organization? How did you spend your time and what did you learn? Let us know your thoughts, and any recommendations you might have for our approach, in the comments!


Inspiration Week Highlights was originally published in InnovationNetwork on Medium, where people are continuing the conversation by highlighting and responding to this story.

Written by cplysy · Categorized: innovationnet

  • « Go to Previous Page
  • Go to page 1
  • Interim pages omitted …
  • Go to page 58
  • Go to page 59
  • Go to page 60
  • Go to page 61
  • Go to page 62
  • Interim pages omitted …
  • Go to page 310
  • Go to Next Page »

Footer

Follow our Work

The easiest way to stay connected to our work is to join our newsletter. You’ll get updates on projects, learn about new events, and hear stories from those evaluators whom the field continues to actively exclude and erase.

Get Updates

Want to take further action or join a pod? Click here to learn more.

Copyright © 2026 · The May 13 Group · Log in

en English
af Afrikaanssq Shqipam አማርኛar العربيةhy Հայերենaz Azərbaycan dilieu Euskarabe Беларуская моваbn বাংলাbs Bosanskibg Българскиca Catalàceb Cebuanony Chichewazh-CN 简体中文zh-TW 繁體中文co Corsuhr Hrvatskics Čeština‎da Dansknl Nederlandsen Englisheo Esperantoet Eestitl Filipinofi Suomifr Françaisfy Fryskgl Galegoka ქართულიde Deutschel Ελληνικάgu ગુજરાતીht Kreyol ayisyenha Harshen Hausahaw Ōlelo Hawaiʻiiw עִבְרִיתhi हिन्दीhmn Hmonghu Magyaris Íslenskaig Igboid Bahasa Indonesiaga Gaeilgeit Italianoja 日本語jw Basa Jawakn ಕನ್ನಡkk Қазақ тіліkm ភាសាខ្មែរko 한국어ku كوردی‎ky Кыргызчаlo ພາສາລາວla Latinlv Latviešu valodalt Lietuvių kalbalb Lëtzebuergeschmk Македонски јазикmg Malagasyms Bahasa Melayuml മലയാളംmt Maltesemi Te Reo Māorimr मराठीmn Монголmy ဗမာစာne नेपालीno Norsk bokmålps پښتوfa فارسیpl Polskipt Portuguêspa ਪੰਜਾਬੀro Românăru Русскийsm Samoangd Gàidhligsr Српски језикst Sesothosn Shonasd سنڌيsi සිංහලsk Slovenčinasl Slovenščinaso Afsoomaalies Españolsu Basa Sundasw Kiswahilisv Svenskatg Тоҷикӣta தமிழ்te తెలుగుth ไทยtr Türkçeuk Українськаur اردوuz O‘zbekchavi Tiếng Việtcy Cymraegxh isiXhosayi יידישyo Yorùbázu Zulu