allblogs

Jan 28 2025

You should build an audience, before you write your report.

Time for another report design mindset change.

At the most basic level successful communication requires a sender, a message, and a receiver. You, as the report designer, are the sender. The report carries the message. So who is the receiver?

For a lot of organizations the receiver is hypothetical. At least for those wishing to reach outside of their immediate stakeholder groups towards a broader audience. But even when defined, this broader audience is often just a hodgepodge of different personas, not actual people.

As any person with an online business will tell you, reaching an audience isn’t as simple as posting to social media. To be successful, you need to put in work to actually build your audience.

And if you actually want your report to successfully reach a broader audience, you need to do the same.

Email beats social media.

When organizations do put in effort to build an audience, they usually start with social media. Like us on Facebook. Connect with us on LinkedIn. Follow us on Twitter.

But social media is unreliable for a few reasons.

1. Very few of your posts will likely be seen by your followers.

This has become increasingly true over time as what we see is guided by algorithms. These companies can control these algorithms and even shadow ban your content without your knowledge.

2. Most social media companies are for profit.

They prioritize money making over information sharing. This has become worse over the years, forcing many to feel the need to pay for attention even if they have established social media audiences.

3. Social media audiences do not carry over to other platforms.

So if you build a following on TikTok, and TikTok goes down, your following is gone. If your account gets hacked, and then suspended by the social media company, your following is gone. If your social media channel of choice gets purchased by an egomaniac who turns it into some kind of right wing propaganda tool, your following is gone.

But if you start building an email list, a higher percentage of your audience will see your messages when you send them. You can choose your email platform. You can also change your email platform and bring your audience with you.

Email beats search.

Search engines like Google, can deliver report audiences to your report. But it’s also pretty unreliable.

1. Search engine algorithms are a constantly changing blackbox.

Small changes can lead to a large disruption in your ability to reach audiences.

2. Most search engines are for profit.

Just like social media, they put advertising ahead of information sharing.

3. AI is completely changing the way search works.

AI search interprets your reporting in ways that could easily distort your messages. It can even outright lie about your findings or suggest false sources.

Email gives you the ability to reach an audience with your messaging in a way that gives you some level of control otherwise impossible through search.

How to build an audience for your reports.

Unfortunately, building an audience isn’t exactly easy. It takes work, usually through the implementation of a content strategy. It also takes some preparedness, as you should work towards building your audience well before sharing your report.

There are multiple ways to start collecting emails for the people with an interest in your reporting. Here are a few simple strategies you can use.

1. Setup a landing page with a direct invitation for those interested in following your reporting.

This landing page should ask for their contact information (email address).

Tip: this is an opportunity for audience input and feedback through the fields on the form.

2. Hold webinars.

Give presentations on your models, methods, or preliminary findings. People are used to registering for webinars and are willing to provide their email addresses. They are also often willing to opt-in to future updates on your reporting.

Tip: this is also an opportunity for direct audience input and feedback.

3. Write monthly updates in the form of blog posts.

This still requires a landing page to get email addresses but consistent updates can help you to cultivate and serve your reporting audience.

Tip: and yes, this is also an opportunity for audience input and feedback through comments.

Final thoughts.

I know very few evaluation teams that are actually putting in the work to build their report audiences. In larger organizations they might assume their comms team will already have access to these audiences. But this is a pretty big assumption.

The audiences organizations build through their communications strategy are not necessarily the same ones you would want to reach with your reports. Comms teams also often rely heavily on social media.

Are you building the audience for your reporting? If yes, please share in the comments and let us know how.

Jan 27 2025

Why to Stop Saying “Mental Model”

Image with the text: Test how inclusive you’re communicating evaluation learning by asking:
— Who should we build culture with?
— How will my language be understood, and by whom?
— How can we increase learning and understanding and reduce barriers?

Part of our role as evaluators is to communicate learnings so that people can easily understand and translate those learnings for their own work — in many ways, we are failing in this role. As I was working on the Equitable Communications Guide (published last year), I became more aware of the ways we as evaluators are often ableist and exclusionary. It’s human nature to categorize, even other humans, creating shortcuts to determine what — or who — is familiar and safe to be around. But this natural behavior creates “in groups” and “out groups,” and leaves us unaware of the realities outside our own groups. I’ve come to believe the term “mental model” is another way we perpetuate an “in group,” and creates exclusion when we have the opportunity to be more inclusive.

I’ve come to believe the term “mental model” is another way we perpetuate an “in group,” and creates exclusion when we have the opportunity to be more inclusive.

This exclusion is not intentional of course. “Mental model” is a term that has come to mean “the internal representations of external reality that individuals use to understand, interpret, and navigate the world around them. They consist of a set of beliefs, generalizations, and assumptions that make up our worldview” [1]. It was coined by psychologist Kenneth Craik in 1943 [2], and gained traction in the 1980s when Johnson-Laird connected the idea to semantic reasoning in psychology. The term became applicable beyond psychology when Peter Senge began using the concept in the business and management context in the 1990s [3].

I’m not sure when “mental model” entered the evaluation space, but I’ve seen increasing use of it in our field, including in publications and by influential thought leaders in our field. I remember attending an AEA session in 2023 where a panelist shared her excitement upon discovering the term. “Mental model” revealed a nuance in meaning for her that other terms weren’t capturing — it was the best way to describe what she wanted to say.

Language does have an important role in the micro cultures we create, such as within the evaluation field. Edgar Schein’s framework defines the forces that govern workplace culture. He describes shared language, including stories, jargon, and sayings, as “verbal artifacts” of a culture that supports belonging and camaraderie among colleagues [4].

I don’t want to discount the benefits the term “mental model” has brought to the people who use it. But I do want to examine what we lose by using the term.

The term “mental model” creates an in group of people who know what this term means.

The term “mental model” creates an in group of people who know what this term means. And the people who know what it means are typically advanced degree professionals in evaluation and academia. For those who use it in the evaluation space, the term builds culture, connection, and a feeling of belonging with others who are interested in the same ideas. But it also creates outsiders, who are often the people who have the most to learn from our evaluative work.

It’s true that we use many terms that nonprofessional evaluators don’t understand. I would argue that many of these are necessary, but that “mental model” is not one of those. For example, some people outside of evaluation and research haven’t heard of a “theory if change,” don’t understand what a “sample” means, and could not explain data “coding.” But these words are all industry terms that name something specific and exist to create necessary language for something we do that is unique to our industry. The difference between our industry-specific terms with the term “mental model” is that it doesn’t name something new, it adds nuance to commonly used terms that are already widely understood. Nuance that may not be necessary for an audience who can benefit from our knowledge.

We are excluding people who are outside of the evaluation field who can benefit from our evaluation knowledge.

As a comparison — dentists diagnose cavities. As patients, we understand the important industry term “cavity,” but we rarely understand the nuance that is less relevant to our daily lives, like the type of cavity (smooth, fissure, or root) or what class of cavity (there are six). These classifications are vital to a dentist determining the right treatment, yet a good dentist will use “cavity” when speaking with patients and clearly describe the problem without using jargon.

When we use “mental model,” we are saying “Class VI Fissure” without using clear language, excluding people who are outside of the evaluation field who can benefit from our evaluation knowledge. We may lose nuance, but increase understanding when we choose language that more people are familiar with. “Worldview” and “perspective” are excellent alternatives to “mental model” that have broader use and acceptance.

To communicate equitably, we should strive for inclusive language with the most direct route to comprehension.

To communicate equitably, we should strive for inclusive language with the most direct route to comprehension. We should seek to build community outside of our evaluation in-group, and invite more people to the knowledge we have the privilege of gaining as evaluators. To ensure we are inclusive, we should ask:

– Who should we build culture with?

– How will my language be understood, and by whom?

– How can we increase learning and understanding and reduce barriers?

Is using the term “mental model” worth excluding people, and excluding broader understanding of our learning and insight? In my practice, the answer is no.

Why to Stop Saying “Mental Model” was originally published in InnovationNetwork on Medium, where people are continuing the conversation by highlighting and responding to this story.

Jan 24 2025

Stop Creating Spork Reports

What is a spork report?

I want you to imagine visiting a nice little cafe for a light lunch. You decide to order a simple salad and a cup of soup. When the meal shows up, what utensils will you use to eat your food?

Easy, right? A fork for your salad and a spoon for your soup.

So why not a spork?

For the salad, a regular fork will work better than a spork. And for the soup, a regular spoon will work better than a spork. And for the restaurant, it’s not really any harder to offer two utensils, especially if they have a dishwasher.

This is why, in a world of forks and spoons, sporks are just a novelty.

The problem with solving two problems with one product.

The evaluation reports we create are often designed to satisfy two goals.

The first goal is that we need something to help us defend our work. Being a professional evaluator requires that you systematically document your theory, questions, methods, analyses, and recommendations.

The second goal is that we need to tailor our findings so that we can present them to a variety of audiences. Lots of people don’t care about all the things, they just want something specific that meets their particular needs.

A lot of organizations start by writing a technical report. Then they try to make that technical report meet the needs of their particular audience.

It becomes a catch 22. To defend your work, you need to share all the details. But to present your work to many different audiences, you need to cut out some of that important detail.

So the organization creates a spork report.

It doesn’t defend as well as a technical report. And it doesn’t meet the needs of your audience as well as a presentation report.

Different Goals, Different Products.

Out on the web you will find some pretty common, but misguided, evaluation report design advice. It starts by telling you to create a technical report. Then it tells you to tailor that report for your audience.

Don’t do it. This just leads to frustration, and a revision loop.

It leads to a spork report.

Instead, write your technical report. It should be as long as you need it to be so that you can defend your work. It may have some jargon and feel academic. It may never be read by anyone. AND THAT’S OKAY. First and foremost, it needs to exist.

Then, write your presentation reports. These are the ones you adapt to meet the needs of your audiences. They come in all shapes and sizes. They can refer to the technical report, but should be stand alone presentations designed for specific audience members.

Jan 19 2025

Desafíos y Renovación: El Problema de las ONG Internacionales

Resumen General: El Problema de las ONGI de Deborah Doane examina los desafíos y críticas que enfrentan las Organizaciones No Gubernamentales Internacionales (ONGI) en el contexto actual. El objetivo principal del libro es analizar cómo estas organizaciones pueden superar sus problemas estructurales y renovar su enfoque para ser más efectivas y equitativas.

Análisis de Capítulos: Cada capítulo del libro aborda diferentes aspectos de las ONGI, desde su historia y crecimiento hasta los problemas contemporáneos como el racismo estructural y la falta de representación local. Los puntos clave incluyen la necesidad de descentralizar el poder y los recursos hacia las organizaciones locales, y la importancia de la transparencia y la rendición de cuentas.

Innovación y Valor Añadido: Lo innovador de El Problema de las ONGI es su enfoque crítico y constructivo hacia las ONGI, proponiendo soluciones prácticas y realistas para los problemas identificados. El valor añadido del libro radica en su llamado a la acción para una reforma significativa en el sector.

Utilidad Práctica: La información del libro puede aplicarse en la vida diaria y en el trabajo, especialmente para aquellos involucrados en el sector del desarrollo internacional. Ofrece consejos prácticos sobre cómo las ONGI pueden mejorar su impacto y trabajar de manera más equitativa con las comunidades locales.

Críticas y Opiniones: El libro ha recibido críticas positivas por su análisis profundo y su enfoque en soluciones prácticas. Los lectores han valorado la honestidad y la claridad con la que se abordan los problemas del sector.

Comparación con Otras Obras: En comparación con otras obras de Deborah Doane, El Problema de las ONGI se destaca por su enfoque en la reforma estructural y su llamado a la acción. Se diferencia de otros libros sobre el mismo tema por su análisis detallado y sus propuestas concretas para el cambio.

Recomendaciones Personalizadas: Si te gusta El Problema de las ONGI, podrías estar interesado en libros como The End of Development de Andrew Brooks o Decolonizing Aid de Rosalind Eyben, que también abordan temas de poder y equidad en el desarrollo internacional.

Impacto Cultural y Social: El libro ha tenido un impacto significativo en el campo del desarrollo internacional, fomentando debates sobre la necesidad de reformar las ONGI y promover una mayor equidad y justicia en el sector.

Fuentes y Citas: Entre las principales fuentes utilizadas en el libro se encuentran estudios y reportes de organizaciones como Oxfam y Human Rights Watch. El libro ha sido citado en diversas investigaciones y artículos académicos sobre desarrollo internacional.

Pequeña Biografía: Deborah Doane es una experta en desarrollo internacional con más de 30 años de experiencia en el sector. Ha trabajado en diversas ONGI y ha sido una voz crítica en la necesidad de reformar el sector. Entre sus otras obras se encuentran Decolonizing Development (2018) y Power and Privilege in Aid (2020).

Temas y Mensajes Principales: Los temas principales incluyen el poder y privilegio dentro de las ONGI, la necesidad de un cambio estructural profundo y la importancia de la equidad y la justicia en el desarrollo internacional. Deborah Doane transmite el mensaje de que las ONGI deben reimaginar su papel y trabajar en colaboración con actores locales para ser verdaderamente efectivas.

Jan 18 2025

Claves para un Intercambio Efectivo de Conocimientos entre Ciencia y Política

Conectar ciencia y política es crucial para resultados sostenibles. En el artículo «Claves para un Intercambio Efectivo de Conocimientos entre Ciencia y Política» (14 de enero de 2025), se revisaron 56 estudios de caso empíricos para identificar facilitadores clave en esta interfaz.

Facilitadores del Intercambio de Conocimientos

Se identificaron condiciones habilitadoras en dimensiones organizacionales, individuales, financieras, materiales, prácticas, políticas y sociales. Los actores incluyen financiadores, intermediarios del conocimiento, investigadores y tomadores de decisiones.

Capacidad Organizacional: Requiere liderazgo fuerte y objetivos claros. Facilitar el alineamiento de objetivos, co-producir conocimientos y valorar el intercambio de conocimientos son esenciales.

Capacidad Individual: Necesita redes sociales fuertes y habilidades colaborativas. Construir relaciones, establecer confianza y adaptar la comunicación son claves.

Capacidad Financiera y Material: Es esencial asegurar financiamiento suficiente y flexible, y proveer recursos necesarios como objetos de frontera.

Capacidad Práctica: Ofrecer tiempo y espacio flexibles para el aprendizaje y crecimiento, realizar evaluaciones y compartir experiencias son facilitadores importantes.

Capacidad Política y Social: El apoyo político y el compromiso cara a cara son vitales. La comunicación regular y sostenida mantiene el compromiso y la confianza.

¿Los facilitadores identificados resuenan con tu experiencia? ¿Tienes otros para compartir?

Referencias

Kapoor, T., Cvitanovic, C., Klenk, K. y Nguyen, V. M. (2024). Taking knowledge exchange to practice: A scoping review of practical case studies to identify enablers of success in environmental management. Environmental Policy and Governance, 1-18. (DOI): https://doi.org/10.1002/eet.2128