freshspectrum

Over the coming year I hope to spend a little time on this blog showcasing some of the interesting work created by members of my Information Design Academy.

In today’s post we’ll look at something created by Celestyna Galicki that we started calling a Shadow Logic Model. Most of the words (and the Shadow Logic Model design) are Celestyna’s. I only added a few quick observations (and the cartoons).

Who is Celestyna Galicki?

Celestyna is an evaluator currently working in an evaluation capacity building role in local government in New Zealand. You can learn more by visiting Celestyna’s LinkedIn profile.

So just what is a Shadow Logic Model? [Celestyna]

This term describes a situation when the programme’s official logic model does not work (outputs do not lead to outcomes) but there is an unofficial logic model that works and produces outcomes that are important and valuable for someone. “Someone” can be the intended beneficiaries of the official logic model, the funder of the programme, people implementing the programme, the local community etc.

We are not talking about corruption here – money is spent as intended and all funded activities take place. If a shadow logic model is in place, most people involved in the program, and sometimes also the funder, know about it – they want to continue the programme because they find the benefits of the shadow logic model important enough. However, funding structures, political considerations, public opinion and other factors require them to pretend that they implement the official logic model if they want to continue.

How do you discover a shadow logic model? [Celestyna]

There is palpable awkwardness at first when workshopping logic models and evaluation with teams that have a shadow logic model. Discovering the shadow logic model is one of the skills academia did not teach me and I had to develop it with practice.

The way I usually approach this as an evaluator is to acknowledge the “unintended positive outcomes” and their value and evidence alongside the intended outcomes.

One of Celestyna’s Shadow Logic Models [Celestyna]

A few quick observations from Chris

I love this concept and Celestyna’s design. Let’s take a second to break down the design.

• First off there is a simple official logic model. This logic model is then mirrored with an unofficial logic model.
• A volunteer story illustrated with a simple cartoon figure humanizes the program and really sets the model within the appropriate context. The shadow logic model could have existed without this piece but I don’t think it would be nearly as compelling.
• At the bottom there is a single sentence takeaway which does a good job of ending the design with a quick “so what?”

What value exists in finding and constructing shadow logic models? [Celestyna]

The outcomes of official and shadow logic models provide important insights into what is and is not considered valuable by those who make decisions.

Common shadow outcomes include:

• social contact and friendship for those lonely and isolated (as in the example in the illustration)
• relationship building / relationship repair between the community and authorities or funders, signalling that the authorities care about something and take it seriously
• outcomes valued by indigenous or migrant communities that do not fit into Western frameworks, e.g. mana enhancement
• the benefits of basic services such as free transport or free food provided as part of the programme
• improving the reputation of a place/neighbourhood etc.

So what do you think? Could you see yourself creating something like this for your own program?

Let us know in the comments

Do you know how to identify a UX Problem with your reports or communication designs?

It’s not all that hard.

But first, a story.

The door without the handle

I took this picture last week on my way back into the office after taking my dog for a walk. What you might not see at first is that the right door is missing a handle.

I know what you’re thinking, “well, just use the left door then.”

But the left door was locked. The left door is ALWAYS locked. The right door is the only one that can be opened from the outside.

So to open the door I had to awkwardly grip that little metal piece on the right door and pull it enough to get my fingers in between the doors and then pull it the rest of the way open.

This was the second time that week that the door handle simply disappeared. It was reported to the facilities crew and eventually (a few hours later) the right hand door handle was replaced.

What is a UX Problem?

I like to think of the door thing as a UX problem. It’s something that is incredibly easy to experience, and sometimes incredibly easy to fix, but often very hard to predict.

How to experience the door problem.

• Walk up to the door and try to open it from the outside.

Possible fixes for the door problem.

• Replace the handle.
• Prop the door open.
• Unlock the left door.
• Have someone sit by the door and open it when someone walks up.

How to predict and prevent the door problem?

• You don’t.
• Seriously, who could have predicted that at some point the door handle would have just disappeared?
• I mean I’m still really curious as to how this happened twice, but I’m just a tenant in a co-working space and don’t actually know the facilities people.
• Well, sure you could prevent it by welding the handles and make them really impossible to take off, but this kind of seems like overkill for an incredibly rare problem.

UX Problems don’t usually fix themselves.

Let’s say you have a newsletter that you send out. And anytime someone goes to click on one of the links in your newsletter, they get taken to the wrong place.

Very few people will ever tell you. They’ll just shrug and move on with their days.

Same thing goes for a confusing section of a report. Very few people are ever going to complain that they don’t understand what you are talking about. The vast majority will just stop reading.

So how do you find your UX problems?

Look at your reports through a reader’s eye view.

UX problems might be hard to predict, but they’re usually incredibly easy to experience.

The ideal way is to test your designs with real people. Even if it’s not a person who is actually a part of your primary audience. You can ask a coworker to read your report or design and tell them to pretend they were a member of your target audience.

At the very least, you should read your reports from an audience perspective. And not just before they’re published but after they’re published too.

Because most of the UX Problems that happen with reports are not because of how the reports are written but how they are published and shared. And we rarely check our work once we pass it on to the web & communication’s teams or share it on social.

Want to go deeper?

I’m currently recruiting participants for the spring cohort of my information design academy. You should consider joining us!

Without fail, this question would come up when I would present to evaluation teams on why they should consider changing the way they report.

It would usually come from a senior member of the team. Likely one that had never heard of me before another team member suggested they bring me in for a conversation. The question would go something like this…

“Well, I can see how shorter reports would be easier to read and share online. But we have a responsibility to document our work for our clients. How does the way you suggest reporting meet that requirement?”

And here’s the thing…

it doesn’t.

Because that’s not the kind of report I think needs a fundamental change.

The two types of reports and one of our biggest mistakes.

Okay, there are way more than two types of reports. But for the sake of argument, and for this blog post, let’s pretend that there are only two.

The two types of reports:

• The kind of report that systematically documents your work.
• The kind of report that you use to present your findings to an audience.

One of the biggest mistakes I see organizations make when it comes to reporting is that they think they can do both things with the same report.

This is the easiest way to fail at both.

Here is the usual story. A team has been writing the same kind of documentation style report for decades. It’s boring, long, and comprehensive.

The report walks you through the team’s work in painstaking detail, talking through the research or evaluation questions, walking through the program model, giving an overview of the methodological choices the team made, highlighting different quirks in the data collection process, explaining how the team analyzed the findings, and then writing out all the conclusions that were reached.

Then someone decides that this report is way too boring, long, and comprehensive to be read by senior management. They want something shorter, more visual, and more interesting.

So the next time this team goes to create a report, they try to do all the stuff they did before but make it shorter, more visual, and more interesting.

All of a sudden most of that detail is gone. In its place is a report that’s shorter, and more visual, but still too detailed to be interesting for most audiences. And it’s also too brief to provide good enough documentation of the project.

So now everyone is unhappy.

Creating a “Documenting Your Work” Report

Be boring.

Seriously, when documenting, your goal is not to create a report that people will be itching to read cover to cover. Your goal is to systematically document all the things. As a professional data person, this documentation is important. Maybe nobody will read it, or only a very few will read it, but it should exist.

Let this report be boring.

Let this report be long, if that’s what it wants to be.

Also, and I know this will be tempting, don’t try to make it more visually appealing or better designed. Just write it in Word or Google Docs. Don’t even bother prettying up the charts, you can do that later in your presentation report(s).

The more time and money you spend trying to pretty up your documentation report, the less time and money you have to spend creating reports for people.

Creating a “Presenting to an Audience” Report

This is the report (or reports) designed to communicate your findings (or other important information) to an audience.

The cool thing about having already documented your project with a boring report is that now you can really get focused on reporting the stuff your audience wants or needs.

Here is the advice I give my workshop peeps. You can get way more value out of creating a string of five small PowerPoint slide sized micrographics than you ever would by spending more time redesigning your long report. The sad part is that the latter is where most large organizations use their design budget while almost completely ignoring the former.

Want a simple, don’t want to think about it, approach? Do this.

Step 1. Create at least 5 micrographics (i.e. annotated charts or small infographics) each designed to reach very specific members of your audience.

Step 2. Create a slidedoc report of about 30 slides to use as an executive summary style report. You’ll likely integrate your micrographics into this slidedoc.

Step 3. Then adapt those 30 slides into a presentation deck (same thing as the slide doc but without most of the words). This is what you will use for any presentation or webinar where you will be appearing with the slides. You can offer the slidedoc version as a download following the presentation.

Step 4. Then if you have time and budget, create a 25 – 30 page visual PDF report. This is your final visual report that might not cover EVERYTHING, but does a good job summarizing the big points.

What are your thoughts?

• Do you already separate documentation from presentation with your own reports?
• If you already create visual reports, how do you document your work?
• Do you have any go-to report styles for your presentation reports?

At the end of every year you get loads of advice for how to approach the next one.

Lots of that advice centers around resolutions, words, or statements. I decided to do something a little different this year and give myself a list of how questions.

Will it be useful in the coming year? Who knows? But it did feel like a useful activity for today.

Here are my own questions and thoughts, and if you feel bold enough to share your own I would love to see them in the comments.

How can I make my business boring?

It’s now been well over 6 years since I went indie.

But it still kind of feels like I’m running a “just winging it” business. And you know what, I think it’s more than a feeling. I think that is the business I’ve built.

I tried a bunch of different things. Then I tried a bunch of other different things. Then I lucked into a couple really good clients. And those couple of really good clients have sustained me over the years.

And maybe that’s what it means to be an indie consultant. Or at least, what it means for me to be an indie consultant.

There are times I feel like I should be the motivational speaker companies hire to bring in and convince their employees not to leave and start their own businesses. Because as much as I love my job, I also feel like I’m one bad break away from failure.

This feeling comes and goes. But when it’s here I long for something a little more boring and a little less risky.

How do I get people to join my design workshop?

I relaunched my design workshop in the fall of 2021, and it’s still going strong. The workshop gives me a space to teach, show, coach, and mentor. And the feedback I’ve received from participants has been fantastic.

The workshop was built the only way I know how, through a continuous series of iterations based on feedback and live experiences. The constant changes forced me to ask hard questions about what I truly offer and what evaluators really need. If you’re wondering about the conclusions I’ve arrived to you should check out my new workshop page.

But one thing I haven’t learned is how to get more people to join. I have a few ideas, but this is certainly going to be an open question for the coming year.

How can I replace my best client and contract?

Back in 2019 I had a short conversation with Jan Fields of the Michigan Public Health Institute. Jan quickly became my best client, giving me the financial cover to keep my independent consulting business running strong. He’s also just a great person who I love working with.

Over these past few years I’ve been able to develop and facilitate a virtual Evaluation Community of Practice for the CDC’s Overdose Data to Action program. It’s been a mix of webinar facilitation, web development, digital communications, teaching, and community building.

But like all Federal contracts, this one will come to an end later this year. There is hope that it will pick back up, but as an indie subcontracting my way onto the overall project, there is definitely no certainty that my role will continue. I still have time but I fully expect this question to occupy my mind throughout the coming year.

How do I keep the comics flowing?

I enjoy drawing comics. They are certainly a differentiator and help to keep me in the public eye (at least inside the evaluation world). But it’s not like they are huge moneymakers.

As I tell my workshop participants, creativity is a process. And the process of comic making can also have value, especially in terms of networking and learning. I have learned more about evaluation through process of cartooning than through any other professional activity.

I expect that in the coming year I’ll be searching for more ways to keep the comics coming. What this will look like is an open question, but one I will definitely be addressing in 2023.

How do I successfully center my family?

The biggest reason I want to make my work boring is to take it off my mind at the end of the day.

The independent consulting life gives you a kind of freedom, but it also has a way of staying in your mind far longer than any other job I’ve ever held. It’s hard to leave work at work when you, as an individual, are the office.

This has been a constant challenge since I started my business. But I hope to be more successful in the coming year. Because what good is business success if it makes me less of a successful father and husband?

Can you come up with 5 how questions for your year ahead?

You don’t have to blog your questions or reasoning. But I found this to be a really nice exercise and totally suggest giving it a try!

Every holiday season I try to draw a set of Christmas comics. For whatever reason, it was harder this year.

SIMON: Do you still pick up the pen?

SPIEGELMAN: I pick it up, but it’s a very heavy instrument right now. And I’ve got little notes and little doodles and drawings. I have no idea what they can come to, and I’m hoping that the pen gets lighter if I get to use it every day and build up my finger muscles, you know?

Art Spiegelman on the re-issue of his book ‘Breakdowns’

I think in the new year I need to find some other way to pursue inspiration. And I have an idea, but will talk about that in January.

For now, I hope you’ll find at least a chuckle or two in this year’s set.

Zero Carbon Santa

I wonder what Santa could use instead of coal.

Inbox Zero

Santa probably gets a lot of emails. But I doubt he’s the one reading them all. Some elf probably has that responsibility…

Ugly Sweater Contest

There was an ugly sweater contest at my office last week. I forgot about it but went to work wearing a sweater. The sweater wasn’t ugly, but it did inspire this cartoon.

Day Rate

When I talk to other indie consultants, especially new ones, my key piece of advice is that your hourly or daily rate doesn’t really matter. Budgets matter a lot more.

Take reindeer for example, they only one day a year. They might get a really good rate for that one day, but is it enough to keep them in reindeer food and shelter for the remaining 364.

Logic model is a little too simple.

Trying to think of evaluation stuff I haven’t talked about during past holidays. This one is based on the idea that a lame logic model can lead to a lame evaluation. Also, these are elves, but I think we’re a little too zoomed in for that to be noticeable.

Annual Reports

Do you think Santa makes the elves deliver a year end annual report? Seems like it would be a little bit of stressful deadline to me.

I hope you have a fantastic holiday!

Whatever holiday you celebrate, I hope it is a good one. See ya next week