The Evaluation Mindset: When Nobody Listens

If only we had known we were vastly underprepared for a global pandemic…

I say that tongue-in-cheek. Of course we knew. President after president was briefed with data and evidence showing the potential devastation a pandemic could have on our global society.

In advance of a pandemic, anything you say sounds alarmist,” Leavitt explained. “After a pandemic starts, everything you’ve done is inadequate.

Inside America’s 2-Decade Failure to Prepare for Coronavirus

I would also guess that across the world, epidemiologists, doctors, and health department staff are thinking, “I kept saying this would happen. But nobody would listen!”

As anyone in working in the realm of evidence knows, this kind of thing happens all the time with all sorts of problems.

The reality behind informed decision making is that while having the knowledge is necessary, it is never sufficient.

The curse of Cassandra

Imagine an oracle from a book or movie, say Professor Trelawney from Harry Potter. Quite a bit eccentric but offers important insight, by way of a prophecy, that is clouded in riddles.

Our focus is then on the hero who is tasked with disentangling the riddle. The oracle’s role in modern storytelling is usually just to deliver the riddle and then disappear.

The origin of the character type can be traced to Cassandra from Greek mythology. In that story Cassandra was given the gift to know the future. Subsequently she was cursed so that nobody would ever believe her prophecies.

She had the power to know, but not be believed.

It is a curse that unfortunately hits close to home for far too many evaluators. Knowing is only half the battle.

If they don’t get it, they won’t use it.

Evaluators love their methods. So much so that they often stick them at the front of the report.

How else is everyone going to know the precise steps we went through to come up with the answers and advice we plan to share?

The knee jerk reaction to nobody listening or caring about your methods is to skip to the conclusions and recommendations. Give the audience what they want! Put them up front, then leave the methods in the back for anyone who wants to go deeper.

But here’s the thing. That’s not enough.

The problem isn’t that the methods section is boring (well it might be, but that’s not the biggest problem).

The biggest problem is that in sharing your work and the eventual solutions, you forgot to show your audience why they should care. Maybe you assumed they already knew the problem, or they don’t care, so you skipped diving deeper.

But that could be a big mistake.

The best way to start anything, whether it’s an evaluation report, presentation, blog post, or cartoon set, is to start where you started. What is the essential problem or challenge? Why should anyone care?

Because if they don’t get why they should care, they’ll never listen.

When the HiPPO rules.

Do you know the HiPPO?

It’s an acronym for the Highest Paid Person’s Opinion. Unfortunately, so many design decisions are driven not by the evidence but by the HiPPO.

If you’re lucky, maybe the HiPPO is the most informed and should be driving the decisions. But I wouldn’t necessarily count on that, even when it seems true.

Here are some tips to change the situation.

• It’s never a good idea to make it you versus them. Yes, speak up every once and awhile, but remember there is a power imbalance.
• Bring in a third party. UX and Human Centered Design feature lots of user testing. It’s so much more effective to have a user tester indirectly tell a HiPPO their idea stinks.
• Fall back on data and evidence. If you are truly using evidence to inform your perspective you shouldn’t need to cherry pick. “While that could be true, the evidence does not support moving in that direction.”
• Data parties and placemats. In other words, have stakeholders analyze the data together. Sure you can do it yourself, but if you include stakeholders in the analysis they will be more likely to follow the recommendations.

Also, if you can’t identify the HiPPO on a project team, maybe it’s you.

Deliver a better presentation.

A better presentation isn’t just about making the data and evidence easier to understand. Remember, knowing is only half the battle.

A better presentation connects with the audience. And unless the audience is super hungry for the information, it doesn’t just show the data.

Show them why they should care.

And if they shouldn’t care…why are you presenting?

April 15, 3:30 PM Eastern/12:30 PM Pacific

Come join us for tomorrow’s unwebinar, our guest facilitator will be Dana Wanzer. The seed topic, The Integration of Research and Practice in Evaluation.

You can register here > https://www.crowdcast.io/e/evalcentral/

Topic inspiration: ROE TIG Week: Research on Evaluation – A Glance Towards Integrative Evaluation Science 

If you haven’t attended one, these sessions have been a blast.

The audience is growing and might even push me to have to pay for the bigger Crowdcast account sooner than later.

Lil Help?

So between Eval Central and Fresh Spectrum my tech expenses are increasing.

I’m just an indie consultant, so my technical overhead all comes out of pocket. So if you appreciate my work would you consider becoming a Patron?