The May 13 Group

the next day for evaluation

You are here: Home / Archives for allblogs

allblogs

Start the School Year By Hearing from Stakeholders

Growing up in New Jersey, the day after Labor Day always marked the start of a new school year … and the day I finally got to wear the new outfit I had carefully planned and crack open my new, pristine notebooks.

If you couldn’t tell, I have always loved the excitement of returning to school. 

Unfortunately, for many children, families, and educators, this year felt different. 

Some of the usual excitement and jitters have been replaced by trepidation about what to expect from a year like no other.

Concerns about health and safety, academic progress, and schedule juggling have been abundant in my conversations with teachers and the staff and family surveys I have analyzed. 

So how will schools and districts know if they are adequately addressing their stakeholders’ fears?

Well… they’ve got to ask them. 

Colleagues in a number of recent conversations have been discussing the use of continuous improvement cycles. If you’re not familiar with continuous improvement, its hallmark is the Plan-Do-Study-Act (PDSA) Cycle.  

Alicia Grunow of the Carnegie Foundation for the Advancement of Teaching explains the PDSA cycle: 
More simply put, schools and districts need to: 

  • Determine what need or concern they are going to address and how they will address it, 
  • Implement the intervention, program, or practice they proposed, 
  • Collect and analyze data to see if what was implemented actually worked, and
  • Make a plan for what to continue and what to change. 

Then, the cycle starts all over again … quickly. 

We’re not talking about huge, multi-year studies here … This is a relatively quick and simple process! 

Make a plan, implement the plan, figure out if the plan worked, and if not, adjust and try again! 

With school kicking off, schools and districts have already put a short-term plan in place and are putting it into action. 

And this year, short cycles of trial and error are going to be key, as even our modes of schooling could change multiple times throughout the year. 

So how can schools and districts get feedback from their stakeholders NOW to see if their plan worked? 

Instead of a lengthy formal survey, think of creative ways to ask for feedback:​

Asking one or two questions at a time in interactive ways will make it easy for stakeholders of all groups to participate, prevent them from getting tired of surveys, and give you real-time data about how people are feeling.*

*Just make sure the platforms you choose will allow for translation.

Now here’s the kicker: once you collect on data, you have to complete the cycle … ACT!

Make it clear for students, families, and staff that you valued their feedback and are going to put it to use … and tell them how!

Start this crazy school year off right by lifting up the voices of your stakeholders in fun and easy ways and demonstrating that their feedback will guide your next round of planning and action. 

Innovation Implementation Index

What does it truly mean to say that you are an innovative organization? One of the simplest, most powerful measures is implementation. How many innovative initiatives have you launched and put into the world?

There are arguments why the QWERTY keyboard is inferior to other designs, but there’s no argument that its the best innovation we’ve ever had when it comes to typing. Why? It’s the one that was implemented into practice.

Success Marks

Checkmarks on a list usually imply some kind of rote behaviour that isn’t suitable for innovation, although checklists themselves are useful for quality control. We prefer the idea of ‘success marks’ which indicate that an idea has been successfully taken from concept and put into practice — whether or not it yields desired outcomes.

As an index, we propose the following six metrics:

  1. The number of new initiatives that have been implemented into practice
  2. The number of projects that have developed prototypes that have been tested
  3. The prototype ‘death rate’: divide the number of terminated prototypes by the total number of prototypes developed
  4. The number of prototypes moved into implementation (* which is a similar, but not necessarily the same number as #1)
  5. The number of new initiatives started:
  6. The initiative death rate: divide the number of ‘new’ projects started by the number of projects implemented

Number 1 is the most important of all of these.

The reason we count all six as part of an index is that each of these represents some form of concerted action and effort toward moving ideas out of concept into action with the lowest number (Number 1) representing the highest value to the organization.

While having ideas is a precursor to prototypes we don’t we count this because it is easy to game this metric.

We also don’t believe in coming up with metrics about innovation or even prototype quality because the success of implementation and the quality of the prototype don’t always match. You can implement an idea that is still in development, yet still has value that can be realized right away.

Want to show how innovative you are? Consider scoring yourself on these metrics to see how much of what your organization does is talk and how much of it is action.

Want to move ideas into action? We can help. And we can develop the metrics to show what kind of impact your innovation has in the world. Let us show you how.

I have a book now! Introducing Evaluation Illustrated

Seriously, it’s now on Amazon. You can buy it!

So yeah, I have a book.

I guess, technically, I’ve had a book since August 27. That’s when I put it up on Amazon. It’s just, nobody knew about it and I didn’t share it with anyone until I could have a copy in my hands.

It’s 114 of my favorite evaluation cartoons, in paperback.

These are cartoons that if you’ve been following me for awhile you probably already know. Maybe you even have copies of some these cartoons in a folder on your computer waiting to be put into your next presentation slide deck.

But there is something about a book. To be able to flip through the cartoons without a screen. To have something you can give to colleagues and friends. To be able to flag the copy with sticky notes for that next time you meet with a potential client.

Now, I could really use your help.

If an author self publishes a book, and his amazing friends/followers don’t share it on their social networks or write nice Amazon reviews, does it exist?

Maybe Confucius (or Aristotle) (or Einstein) (or Gretzky) (or North American Proverb of Unknown Origin)

Please do these things, in whatever order you choose.

  • Share my book on your social media network of choice (here is the Amazon link)
  • Write me an Amazon Review (if you are reading this you probably know my cartoons well enough to say nice things about my work)
  • Buy the book.
  • Send me a message via email (chris @freshspectrum.com) and let me know your favorite cartoon of mine, and why. My hope would be to read some these via little videos to promote the book.

Thank you so much for all the support you have given to me over the years. And for all of you who have been asking me to put some of my cartoons in a book, thank you for both your encouragement and your patience 🙂

Public Speaking Tips for Researchers and Evaluators from Isaac Castillo

Remember that Excel Test? Where I walked into a job interview, was asked about my prior experience with data, and then—surprise!!—the Director gave me a timed test to assess my true skill level??? And I panicked. Heart pounding. Face flushed. Hands shaking as I rushed to finish the test as quickly as possible??

That was Isaac Castillo…

…a.k.a. an Outstanding Boss, Evaluator, and Public Speaking Extraordinaire.

I’m so happy he hired me a decade ago, and I’m even happier that we’ve stayed in touch over the years.

Haven’t met Isaac yet? Isaac Castillo is the Director of Outcomes, Assessment and Learning at Venture Philanthropy Partners.  He has 20+ years of experience with research, evaluation and measurement. 

Today, the tables have turned. I got to interview Isaac!

Watch Our Conversation

During this 5-minute interview, we talked about Isaac’s career trajectory and his best public speaking advice for researchers, evaluators, and data analysts.

Prepare Half as Much Content as You Think You Need

Although Isaac has spent two decades working in research and evaluation, he started his career as a competitive debater and public speaking coach.

I asked Isaac for his #1 piece of public speaking advice.

Isaac advised us to prepare half as much content as we think we need.

For example, if you’ve been asked to speak for a half hour, prepare for 15 minutes worth of speaking.

We Always Need More Time Than We Think

Why does this work?

First, we often don’t have a good sense of how long it takes to deliver our content, or how long our speaking styles result in the content being delivered. We always take more time to present our information than we think we do.

When Isaac was speaking in high school and college, he struggled giving timed speeches. It’s taken more than a thousand presentations for Isaac to get a good sense of pacing and how fast he can go. Most of us haven’t given a thousand presentations, so we haven’t had enough practice fine-tuning our timing yet. Isaac says we’re always going to take more time than we think.

We Don’t Want to Seem Rushed

Often times, we accidentally put our most important content at the very end.

Then, when we get to the meaty content, we’ll look at the clock, realize we have 3 minutes left, and then try and wrap it up and rush through it.

If we prepare half as much content as you need, we can take your time. We can go off on some tangents as they feel appropriate. We can answer questions throughout our presentation. We won’t feel rushed.

We Can Focus Better on the Most Important Content

Preparing half as much content as we think we need forces us to really focus on the most important content. Which points do we want to drive home?

It also forces you to get rid of content that isn’t central to the points we’re trying to deliver.

It used to be hard for me to cut content from my presentation. Deleting slides forever felt too permanent. What if I wanted to share those topics in a future presentation? Mentally, I remind myself that those points aren’t gone forever. I’m just saving them for a future presentation.

Leave Our Audience Wanting More

Isaac’s friend is a Beatles fan, and she gave him great advice.

The Beatles intentionally wrote short songs to leave us wanting more, so Isaac has internalized that as well.

Give Enough to Pique Interest

Finally, Isaac said that if we give enough information to pique interest, then our audience will ask questions and want to follow-up with us.

Connect with Isaac Castillo

Connect with Isaac:

How to formulate strong outputs

Outputs are arguably not the most important level of the results chain. It is outcomes that should be the focus of a good plan. Ultimately, that´s what counts.

However, outputs still matter.

Just to be clear: Simply put, outputs refer to changes in skills or abilities, or the availability of new products and services. In plain lingo: Outputs are what we plan to do to achieve a result.

Ok, let’s be a bit more precise: Outputs usually refers to a group of people or an organization that has improved capacities, abilities, skills, knowledge, systems, policies or if something is built, created or repaired as a direct result of support provided. That’s a definition we can work with.

Language is important

When describing what you do, focus on the change, not the process. Language matters.

Don’t say: ‘Local organisations will support young women and men in becoming community leaders.’ This emphasises the process rather than the change.

Instead, emphasis what will be different as a result of your support. Say: ‘Young women and men have the skills and motivation to be community leaders’. 

Make it time-bound

An organization’s support is typically not open-ended. You usually expect to wrap up what you do at a certain time. Emphasise that your activities are carried out within a certain time frame. So it’s always helpful to include in the formulation for example ‘By January 2019, …’. 

A formula for describing what you do

To ensure that you accurately describe what you do, use the following formula:

Thomas Winderl, 08.09.2020

The post How to formulate strong outputs appeared first on Thomas Winderl.

en English
af Afrikaanssq Shqipam አማርኛar العربيةhy Հայերենaz Azərbaycan dilieu Euskarabe Беларуская моваbn বাংলাbs Bosanskibg Българскиca Catalàceb Cebuanony Chichewazh-CN 简体中文zh-TW 繁體中文co Corsuhr Hrvatskics Čeština‎da Dansknl Nederlandsen Englisheo Esperantoet Eestitl Filipinofi Suomifr Françaisfy Fryskgl Galegoka ქართულიde Deutschel Ελληνικάgu ગુજરાતીht Kreyol ayisyenha Harshen Hausahaw Ōlelo Hawaiʻiiw עִבְרִיתhi हिन्दीhmn Hmonghu Magyaris Íslenskaig Igboid Bahasa Indonesiaga Gaeilgeit Italianoja 日本語jw Basa Jawakn ಕನ್ನಡkk Қазақ тіліkm ភាសាខ្មែរko 한국어ku كوردی‎ky Кыргызчаlo ພາສາລາວla Latinlv Latviešu valodalt Lietuvių kalbalb Lëtzebuergeschmk Македонски јазикmg Malagasyms Bahasa Melayuml മലയാളംmt Maltesemi Te Reo Māorimr मराठीmn Монголmy ဗမာစာne नेपालीno Norsk bokmålps پښتوfa فارسیpl Polskipt Portuguêspa ਪੰਜਾਬੀro Românăru Русскийsm Samoangd Gàidhligsr Српски језикst Sesothosn Shonasd سنڌيsi සිංහලsk Slovenčinasl Slovenščinaso Afsoomaalies Españolsu Basa Sundasw Kiswahilisv Svenskatg Тоҷикӣta தமிழ்te తెలుగుth ไทยtr Türkçeuk Українськаur اردوuz O‘zbekchavi Tiếng Việtcy Cymraegxh isiXhosayi יידישyo Yorùbázu Zulu