• Skip to main content
  • Skip to footer
  • Home

The May 13 Group

the next day for evaluation

  • Get Involved
  • Our Work
  • About Us
You are here: Home / Archives for allblogs

allblogs

Feb 23 2021

Comment on How to formulate strong outputs by Mirian

Very simple and practical information on getting the formulation right. Do you have a similar formula on outcome and impact level? That would be great!

Written by cplysy · Categorized: thomaswinderl

Feb 23 2021

Your data tracking cheat sheet

In my last post, I talked about how logic models — although they can be a chore — can actually be a great visual roadmap of your program’s components, benchmarks, and goals. 

Working collaboratively to develop a logic model can be a unifying experience for teams, a way to get everyone on the same page about the work. 

But what keeps your finished logic model from disappearing into the dark void of your computer’s file system, never to be found or opened again?

If you’ve been reading my blog for awhile, you may know where I’m going… 

You can use your logic model to help design your team’s data tracking system! 
​
Although people design them differently, logic models always have a column for either the measures that you’ll use to assess progress (as I have in my template below) or the immediate outputs that would occur as a result of your activities.

Picture

If you’re measuring your family engagement efforts, some examples could include: 

  • Number of families attending school events
  • Number of home visits conducted
  • Percent of children whose families have engaged with the school in some way throughout the year
  • Number of parent/family leaders 

Your logic model is basically a cheat sheet to the data points you’ll want to track for your program!

​So if you follow the steps in my free guide to tracking your engagement data, you’ll see that you’ve already answered some of the questions in Step 1 — why you’re tracking the data. 

I don’t know about you, but I think that identifying the purpose behind each part of our work is often the hardest. But since you’ve already connected your measures to your short- and long-term goals, you’re ahead of the game!

With your logic model in hand, you’ll be ready to tackle the rest of the questions with ease and start tracking. 

If you haven’t gotten your hands on my free guide, use the form below to get your copy!

Written by cplysy · Categorized: engagewithdata

Feb 22 2021

Comment on Museums and Climate Change: A Form-follows-Function Perspective by Emlyn Koster, PhD

As resources for reflection, museum experiences are at their most impactful when, one life at a time, they turn apathy into interest, interest into insight, and insight into action. As soon as public health conditions permit, museum toolkits should add topical conversations to their menu of exhibitions and programs. A tradition in museum offerings, lectures need to be as much about the present and future as the past. Every onsite, online and offsite step towards a grasp of the perils facing environments and societies — which increasingly the sciences and humanities view as inextricably linked — is like adding a fresh droplet to a needed glass of water. New opportunities for each age and stage of learning need to occur alongside new boardroom attitudes, both inside and outside the museum. Although consequential changes in the atmosphere are inseparable from humanity’s adverse impacts on the Earth’s other outer shells, the distinction between weather and climate is a tangible entry point towards scientific literacy for us all. Every now and then, ponder the words conversation and conservation: surely such a small difference should never have amounted to such a big problem!

Written by cplysy · Categorized: rka

Feb 19 2021

What is Impact Evaluation? Cartoon Glossary

This is a series of posts providing quick overviews of important topics in research and evaluation. Each post in this series will include at least 3 cartoons from my archives and at least 3 links to recommended resources. I only give quotes here and recommend that you follow the links below each quote for more detailed information.

Freshspectrum Cartoon by Chris Lysy. 
"I called you all here because I need you to stop helping people. It's really messing up my impact assessment."

An impact evaluation provides information about the impacts produced by an intervention – positive and negative, intended and unintended, direct and indirect. This means that an impact evaluation must establish what has been the cause of observed changes (in this case ‘impacts’) referred to as causal attribution (also referred to as causal inference).

If an impact evaluation fails to systematically undertake causal attribution, there is a greater risk that the evaluation will produce incorrect findings and lead to incorrect decisions. For example, deciding to scale up when the programme is actually ineffective or effective only in certain limited situations, or deciding to exit when a programme could be made to work if limiting factors were addressed.

Better Evaluation’s Page on Impact Evaluation

Freshspectrum Cartoon by Chris Lysy. 
"You've been given a great gift, George: A chance to see what the world would be like without you."
"So like a really well designed counterfactual in an impact evaluation?"

An impact evaluation should only be undertaken when its intended use can be clearly identified and when it is likely to be able to produce useful findings, taking into account the availability of resources and the timing of decisions about the programme or policy under investigation. A formal evaluability assessment (EA) might first need to be conducted to assess these aspects.

Formative impact evaluations are undertaken to inform decisions in regard to making changes to a programme or policy. While many formative evaluations focus on processes, impact evaluations can be used formatively if an intervention is ongoing. For example, the findings of an impact evaluation can be used to improve implementation of a programme for the next intake of participants.

Summative impact evaluations are undertaken to inform decisions about whether to continue, discontinue, replicate or scale up an intervention. Ideally, a summative impact evaluation not only produces findings about ‘what works’ but also provides information about what is needed to make the intervention work for different groups in different settings, which can then be used to inform decisions.

Overview of Impact Evaluation: Methodological Briefs – Impact Evaluation No. 1. Written by Patricia Rogers in 2014.

Freshspectrum Cartoon by Chris Lysy. 
"We need to accept the fact that what we are doing is measuring with the aim of reducing the uncertainty about the contribution made, not proving the contribution made. John Mayne, 1999"

A key question in the assessment of programmes and projects is that of attribution: to what extent are observed results due to programme activities rather than other factors? What we want to know is whether or not the programme has made a difference—whether or not it has added value. Experimental or quasi-experimental designs that might answer these questions are often not feasible or not practical. In such cases, contribution analysis can help managers come to reasonably robust conclusions about the contribution being made by programmes to observed results.

Contribution analysis explores attribution through assessing the contribution a programme is making to observed results. It sets out to verify the theory of change behind a programme and, at the same time, takes into consideration other influencing factors. Causality is inferred from the following evidence:

1. The programme is based on a reasoned theory of change: the assumptions behind why the program is expected to work are sound, are plausible, and are agreed upon by at least some of the key players.

2. The activities of the programme were implemented.

3. The theory of change is verified by evidence: the chain of expected results occurred.

4. Other factors influencing the programme were assessed and were either shown not to have made a significant contribution or, if they did, the relative contribution was recognised.

Contribution analysis is useful in situations where the programme is not experimental—there is little or no scope for varying how the program is implemented—and the programme has been funded on the basis of a theory of change. Many managers and 4 evaluators assessing the performance of programmes face this situation. Kotvojs (2006) describes one way of using contribution analysis in a development context, “as a means to consider progress towards outputs and intermediate and end outcomes” (p. 1).

John Mayne’s 2008 ILAC Brief: Contribution analysis: An approach to exploring cause and effect

Written by cplysy · Categorized: freshspectrum

Feb 17 2021

Vision-Driven Leadership: Now More Than Ever

Emlyn Koster urges the museum sector to combine operational recovery efforts with strategic pathways towards a more holistic societal and environmental future. 

“The very subject of our discussion shows the painful anxiety and uncertainty with which we search for our proper function in the national struggle for a better future.” What sounds like a synopsis of today’s situation was part of a speech during World War II at AAM’s 1942 Annual Meeting by A.E. Parr, then Director of the American Museum of Natural History. Not since comparable crises during the last century has the total responsibility of those appointed Director, Executive Director, or President & CEO — the leadership position ultimately accountable for what goes wrong and/or for what goes well in a museum — been so demanding yet so seldom talked about.

Since the pandemic began, the museum sector’s countless webinars have mainly involved mid-level staff focused on operational efforts, such as online programming and member retention. Airing of strategic considerations by senior leadership, such as the implications of major changes in social norms and business models, has been almost absent. The vital theme of International Museum Day 2021 is “The Future of Museums: Recover and Reimagine”. I fervently hope that “reimagination” features the pursuit of pathways into illuminating the perilous needs of the 21st century. Our ability to be externally meaningful and supportable in profoundly new ways is at stake.

As I switched from geology to museology 32 years ago with my focus shifting from the past to the future and from local to ‘glocal’ (a seamless local and global view), I became interested in the anatomy of leadership. Insights about what institutional relevance entails flowed from multiple sources. These included executive workshops in Toronto and Manhattan while CEO of the Ontario Science Centre and Liberty Science Center, an international workshop for nonprofit CEOs at the Harvard Business School, being a resource to the Getty and Noyce leadership institutes, and a board role at the Institute of Ethical Leadership at the Rutgers Business School. 

Close up of a compass pointing towards the word "leadership."
We need vision-driven leadership now more than ever. Image credit.

My most intense leadership experience was in the aftermath of the September 11, 2001 terrorist attacks while I was CEO of Liberty Science Center which is located across the Hudson River from the World Trade Center. Flowing from this institution’s innovative experiences in community engagement which was later described in a collection of exemplars of socially responsible museums, it partnered with a trauma psychologist and The Families of September 11. Several years later, I came across a resonant outlook by Jacqueline Gijssen, Vancouver’s cultural planner, in the magazine of the Canadian Museums Association: this imagined a future in which “the museum becomes critical to the long-range health of a place, central to think-tanks, and community transformations… one of those organizations a mayor calls upon when a crisis hits… an institution that others actively seek for guidance and expertise”. An analogous focus on environmental stewardship was highlighted in the conclusion of AAM’s decadal re-accreditation of the North Carolina Museum of Natural Sciences during my tenure as CEO: ”This institution has forthrightly evolved its interpretative philosophy and strategy to address bigger stories about humans as an inseparable element in the ecosystem of all life, and therefore to be concerned about matters of conservation and sustainability”.

The combination of those experiences and 2020’s existential crises have impelled me to urge an unprecedented paradigm shift in the museum sector (see my forthcoming article, “Paradigm Shift to Illuminate Our Disrupted Planet, in the Spring 2021 issue of Exhibition). In leadership terms, its baseline is a realization that vision then values then mission then strategy is the optimal order of institutional planning considerations. Also, and in integrated ways, the contributions of three thought-leaders known to me for several decades came into sharper focus.

  • In the compass recommended by Stephen Covey, the first three principles of individual effectiveness are be proactive®, begin with the end in mind®, and put first things first®. These underscore why vision and values should be early considerations. He also advised on the need for resilient interpersonal relationships: likening them to bank accounts, deposits must outweigh withdrawals with any overdrafts promptly attended to.
  • Covey also illustrated the conclusion of Peter Drucker that efficiency is doing things right and effectiveness is doing the right things. He likened efficiency to climbing the ladder of success and effectiveness to first determining the strongest wall for the ladder.
  • Burt Nanus used four dimensions to describe a balanced leadership approach for visionary organizations. Axes labeled internal/external and now/future frame the roles of coach (internal/now), change agent (internal/future), spokesperson (external/now), and direction setter (external/future).

When a museum appoints a new chief executive due to the resignation, retirement or termination of her/his predecessor, a new strategic chapter in the institution’s evolution inevitably begins. What is in the ethical, operational and strategic toolkit of experiences and insights that the next leader brings? And what is the probability that this individual will be up to mobilizing resources to respond to the institution’s evolving opportunities and unforeseeable needs? Referring to a ladder of increased abilities, those in leadership roles are ideally aware, both in themselves and in those around them, that unconscious incompetence à conscious incompetence à conscious competence à unconscious competence à reflective competence are the rungs of intertwined personal and professional growth.

“If you want the same old, same old; the tried and the true; the safe and secure, then visionary leadership is not for you”.

-Anne Ackerson & Joan Baldwin, Leadership Matters: Leading Museums in an Age of Discord

Museums that prioritize popularity over meaningfulness and that launch capital campaigns to become bigger but not more relevant to surrounding needs are, in my view, wrong-headed. I have long advocated for Aristotle’s philosophy that leadership should be about the harmonious pursuit of positive consequences in the world and John Cotton Dana’s viewpoint that a museum should fit itself to the needs of its surroundings. Returning to the subject of my January blog, the most pressing need of the museum sector is to illuminate the surging evidence of environmental and societal perils. The Anthropocene, a transdisciplinary concept which I unravel elsewhere, recognizes humanity as the predominant species which, in a geological nanosecond, has ecologically detached itself from the Earth System, endangering the future of both.

Emlyn Koster, PhD (koster.emlyn@gmail.com) has been the CEO of the Royal Tyrrell Museum of Paleontology, Ontario Science Centre, Liberty Science Center, and NC Museum of Natural Sciences. Combining his geological and museological experiences with a humanistic outlook, he is focused on humanity’s escalating disruption of the Earth System. Current appointments include an ambassador for the International Coalition of Sites of Conscience and adjunct professor in Marine, Earth and Atmospheric Sciences at NC State University.

The post Vision-Driven Leadership: Now More Than Ever appeared first on RK&A.

Written by cplysy · Categorized: rka

  • « Go to Previous Page
  • Go to page 1
  • Interim pages omitted …
  • Go to page 203
  • Go to page 204
  • Go to page 205
  • Go to page 206
  • Go to page 207
  • Interim pages omitted …
  • Go to page 310
  • Go to Next Page »

Footer

Follow our Work

The easiest way to stay connected to our work is to join our newsletter. You’ll get updates on projects, learn about new events, and hear stories from those evaluators whom the field continues to actively exclude and erase.

Get Updates

Want to take further action or join a pod? Click here to learn more.

Copyright © 2026 · The May 13 Group · Log in

en English
af Afrikaanssq Shqipam አማርኛar العربيةhy Հայերենaz Azərbaycan dilieu Euskarabe Беларуская моваbn বাংলাbs Bosanskibg Българскиca Catalàceb Cebuanony Chichewazh-CN 简体中文zh-TW 繁體中文co Corsuhr Hrvatskics Čeština‎da Dansknl Nederlandsen Englisheo Esperantoet Eestitl Filipinofi Suomifr Françaisfy Fryskgl Galegoka ქართულიde Deutschel Ελληνικάgu ગુજરાતીht Kreyol ayisyenha Harshen Hausahaw Ōlelo Hawaiʻiiw עִבְרִיתhi हिन्दीhmn Hmonghu Magyaris Íslenskaig Igboid Bahasa Indonesiaga Gaeilgeit Italianoja 日本語jw Basa Jawakn ಕನ್ನಡkk Қазақ тіліkm ភាសាខ្មែរko 한국어ku كوردی‎ky Кыргызчаlo ພາສາລາວla Latinlv Latviešu valodalt Lietuvių kalbalb Lëtzebuergeschmk Македонски јазикmg Malagasyms Bahasa Melayuml മലയാളംmt Maltesemi Te Reo Māorimr मराठीmn Монголmy ဗမာစာne नेपालीno Norsk bokmålps پښتوfa فارسیpl Polskipt Portuguêspa ਪੰਜਾਬੀro Românăru Русскийsm Samoangd Gàidhligsr Српски језикst Sesothosn Shonasd سنڌيsi සිංහලsk Slovenčinasl Slovenščinaso Afsoomaalies Españolsu Basa Sundasw Kiswahilisv Svenskatg Тоҷикӣta தமிழ்te తెలుగుth ไทยtr Türkçeuk Українськаur اردوuz O‘zbekchavi Tiếng Việtcy Cymraegxh isiXhosayi יידישyo Yorùbázu Zulu