• Skip to main content
  • Skip to footer
  • Home

The May 13 Group

the next day for evaluation

  • Get Involved
  • Our Work
  • About Us
You are here: Home / Archives for allblogs

allblogs

May 04 2021

Schrodinger’s Outcome

Today’s post is a little evaluation graphic novella inspired by Schrödinger’s cat. No cats, real or fictional, were harmed while creating this post. I’ll drop a plain text version at the bottom.

Shrodinger's Outcome Graphic Novella by Chris Lysy of freshspectrum.com.  Illustration 1.
Shrodinger's Outcome Graphic Novella by Chris Lysy of freshspectrum.com.  Illustration 2.
Shrodinger's Outcome Graphic Novella by Chris Lysy of freshspectrum.com.  Illustration 3.
Shrodinger's Outcome Graphic Novella by Chris Lysy of freshspectrum.com.  Illustration 4.
Shrodinger's Outcome Graphic Novella by Chris Lysy of freshspectrum.com.  Illustration 5.
Shrodinger's Outcome Graphic Novella by Chris Lysy of freshspectrum.com.  Illustration 6.
Shrodinger's Outcome Graphic Novella by Chris Lysy of freshspectrum.com.  Illustration 7.
Shrodinger's Outcome Graphic Novella by Chris Lysy of freshspectrum.com.  Illustration 8.
Shrodinger's Outcome Graphic Novella by Chris Lysy of freshspectrum.com.  Illustration 9.
Shrodinger's Outcome Graphic Novella by Chris Lysy of freshspectrum.com.  Illustration 10.
Shrodinger's Outcome Graphic Novella by Chris Lysy of freshspectrum.com.  Illustration 11.
Shrodinger's Outcome Graphic Novella by Chris Lysy of freshspectrum.com.  Illustration 12.
Shrodinger's Outcome Graphic Novella by Chris Lysy of freshspectrum.com.  Illustration 13.
Shrodinger's Outcome Graphic Novella by Chris Lysy of freshspectrum.com.  Illustration 14.
Shrodinger's Outcome Graphic Novella by Chris Lysy of freshspectrum.com.  Illustration 15.
Shrodinger's Outcome Graphic Novella by Chris Lysy of freshspectrum.com.  Illustration 16.

Plain Text Version

Schrodinger’s Outcome

Imagine a nonprofit program. Let’s say a food pantry.

The food pantry was created to provide free emergency food assistance to residents within a specific city or town.

It receives funding in the form of donations and grants.  It also draws volunteers from the surrounding area.

Is it successful?

Maybe yes, maybe no.

Maybe yes and no.

There is a point to which every activity and program exists as both wildly successful and a total failure.

We can reduce that uncertainty by explicitly defining the program activities, scope, and outcomes.  Then measuring.

But if you already see the program as wildly successful, why measure?

Would you believe it if an evaluator showed you evidence that the program wasn’t a success?

And if you already see the program as a total failure, why measure?

Would you believe it if an evaluator showed you evidence that the program wasn’t a failure?

The biggest challenge in evaluation isn’t defining, measuring, collecting, or interpreting evidence.

The biggest challenge is defining, measuring, collecting, and interpreting evidence while developing enough stakeholder trust in your process and your team that your analysis will actually be heard and believed.  

Evaluation is not just answering questions.  

But confirming or rejecting the things we think we do know.

And then sharing that information so that others will see what we see.

Written by cplysy · Categorized: freshspectrum

May 04 2021

Meet Engage with Data!

Meet Engage with Data!

Let me be real for a second: when I started Structured Solutions five and a half years ago, I was definitely building the plane as I flew it.

So when I had to choose a name, I came up with one that I thought sounded catchy (read: it used alliteration) and conveyed a message about how I operated.

As the years went on, I became increasingly unhappy with my business name. It just didn’t feel like me. 

So today, I am THRILLED to announce that my business has a new name: Engage with Data!

Engage with Data Logo

To me, this name represents two of my core beliefs:

1) That it’s critical for educators to use their data to guide their family engagement efforts; and 

2) That we should all get our hands dirty and really dig in to our data. 

The heart of my company hasn’t changed – the work is still the same, and so is my mission – to help educational organizations use the data they have to tell the story of the impact they’re having on children and families. 

Here’s what has changed:

I have worked hard to make my new brand a lot more fun and authentic to who I am and how I operate. 

I would love for you to explore (and share!) the new website: www.engagewithdata.com.

I hope that as you do, you’ll see a lot more of me shining through.

To those who have been helping me rebrand and get ready for today, I am truly grateful. 

To those who read my blogs and support my work, THANK YOU! Engage with Data would be nothing without you.

I am so excited to see what this new chapter in my business brings. 

Written by cplysy · Categorized: engagewithdata

May 03 2021

Improving Our Museum Labels Through A Harm Reduction Lens: Part 3

By: Rachel Nicholson, Jocelyn Edens, and Ariana Chaivaranon

In our workshops with curatorial colleagues (which we wrote about in the last post), we continually heard certain ideas rise to the surface about shared principles for interpretive text at the Nelson-Atkins. These included:

  • Complexity: a label can be an invitation to explore the object and ideas further and does not need to offer a resolved story.
  • Specificity regarding the object and language.
  • Empathy especially for visitors, subjects, and makers who have been disempowered.
  • Making deliberate choices to share the most interesting and relevant stories of an object.
  • Thinking holistically about the experience of visiting a gallery and of visiting the museum.

Working off of these initial ideas, we as an Interpretation Team decided to divide and conquer, each taking on a different collection area and working with Curators to reimagine the labels they had identified. Each of us used “story jams” to explore new ideas for these objects. This model was shared with us by Antenna, an audio and multimedia production company with whom we work.

A screenshot of a PowerPoint Slide with a blue band at the left and a white background. Black text on a white background to the right outlines the story jam questions.
For story jams, we used the above questions to guide conversations, asking people to make observations first as a visitor or someone seeing the object for the first time, and then as an expert.

In each story jam, of which there were 3-4 for each collection area, we gathered colleagues and volunteers across the institution with different areas of expertise and experiences with art including visitor services officers, access staff, and membership and social media specialists.  Working collaboratively, we identified the most interesting and pressing stories that each object could tell. While our overall approaches were similar, there were also some differences since we worked with different colleagues and collections. Therefore, for this post, we’ll each share a bit about our process and our learnings.

Jocelyn Edens, Interpretive Planner, worked with objects in our Chinese Art collection, focusing in one case on how to tell complex stories about objects that may have been reconstructed and are therefore not “original.” Ariana Chaivaranon, Interpretive Planner, focused on our South and Southeast Asian Art collections, confronting questions of how to help visitors understand how an object’s context can change its reception. Lastly, Rachel Nicholson worked with objects in the European Art collection, tackling questions of violent subject matter and how to acknowledge 21st century experiences and understanding of topics such as sexual assault in the context of 17th century paintings.

Adding Complexity to our Chinese Art Collection

In one story jam for objects in the Chinese collection, we spent a lot of time discussing a limestone relief carving made in the Northern Wei Dynasty, around 522 C.E. Our curator identified the label for this object as needing a re-write for a few reasons: 1) it was physically worn and dirty, 2) it privileged stories about men that aren’t even visible rather than focusing on the women in the object, and 3) it was a slog for visitors to read at 346 words long (we aim for 70-90 words for object labels). Crucially, new research showed that a large portion of the panel had been reconstructed in the 1930s, so writing a new label would give us an opportunity to be transparent with visitors about the object’s the life.

Offering Procession of the Empress as Donor with Her Court, Chinese, about 522 C.E. Fine, dark-gray limestone; 80 inches x 9 feet 1 ½ inches. Purchase: William Rockhill Nelson Trust. 40-38

In discussion, we first raised questions about the object based on looking. Most of these questions focused on decoding the composition, the original context of the object, and its materials and techniques. When our curator introduced the story of its reconstruction, new questions and observations emerged: are objects in other museums from the same site in the same condition? Who were the artists who helped create this reconstruction? What can we glean about their mastery and skill, alongside that of the original carvers? What other objects in our collection can tell stories about copies and reconstructions?

As we develop a new label that incorporates this story and leans into complexity—that is, offers unresolved stories that invite more questions and further exploration—our big question is how it will affect the way visitors make meaning from this object, as well as others near it in the gallery. Will visitors feel empowered to ask new questions and explore these objects in all their complexity? Will they feel deceived or tricked or disappointed?

Specificity in the South and Southeast Asian Art Collection

The story jam participants for the South and Southeast Asian collection tackled some of the core tensions of the display of Hindu processional sculptures in U.S. encyclopedic art museums. An early 1200s sculpture of Shiva Nataraja is the centerpiece of one of the fictive “temple rooms” original to our 1932 building. Although non-Hindus often mistake the room for a temple reconstruction, the object display is far removed from its original religious context. Barring the opportunity for reinstallation, we set out to support both Hindu and non-Hindu visitors’ experiences through the label.

A photo of a dark gray, bronze statue against a gray background. The god Shiva raises his right leg and extends his four arms in a dance.
Shiva Nataraja, Lord of the Dance, Indian, early 13th century. Bronze; 34 ¼ x 27 ½ x 13 inches. Purchase: William Rockhill Nelson Trust, 34-7.

At first, story jam participants without a knowledge of Hinduism commented on the sculpture’s perfectly cast and precisely proportioned bronze body. Their interests shifted when Hindu story jam participants or those familiar with Hindu practices noted that to them, Shiva appeared undressed in the museum. In its original home in a South Indian temple, the sculpture would have been adorned daily with oils, cloth, and flowers. Our revised label invites visitors to reflect on the differing practices of seeing the sculpture in the museum context, in a temple, and in a religious festival procession.

The revised label also prompts visitors to consider the embodied experience of aspects of Hindu worship, such as dance, moving around the object, and darshan (mutual seeing between a worshiper and a sacred image). We replaced unspecific, harmful language that incorrectly described Apasmarapurusha, the figure crushed by Shiva’s dancing feet, as a “dwarf,” invited directed looking, and used non-ableist verbs to encourage visitors to “circle” the sculpture, taking it in from all angles. Ultimately, we hope to inspire visitors to reflect on their personal, spiritual, and bodily relationships to this sculpture of Shiva today.

Thinking Holistically about our European Art Collection

Our European art curators identified about 30 labels to be replaced. Rather than workshopping all of them, we broke them down into themes and chose objects that embodied each big idea. These themes included: violent subject matter, unacknowledged power dynamics, and harmful tropes.

For one specific label, we tackled a subject common in many European art collections: Europa and the Bull. In this story from the ancient Roman poet Ovid’s Metamorphosis, the god Jupiter transforms himself into a bull, seduces and abducts Europa. Our current label focused mostly on the artist and his style and did little to address the story. What arose from our story jam, however, were lots of questions about the scene itself: why is the bull the central figure? Who is the victim in this story? Why does the bull look “Disneyfied”?

Bernardo Cavallino and follower (Johann Heinrich Schönfeld? 1609-1683) (1616-ca. 1656). Europa and the Bull, about 1645. Oil on canvas; 24 x 31 13/16 inches. Purchase: William Rockhill Nelson Trust, 31-50.

First, we spent time looking closely at the painting and noticed that the bull is looking directly at viewers and almost winking. For many in the room, this felt like the way into the story. We could start with the specific object, move to the story of how Jupiter tricked Europa, and then touch upon broader ideas around gender and power dynamics in myths. Our curator shared that the title of the painting had also changed throughout time, from The Abduction of Europa to Europa and the Bull, opening a conversation about an object’s life and how our understanding of these myths and paintings can shift.

We also realized that the broader discussion of power dynamics would be better to include in a section panel in the gallery, near an entrance. This would allow us to address these ideas upfront, without having to then repeat ourselves on every specific object label in the gallery that dealt with a similar myth. In creating a broader theme panel, we could spend time in specific object labels focusing on the work of art itself. This push to think holistically about a whole gallery experience rather than just one label helped alleviate some pressure on all object labels, opening space to touch on big themes while also being specific in our interpretation of specific objects.

What’s next?

This process continues to evolve and right now we are in the midst of codifying label principles and evaluating these new labels. We hope to share more about where we’re headed in the next post.

About the Authors

Rachel Nicholson is the Director, Interpretation, Evaluation & Visitor Research at the Nelson-Atkins Museum of Art.  You can reach her at rnicholson@nelson-atkins.org. Every two weeks throughout April and May 2021, Rachel will share her team’s efforts to rewrite the Nelson-Atkins’ permanent collection gallery labels through a harm reduction lens. Read her first two posts here.

Jocelyn Edens, is an Interpretive Planner at the Nelson-Atkins Museum of Art.

Ariana Chaivaranon is an artist and an Interpretive Planner at the Nelson-Atkins Museum of Art. Born in Thailand, Chaivaranon studied Visual and Environmental Studies and the History of Art at Harvard. Chaivaranon is a board member of Plug, a curatorial collaboration and exhibition space supporting emerging artists in Kansas City.

Don’t want to miss a post? Subscribe to our mailing list (just fill out the form at the right)! 

The post Improving Our Museum Labels Through A Harm Reduction Lens: Part 3 appeared first on RK&A.

Written by cplysy · Categorized: rka

May 02 2021

Evaluation Roundup – April 2021

Welcome to our monthly roundup of new and noteworthy evaluation news and resources – here is the latest.

Have something you’d like to see here? Tweet us @EvalAcademy!

New and Noteworthy — Reads

Shifting the Evaluation Paradigm: The Equitable Evaluation Framework

Hot of the press! The Equitable Evaluation Initiative in partnership with Grantmakers for Effective Organizations just released a new publication called Shifting the Evaluation Paradigm: The Equitable Evaluation Framework. This new joint publication provides an overview of the Equitable Evaluation Framework (EEF) and the early steps of engagement with foundation partners. The publication highlights insights in mindset shifts and tensions that are occurring behind the curtains of six foundation practice partners.

Increasing the Value of Foundation Evaluation

Engage R+D partnered with the Bill and Melinda Gates Foundation to apply a creative, participatory technique – known as a design charrette – to engage a broad variety of stakeholders in collaboratively designing a summative evaluation of one of the Foundation’s signature K-12 investments. Through this case study a learning brief was developed that explores “what can happen when more voices are included in the process of evaluation design.”

How to Teach Evaluation

The Canadian Journal of Program Evaluation recently released a special issue on how evaluation can be taught and learned. The issue curates the Practice Notes of evaluation educators from diverse backgrounds. If you are looking for evaluation teaching techniques, then definitely check out this volume from the Canadian Journal of Program Evaluation.

Real-Time Evaluation – Your Time Has Arrived!

Patricia Rogers of BetterEvaluation wrote a working paper on what can be learned from previous Real-time evaluations (RTE) practiced over the past 20 years. Our world is changing quickly, and people are looking for evaluation approaches that will meet the needs of rapidly changing initiatives. For an overview of why we need more real-time evaluation, read her most recent article on BetterEvaluation.

New and Noteworthy — Events

Canadian Evaluation Society 2021 Virtual Conference

Organized by: Canadian Evaluation Society (CES)

Date: May 9 -14 (Workshops from April 30)

Methods Masterclass

Organized by: Canadian Evaluation Society (Ontario Chapter)

Dates: May 11, 1:15 – 2:15 pm

Topics:

  • Learning Outcome Harvesting Experientially  

  • Five Quick and Easy Tips for Better Data Visualization  

  • Performance Measurement: From Theory to Practice to Transformation  

  • Visual Methods for Developmental Evaluation and Learning  

  • Expanding Online Data Collection Beyond Surveys

Transformation: What it is. Why it matters. How to evaluate it.

Organized by: Community-University Partnership for the Study of Children, Youth, and Family

Dates: May 19, 12:00 – 1:00 pm MDT

Facilitators: Michael Quinn-Patton & Mark Cabaj

Courses

Feminist Evaluation: Not your standard gender-responsive approach

Instructor: EnCompass

Start Date: May 25, 2021 (12:00 pm EST)

Written by cplysy · Categorized: evalacademy

May 02 2021

9 Tips to Make the Most Out of Your Virtual Conference Experience

 

In a year that saw massive and rapid shifts to online events and virtual conferences, gone are the days of rolling up to conference hotels, resigning yourself to dry pastries and tiny cups of coffee at snack breaks, and rushing endlessly down a maze of patterned carpet hallways asking random passerby’s “do you know where conference hall A is?”

pexels-marcus-aurelius-4064696.jpg

The world has changed and so has our conferencing experience. In some ways, there are perks – you can sleep in your own bed, make yourself a large and delicious cup of coffee, and munch away on breakfast during the opening remarks from the warmth of your own home. But without careful planning, you can easily end up in the same harried conference cycle of sprinting to find food and use the washroom between sessions, staying up late trying to manage work emails and deadlines while attending conference sessions all day without the benefit of room service and the camaraderie of like-minded individuals buoying your spirits with random small talk and chance encounters to talk shop. 

While we aren’t able to stop your family and pets from ambling by and needing something from you in the middle of an engaging session, we do have some tips to help you maximize your virtual conference experience. Here are 9 things you can do to maximize your virtual conference experience. 

 

Before the Conference

1. Plan which sessions you want to attend.

Yes, you do this for in-person conferences as well, but virtual conferences are a little different. Prioritize interactive sessions over more static presentations if sessions are being recorded. Since you can easily switch between sessions with the click of a button, plan a backup session in case your first choice is full (it can happen with virtual rooms!) or if the session turns out to be completely different than advertised. While you don’t have to sheepishly slink out the door to leave a session, remember that you are still taking up a virtual spot and presenters can often see when people leave the online meeting.

2. Set an out-of-office message.

Sure, you’re at your computer all day and likely aren’t conversing with colleagues over a buffet lunch, but give yourself the grace and space to focus during the conference. Set an out-of-office message, even if it is just to tell coworkers and clients that you won’t be responding to emails in your usual timely fashion and relieve the pressure you put on yourself to keep with work.

3. Prepare your snacks in advance.

Don’t get caught thinking that 15 minutes in between virtual sessions will be enough time to use the washroom, finish up your notes, make a snack, let the dog out, change over a load of laundry, answer a few emails, and hop into your next zoom session. Conferencing from home comes with its own set of tasks that eat up the time between sessions, don’t let preparing food become one of those things. Set up a snack station or at least buy some easy-to-grab things to eat while you conference away.

4. Set up a conference space that is separate from your workspace.

Our brains associate certain tasks with specific spaces. Setting up in your usual workspace puts your brain in ‘work mode’ and makes you more likely to tune out of the conference and tune into work. While not everyone is privileged to have even one distraction-free space to work in, let alone a second space they can “conference” from, small changes, like using a different side of the desk, changing up what you look at, using a different notebook or a different coloured pen can help your brain switch into conference mode. 

At the last virtual conference I attended, I literally switched which side of the kitchen table I sat on for “work” vs “conferencing” and used a different notebook, pens, and mug. 

During the Conference

5. Use technology to your advantage.

Many virtual platforms allow you to screenshot which can help you to capture information quickly or refer to an image at a later date. As a bonus, you can quickly share your screenshots on Twitter (if that’s your thing).

6. Get outside.

Zoom fatigue is real. Use your lunch break or set aside some time to get outside each day. Staring at a screen for conferencing, and for the work you may be doing around conference sessions, is seriously draining. Make sure to move your body, since you are no longer hustling down long hotel corridors between sessions. 

 

7. Make a point to connect with other attendees.

It can be easy to stick to your own bubble during a conference. Being attached to a computer and video camera all day is draining and the last thing some of us want to do is schedule one more video interaction after a day of staring at our screens. Take advantage of the many networking opportunities that virtual conferences are offering and make at least one connection. If you’re too exhausted from the conference itself, set up a time to connect a week or so out when you aren’t as drained.

After the Conference

8. Schedule a time to watch on-demand content.

In some cases, virtual conference content can be watched on-demand or is available after the conference. Purposefully schedule time in your calendar to watch the content or you’ll walk away with a folder full of bookmarks and wish lists that quickly gets forgotten in the flow of regular to-do’s. Capitalize on the opportunity for archived content that has come along with virtual conferencing. 

 

9. Review conference material and pull out key highlights.

In these weird, hazy pandemic days where time slides by with few of its normal markers, you can quickly lose sight of the key points or ideas you learned at the conference. Put a reminder in your calendar and set aside time to review your notes, resources, and key take-aways. Create actionable steps to take what you learned and put it into practice. This includes reaching out to those you connected with or presenters you had questions for.

Eval Academy will be at the upcoming Canadian Evaluation Society conference. Tweet along with us on Twitter or connect with us at our virtual booth! We love talking to our eval pals.

 

To learn more about applying evaluation in practice, check out more of our articles, or connect with us over on Twitter (@EvalAcadmey) or LinkedIn.


Sign up for our newsletter

We’ll let you know about our new content, and curate the best new evaluation resources from around the web!


We respect your privacy.

Thank you!


 

Written by cplysy · Categorized: evalacademy

  • « Go to Previous Page
  • Go to page 1
  • Interim pages omitted …
  • Go to page 188
  • Go to page 189
  • Go to page 190
  • Go to page 191
  • Go to page 192
  • Interim pages omitted …
  • Go to page 310
  • Go to Next Page »

Footer

Follow our Work

The easiest way to stay connected to our work is to join our newsletter. You’ll get updates on projects, learn about new events, and hear stories from those evaluators whom the field continues to actively exclude and erase.

Get Updates

Want to take further action or join a pod? Click here to learn more.

Copyright © 2026 · The May 13 Group · Log in

en English
af Afrikaanssq Shqipam አማርኛar العربيةhy Հայերենaz Azərbaycan dilieu Euskarabe Беларуская моваbn বাংলাbs Bosanskibg Българскиca Catalàceb Cebuanony Chichewazh-CN 简体中文zh-TW 繁體中文co Corsuhr Hrvatskics Čeština‎da Dansknl Nederlandsen Englisheo Esperantoet Eestitl Filipinofi Suomifr Françaisfy Fryskgl Galegoka ქართულიde Deutschel Ελληνικάgu ગુજરાતીht Kreyol ayisyenha Harshen Hausahaw Ōlelo Hawaiʻiiw עִבְרִיתhi हिन्दीhmn Hmonghu Magyaris Íslenskaig Igboid Bahasa Indonesiaga Gaeilgeit Italianoja 日本語jw Basa Jawakn ಕನ್ನಡkk Қазақ тіліkm ភាសាខ្មែរko 한국어ku كوردی‎ky Кыргызчаlo ພາສາລາວla Latinlv Latviešu valodalt Lietuvių kalbalb Lëtzebuergeschmk Македонски јазикmg Malagasyms Bahasa Melayuml മലയാളംmt Maltesemi Te Reo Māorimr मराठीmn Монголmy ဗမာစာne नेपालीno Norsk bokmålps پښتوfa فارسیpl Polskipt Portuguêspa ਪੰਜਾਬੀro Românăru Русскийsm Samoangd Gàidhligsr Српски језикst Sesothosn Shonasd سنڌيsi සිංහලsk Slovenčinasl Slovenščinaso Afsoomaalies Españolsu Basa Sundasw Kiswahilisv Svenskatg Тоҷикӣta தமிழ்te తెలుగుth ไทยtr Türkçeuk Українськаur اردوuz O‘zbekchavi Tiếng Việtcy Cymraegxh isiXhosayi יידישyo Yorùbázu Zulu