• Skip to main content
  • Skip to footer
  • Home

The May 13 Group

the next day for evaluation

  • Get Involved
  • Our Work
  • About Us
You are here: Home / Archives for allblogs

allblogs

Aug 30 2021

Comment on Dissertation RQ2: To what extent do interpersonal factors relate to use beyond research factors? by Dissertation RQ3: How do researchers and evaluators differ in use, interpersonal factors, and research/evaluation factors?

[…] Question 2: To what extent do interpersonal factors relate to use beyond research factors? […]

Written by cplysy · Categorized: danawanzer

Aug 30 2021

Comment on Factors that promote use: A conceptual framework by Dissertation RQ3: How do researchers and evaluators differ in use, interpersonal factors, and research/evaluation factors?

[…] Factors that promote use: A conceptual framework […]

Written by cplysy · Categorized: danawanzer

Aug 30 2021

Dissertation RQ3: How do researchers and evaluators differ in use, interpersonal factors, and research/evaluation factors?

This blog post is a modified segment of my dissertation, done under the supervision of Dr. Tiffany Berry at Claremont Graduate University. You can read the full dissertation on the Open Science Framework here. The rest of the blog posts in this series on my dissertation are linked below:

  1. Factors that promote use: A conceptual framework
  2. Defining evidence use
  3. Overview of my dissertation study: sample, recruitment, & measures
  4. Question 1: To what extent are interpersonal and research factors related to use?  
  5. Question 2: To what extent do interpersonal factors relate to use beyond research factors?
  6. Question 3: How do researchers and evaluators differ in use, interpersonal factors, and research factors? 

I was also interested in seeing how use, interpersonal factors, and research factors differed between researchers and evaluators. This is somewhat of an extension from one of my research studies examining how researchers and evaluators define evaluation and differentiate evaluation from research (if at all).

Participants who self-reported themselves as evaluators rated relationships, communication, cooperative interdependence, and commitment to use higher than participants who self-reported as researchers. There were no differences between the two groups on research factors or evidence use.

I also compared correlations between interpersonal and research factors with evidence use between self-reported evaluators and researchers. Correlations for interpersonal factors and instrumental and conceptual use were generally stronger for researchers than for evaluators. Relevance was a stronger correlate of instrumental and conceptual use for evaluators than for researchers, whereas rigor was a stronger correlate of instrumental and process use for researchers than for evaluators. However, it should be noted that z-scores comparing correlation strength were not statistically significant.

Overall, although evaluators rated interpersonal factors higher than evaluators and no differences in reported use, there were stronger correlations between interpersonal factors and use for researchers. These findings suggest that evaluators and researchers are achieving the same outcomes (i.e., the same levels of instrumental, conceptual, and process use) but get to those outcomes differently. For example, researchers may see greater impact of interpersonal factors and rigor compared to evaluators whereas evaluators may see greater impact of relevance and stakeholder involvement.  

Written by Dana Wanzer · Categorized: danawanzer

Aug 27 2021

Museums during Existential Crises: Long and Wide Lenses for Directional Thinking

By: Emlyn Koster

An Earth scientist with 32 years at the helm of four large nature and science museums and with publications and presentations continuing to delve into what leadership of external relevance entails, Emlyn reflects on the potentially greater value of museums in a world beset by crises.

The period since early 2019 has been an unprecedented slog of stressful uncertainty for every type of organization. Yet with the museum concept inspired by The Muses―the goddesses of the arts and sciences in Greek mythology—museums should, in particular I believe, be striving to use long and wide lenses to shape their most valuable responses to today’s circumstances.

During the pandemic, understandably, online exchanges among museums have been much more about pressing operational matters than big-picture strategic matters. Also, published leadership-level musings have been rare. Each a commendable sharing of reflective thoughts, these include the perspectives of directors at a cultural anthropology museum, art and nature museum, a group of middle managers at four natural history and marine science museums, and two former science museum directors.

Wicked Problems

Before this decade, an ‘existential crisis’ described individuals consumed with anxiety about how they see themselves and their purpose. Announced as new vocabulary by Dictionary.com, it began to also be a whole society term as climate change, the Covid-19 pandemic, and structural racism dominated the news.

Introduced in the 1960s, the term ‘wicked problem’ has long struck me as a jarring label because wicked refers to a morally wrong stance. However, it seems apt for the situation of the Covid-19 pandemic and its variants because they have harmed our diverse society so deeply and inequitably. John Camillus of the University of Pittsburgh Graduate School of Business stated that wicked problems “can’t be solved, but [they] can be tamed” and that “solutions to wicked problems are not true or false, but good or bad”. Pathways to better times, he concluded, lie with continuous scanning of pertinent circumstances with assumption monitoring and adaptive decision-making.

A Long Lens

I have been surprised by the scarcity of references to motivational outlooks of the museum sector during the Great Depression, World Wars I and II, and after the September 11, 2001 terrorist attacks. Here are several examples. Presentations at the 1942 AAM Annual Meeting stated: “With every crisis, there comes a spiritual awakening, and also an intellectual advancement… Our institutions are not merely guardians of the past, but are factors in the building and molding of character for the future” (L. Hubbard Shattuck, 1942, Wartime duties of historical museums, Museum News, October 1, 6-8) and “The very subject of our discussion shows the painful anxiety and uncertainty with which we search for our proper function in the national struggle for a better future” (A.E. Parr, 1942, The Wartime duties of natural history museums, Museum News, September 1, 6-8). In a cautionary note, “When the funds began to flow again, museums quickly forgot the shock of the Depression as well as their moments of innovation on behalf of the public… An opportunity to be societal role models for the wisest use of resources and talent was lost” (Marjorie Schwarzer, 2009, Bringing it to the people: lessons from the first Great Depression, Museum, May-June, 49-54). Recalling an initiative by The Families of September 11 and Liberty Science Center: “Our careers and backgrounds are so dissimilar that we could barely imagine how our professional lives could ever intersect: a mental health professional specializing in trauma and a president of a science center… We committed to a vision for the future based on what we learned”.

A Wide Lens

A stack of leadership books against a plain background. The books have titles of key leadership words on their spines, like "Integrity."
Source

Also, I often lean on recommendations about organizational leadership that have passed the test of time. Here are several favorites. Aristotle (384-322 BC) viewed leadership as the harmonious pursuit of positive consequences in the world: significant, I think, because of his emphasis on linked internal and external dynamics. John Cotton Dana (1856-1929) advocated that museums align their activities with the needs of their communities: significant, I think, because he prioritized needs above wants. Stephen Covey (1932-2012) echoed Peter Drucker (1909-2005) in stressing the distinction between effectiveness and efficiency: respectively doing the right things and doing things right which they emphasized are the main responsibilities of leadership and management. John Kotter (1947- ), Emeritus Professor of Leadership at the Harvard Business School, listed these steps for a successful renewal:

  • Establish a sense of urgency
  • Form a powerful guiding coalition
  • Create a vision
  • Communicate the vision
  • Empower others to act on the vision
  • Plan for and create short-term wins
  • Consolidate improvements and produce still more change
  • Institutionalize new approaches

Because vision is a central thread, I also wish to cite a first page statement in a 1992 book titled ‘Visionary Leadership’: “There is no more powerful engine driving an organization toward excellence and long-range success than an attractive, worthwhile, and achievable vision of the future, widely shared”. A contrary view, which I also applaud because it encourages the weighing of options, has recently been advanced by a university art museum director.

Planning Parameters

A fair question in today’s chaotic world surrounds what is achievable? A quarter century ago, the 150th anniversary symposium of The Smithsonian Institution heard this admonition: “The mission statement of most museums which often states “Our mission is to collect, preserve and interpret fill-in-the-blank will no longer do. Such statements do not answer the vital question of ‘So what?’””. Arguably more than ever, museums need to specify their ‘so what?’ intentions in past-present-future contexts. This behooves museums to have less lofty missions with realistic content (exhibitions/programs/conversations) and audience (onsite/offsite/online) goals. In my view, this adaptive step becomes part of what continues to be a robust approach to strategic planning―namely, the following sequential, but needing to be frequently revisited, steps to distinctively define an organization and attract friends and funds to it:

  • Values: Enduring attitudinal traits, typically 5-7, that define the organization’s core culture and that must not be violated by any transaction or decision (in a new organization these are a mix of early realities and credible aspirations). 
  • Stakeholders: Those with vested interests in the well-being of the organization, both those who can affect and be affected by its performance, most notably its core audience(s).  
  • Mission:  Jargon-free, memorable, often one-sentence statement of the organization’s purpose / ‘so what?’ / raison d’être / beating heart in relation to external needs (a marketing tagline / slogan is a potential addition). 
  • Vision: Forward-looking, inspiring, one or two paragraph statement of a more advanced stage of the organization’s impacts. 
  • Strategy: High-level roadmap of the organization’s intentions as a guide to how value-conforming efforts will best advance the mission in the direction of the vision (there is definitional variation between several interrelated terms, including strategies, goals, key result areas, and objectives).

“Greatness is Not a Function of Circumstance”

Facing wicked problems requires bold and innovative leadership with a preparedness to revisit strategic plans for timely adjustments. In his manifesto for the social sector, Jim Collins (1958- ) concluded: “Greatness is not a function of circumstance. Greatness, it turns out, is largely a matter of conscious choice, and discipline”.

About the Author

Emlyn Koster, PhD was the CEO at Canada’s Royal Tyrrell Museum of Paleontology and Ontario Science Centre between 1986-96 and at NJ’s Liberty Science Center and the NC Museum of Natural Sciences between 1996-2018. Concurrently, he had advisory roles at the Getty and Noyce Leadership Institutes and attended ‘Strategic Perspectives for Nonprofit Management’ at the Harvard Business School. An ambassador for the International Coalition for Sites of Conscience and an adjunct professor at NC State University, he writes, speaks and advises on the Anthropocene which is transdisciplinary shorthand for human disruption of the Earth System. You can read his previous blog posts here.

The post Museums during Existential Crises: Long and Wide Lenses for Directional Thinking appeared first on RK&A.

Written by cplysy · Categorized: rka

Aug 27 2021

A gut check

A gut check

While I am lucky that I get to consult with schools and districts in their work with families, it isn’t often that I get to engage with students and families directly (especially since COVID derailed my weekly mentoring sessions at a high school here in Columbus).

So I was especially delighted this week, when, through my part-time work at Ohio State, I got the opportunity to begin a series of focus groups with kinship caregivers (grandparents, aunts/uncles, and other relatives or close friends who are the primary guardians for their relatives’ children).

Through this study, we are trying to learn how the pandemic has impacted kinship families and how we can advocate for the resources and supports they really need to improve the quality of life of the children in their care and themselves.

To me, this study is a great example of engaging with data – using our conversations with key stakeholders to guide future interventions and advocacy efforts. 

Well, the first focus group was last night, and what I heard really impacted me. 

I talk a lot about telling your story and how using qualitative data with numbers and stats can amplify the effect of our analysis.

This focus group reminded me of just how powerful that effect can be. 

You see, for my work at OSU, I’ve done a lot of digging into the literature about kinship care and the national statistics about how children in and around the child welfare system fare in different settings. 

Yet hearing a particular focus group participant share their story last night made it all come to life for me, in a way I haven’t experienced since my time working in schools. 

This caregiver shared the raw, painful emotions they were experiencing as a result of caring for multiple children with few supports but plenty of challenging family dynamics, fears, and frustrations. It was impossible to not be moved by their story.

From reading other studies about kinship care or even reviewing survey data from this and other studies, I knew that all of these factors were common for kinship caregivers. 

Yet this qualitative data – this person’s actual experiences shared in their own voice – helped me better understand just how real these phenomena are and just how important it is to use our findings to help improve lives.  

This was a gut check for me – a reminder to always seek the stories in our data, connect them to real people, and use my place of privilege to try to make their world a bit better. 

As we re-enter schools this fall in a once-again scary and uncertain time, remember that the lines in your spreadsheets and the names on your event sign-in sheets are real people, often living difficult lives in an especially challenging time. 

Thankfully, as educators, we have the ability to connect our students and families with resources and supports, if only we are able to hear their stories.

For more information on using qualitative data in your engagement work, check out some past posts here, here, and here.

Written by cplysy · Categorized: engagewithdata

  • « Go to Previous Page
  • Go to page 1
  • Interim pages omitted …
  • Go to page 167
  • Go to page 168
  • Go to page 169
  • Go to page 170
  • Go to page 171
  • Interim pages omitted …
  • Go to page 310
  • Go to Next Page »

Footer

Follow our Work

The easiest way to stay connected to our work is to join our newsletter. You’ll get updates on projects, learn about new events, and hear stories from those evaluators whom the field continues to actively exclude and erase.

Get Updates

Want to take further action or join a pod? Click here to learn more.

Copyright © 2026 · The May 13 Group · Log in

en English
af Afrikaanssq Shqipam አማርኛar العربيةhy Հայերենaz Azərbaycan dilieu Euskarabe Беларуская моваbn বাংলাbs Bosanskibg Българскиca Catalàceb Cebuanony Chichewazh-CN 简体中文zh-TW 繁體中文co Corsuhr Hrvatskics Čeština‎da Dansknl Nederlandsen Englisheo Esperantoet Eestitl Filipinofi Suomifr Françaisfy Fryskgl Galegoka ქართულიde Deutschel Ελληνικάgu ગુજરાતીht Kreyol ayisyenha Harshen Hausahaw Ōlelo Hawaiʻiiw עִבְרִיתhi हिन्दीhmn Hmonghu Magyaris Íslenskaig Igboid Bahasa Indonesiaga Gaeilgeit Italianoja 日本語jw Basa Jawakn ಕನ್ನಡkk Қазақ тіліkm ភាសាខ្មែរko 한국어ku كوردی‎ky Кыргызчаlo ພາສາລາວla Latinlv Latviešu valodalt Lietuvių kalbalb Lëtzebuergeschmk Македонски јазикmg Malagasyms Bahasa Melayuml മലയാളംmt Maltesemi Te Reo Māorimr मराठीmn Монголmy ဗမာစာne नेपालीno Norsk bokmålps پښتوfa فارسیpl Polskipt Portuguêspa ਪੰਜਾਬੀro Românăru Русскийsm Samoangd Gàidhligsr Српски језикst Sesothosn Shonasd سنڌيsi සිංහලsk Slovenčinasl Slovenščinaso Afsoomaalies Españolsu Basa Sundasw Kiswahilisv Svenskatg Тоҷикӣta தமிழ்te తెలుగుth ไทยtr Türkçeuk Українськаur اردوuz O‘zbekchavi Tiếng Việtcy Cymraegxh isiXhosayi יידישyo Yorùbázu Zulu