[…] Factors that promote use: A conceptual framework […]
allblogs
Dissertation: Overview of the study
This blog post is a modified segment of my dissertation, done under the supervision of Dr. Tiffany Berry at Claremont Graduate University. You can read the full dissertation on the Open Science Framework here. The rest of the blog posts in this series on my dissertation are linked below:
- Factors that promote use: A conceptual framework
- Defining evidence use
- Overview of my dissertation study: sample, recruitment, & measures
- Question 1: To what extent are interpersonal and research factors related to use?
- Question 2: To what extent do interpersonal factors relate to use beyond research factors?
- Question 3: How do researchers and evaluators differ in use, interpersonal factors, and research factors?
This study is a good example of working with what you end up with. The original methods of this study failed miserably, and we had to shift the methods so at least we ended up with something.
Originally, I sought to recruit practitioners through researchers and evaluators asking their practitioner partners. That would also aid in having actual partnerships represented. Wow did that fail miserably! In the end, I ended up with about 11 paired partners, but that was not enough given the methods I used.
Fortunately, I recruited through both the American Evaluation Association (AEA), and American Educational Research Association (AERA), as well as a bit through existing RPPs online and through social media. I ended up with roughly even numbers of researchers (n = 94), evaluators (n = 116), and practitioners (n = 82) after a meager ~13% response rate and roughly 50% completion rate.
The survey in this study went through the Questionnaire Appraisal System and cognitive interviews (n = 6) for pretesting. If you’re interested in what changed as a result of both those processes, check out the appendices in the full dissertation.
The survey first entailed asking participants whether they primarily identified as a researcher, evaluator, or practitioner in the particular partnership work they would talk about. Definitions were provided so it was clear what I meant by these terms. Next, to prime participants to think about their partnership, they were asked to provide some demographic information about their partnership, including sector, location, number of members, how long it had been together, and the purpose of the partnership.
Third, they were provided three scales on evidence use: one on instrumental use using a mixture of items from three separate questionnaires, one on conceptual use by NCRPP (2016), and one on process use that was a mixture of items from two separate questionnaires. Then participants completed scales on interpersonal factors: relationships (High Quality Work Relationships Scale by Carmeli et al., 2009), communication (combined scale), commitment to use (self-created), cooperative interdependence (adapted from Johnson & Norem-Hebeisen, 1979), and stakeholder involvement (adapted from Weaver & Cousins, 2004). Last, they completed two scales on research factors: relevance and rigor, both of which were self-created.
Finally, participants responded to more personal and partnership demographics, such as their position in the partnership, level of involvement and decision-making, education level, and more. The survey took participants a median 23 minutes to complete, making the survey a bit too long and likely contributed to the low participation and completion rates.
Most participants worked in education (57%), in the United States (88%), and had been in partnership for 5 or more years (41%). The primary purpose of the partnerships was either to conduct and use rigorous research (61%) or impact local improvement efforts (26%).
Want more details on the internal consistency of the scales, the items in the scales, or the sample characteristics? Check out Chapter 2 and the appendices in the full dissertation.
Gen Z are Identity Crafters: What does this mean for Cultural Institutions?
By: Sadiya Akasha
Population demographics have been shifting towards greater diversity for the last few generations with Generation Z being the most diverse generation yet. Research published by the Pew Research Center shows that nearly 48% of Gen Zers born between 1997 and 2012 belong to a racial or ethnic minority, compared to 39% of Millennials. As cultural institutions pivot towards serving this generation of visitors, it seems like a particularly salient time to query how this change in demographics might affect strategy, planning, and outreach for visitor engagement.
Researching and Crafting Identities
Many Gen Zers find themselves struggling with an outdated demographic model where they are expected to keep their race as clearly delineated as a shorthand way to describe themselves. In 2019, Vice Magazine led a research study enrolling older members of their Gen Z readership (aged 16 – 22) to answer questions about identity. Their findings suggest that this generation experiences many aspects of their identity, like racial and ethnic background, gender, and faith, as both more fluid and more holistic than our Western culture currently accommodates. For example, a study participant of Filipino and Korean American descent described her difficulty in simply checking a demographic box on forms:
“I remember for the longest time Filipino wasn’t an option,” she said. “And so my sister and I thought that we were Pacific Islanders just because of what the options were. I really don’t like the fact that you either have to be Hispanic or not Hispanic. Also, what does Hispanic even mean? The Spanish colonized the Philippines, so for you to say that I’m Filipino but not Hispanic or Hispanic, it’s so weird. The entire language needs to be changed.”
Beyond being multicultural, members of Gen Z also exhibit a flexible attitude with their adoption of religious traditions. Gen Zers approach their relationship with religion by researching the faiths of their living family members, and even their ancestors, in order to identify the elements that best align with their values and create an entirely new blend of religious rituals to better suit their needs. In this fashion, members of Gen Z are able to craft a well-researched and well-rounded identity that incorporates many of their cultural and religious practices into their daily lives.
But Gen Z doesn’t stop there! Since their identity is molded around their values, activism is also a part of their self-identity. The Pew Research Center has identified that a much larger percentage of Gen Z are born to immigrant parents in the United States when compared to Millennials. This means that on top of being more diverse, Gen Z is also less likely to be foreign-born. Early signs indicate that Gen Z is already playing an active role in civic engagement. The Vice Magazine study conducted research across the US and UK and found that 76% of Gen Z identify as activists and 54% plan to get involved in politics. It’s easy to see that for the generation whose members include the likes of Malala Yousafzai, X Gonzalez, and Greta Thunberg, activism and civic engagement is as much a part of their identity as their chosen gender, culture, religion, and so on.
Dynamic Intersectionality in Action
For Gen Z, all of these identifiers are not a hierarchical list but a multifaceted, self-consistent exposition on who they are becoming at present. For Gen Z, identity is actively shaped and carved like a sculpture out of stone, not accepted as handed down or absorbed through our cultural milieu. This kind of exploration and constant self-examination is such a foundational trait to Gen Z that it is clearly reflected in their overwhelming support of others’ rights to self-designation and exploration. The Culture Marketing Council released a comprehensive study on Gen Z in 2020 showing that values of freedom, equality & healthcare for all matter most to Gen Z, with heightened priority on the “For All”.
Gen Z is the most racially diverse and multiethnic generation to date. Their multicultural background is different from that of their predecessors in that many are US-born to immigrant parents rather than foreign-born themselves, and that they are much more likely to come from a mixed racial and multicultural background. This is reflected in a sense of self that is much more blended and holistic than our Western cultural model currently supports. So if we accept that Gen Z naturally inhabits an intersectional space with a flexible but intentional sense of identity that is ever-evolving, then we might need to re-consider current demographic data collection tools and their underlying assumptions.
Redefining Demographics
I recently collaborated with a prominent modern art museum in London to perform a user study targeting young people (high school and college-age) as well as families with young children (Gen Z and younger). The demographic data for the interviewees listed their ethnic background, in singular, such as Caribbean or Indian or Bengali. Curiously, I noticed interviewees who were not People of Color (POC) were listed as either ‘British’ or ‘White Other’. The idea here was to seek out diverse visitors within the target audience to ensure representation and inclusion for non-English speaking communities. However, what I learned during the interviews was that many of the interviewees who were identified by the museum as POC were in fact London-born Gen Z children of immigrant parents for whom the British culture and English language were an inseparable part of their identity. On the other hand, several of the interviewees who were listed as ‘White Other’ by the museum were first-generation immigrants themselves from other European nations who learned English as a second language in adulthood. It certainly turned some assumptions upside down about who was ‘British’ and which communities needed to be supported in their understanding of interpretive material.
It would be prudent at this point to question our assumptions and rethink our approach to racial demographics. It is clearly out of date and largely irrelevant to the lived experiences of our Gen Z visitors. We should start by asking Gen Z how they would like to be identified and go from there.
What’s next?
In my previous post in this series, I described how Gen Z actively investigates media narratives and questions sources, while in this post I’ve outlined that they take the same approach in crafting their own identities. With this in mind, my final post will investigate how cultural institutions can meet Gen Z where they are by engaging with them as collaborators.
About the Author

Sadiya Akasha is the co-founder and Director of Product Development at Sitara Systems, a design and technology laboratory that creates interactive experiences with emerging technologies. Sadiya partners with cultural institutions to help them conceptualize and deliver technology initiatives by leveraging her background in human-centered design, agile thinking, and audience research. In her free time Sadiya enjoys exploring the rugged yet delicate landscapes of the great Southwest.
The post Gen Z are Identity Crafters: What does this mean for Cultural Institutions? appeared first on RK&A.
Ask Nicole: Hire a Consultant vs. Do it Yourself
Have a question you’d like to be featured? Let me know. I had a discovery call with a prospective client recently, who wants to build an evaluation framework for their staff to implement. The executive director wanted an evaluation framework that was general enough to cover each program but could be tailored to each program’s […]
The post Ask Nicole: Hire a Consultant vs. Do it Yourself appeared first on Nicole Clark Consulting.
Comment on Factors that promote use: A conceptual framework by Defining evidence use
[…] Factors that promote use: A conceptual framework […]