rka

We have been working to actively post on our blog over the past few months in response to how COVID-19 is affecting the museum field.  However, considering the unjust deaths of George Floyd, Breonna Taylor, and others, and in solidarity with the protests unfolding daily against systemic racism in our country, we will be pausing our normal content.  Please listen to the voices of people of color and those advocating for black lives—for example, the National Museum of African American History and Culture’s web portal to support conversations about race.  Black Lives Matter.

The post Black Lives Matter appeared first on RK&A.

As evaluators, our work depends on being able to gather data on and from museums’ audiences.  It probably goes without saying that this has become trickier in a world transformed by a global pandemic.  Museums and cultural organizations are temporarily closed, and we anticipate lower visitation (and different visitation patterns) once they reopen.  This makes the question of where and how we might connect with study participants especially pertinent.  One thing we have been asking ourselves at RK&A is: do we really need to be physically in a museum to collect useful audience data?

The short answer is no.  While historically, museums have, for the most part, heard from their audiences through what evaluators call onsite research, a well-designed remote research study can provide just as thoughtful, valid, and reliable findings as one that gathers data from visitors recruited on the museum floor.

As the name suggests, remote research is research conducted offsite (not at a museum), with the researcher and study participants in different physical locations.  This may sound strange, new, or maybe even scary if you are used to gathering data from audiences in-person, at the museum.  But, stay with me!  While it would certainly be easy to dismiss the idea of doing audience research remotely as simply a temporary research solution while we are at home due to COVID-19, there is robust precedent for doing fully remote studies in “normal times.”  Many of these examples stem from the user experience (UX) and product development industries, but museum evaluators also collect data remotely when it best fits the needs of a project.  For instance, we might contact participants after a program for telephone interviews to hear about their experiences.  As long as a museum’s audience research studies are still rigorously designed and ethical research principles are upheld, there is no reason why they cannot be done fully remotely.

In fact, there are many reasons why museums might choose remote audience research as a primary research strategy over onsite research, even after museums begin to reopen.  For one, since participants are recruited in advance for a remote study, they come to the table more focused and mentally prepared to provide feedback, which is not typically the case when they are recruited on the busy museum floor.  Participating from home also helps people feel more comfortable, and this feeling can ultimately translate into more honest feedback by reducing courtesy bias (when people withhold critiques for fear of offending the museum or researcher), which can be high in an in-person study.  Further, remote studies can allow for broader and more diverse voices to be heard than might be possible when recruiting only among visitors in the museum’s building.  Of course, there are some disadvantages to doing fully remote studies, including the lack of environmental context from not being physically in the museum and, to some extent, privacy concerns.

Remote audience research is a huge topic and there is a lot to consider when deciding if it is the right approach for you (more than I can possibly fit in this post!).  With that in mind, RK&A created a guide to help you understand the rationale behind remote research for museums, what types of studies work best remotely, the advantages and disadvantages of remote research, and the best ways to collect data remotely.  Download RK&A’s Guide to Remote Audience Research for Museums.

Shortened link: https://bit.ly/3dnyOG9

The post Remote Audience Research for Museums — Download the Guide! appeared first on RK&A.

Upon commencing a study that requires intercepting visitors on the museum floor, we are often asked by staff (sometimes nervously), “Will people say yes to participating in a survey, interview, etc.?”  Our answer is always yes, and we often state that you would be surprised how many agree to participate.  Generally about 65 to 80 percent of people that we recruit (in person, at a museum) will say yes to participating in research and evaluation.  A podcast I listened to recently from Hidden Brain called “The Influence You Have: Why We Fail to See Our Power Over Others” underscored this from a psychological perspective.

The podcast highlights the research of Vanessa Bohns, a psychologist at Cornell University, who has studied the influence of a stranger over another person.  In her studies, Bohns has asked research assistants to make various requests from a stranger (using their phone, money, etc.).  In the studies, the research assistants were asked to predict how often people will say yes to their request.  Findings reveal that research assistants often underestimated their influence on strangers.

The podcast host Shankar Vedantam summarizes Bohns’ reflections on her studies, which stemmed from her own experience as an anxious student research assistant collecting data in New York City’s Penn Station:

“It felt difficult [making requests of strangers] because she had seen the interaction only from the point of view of her own insecurities. She hadn’t seen the encounters through the point of view of the people she was asking for help. From their perspective, an anxious young woman was asking for something trivial. They had to weigh whether to put aside what they were doing and help her for a few minutes. If they said no, it could make them look like jerks….It’s absolutely true that many of us are influenced by situations, that many of us will do things because the situation prompts it.”

As a researcher and evaluator, this evidence is all quite positive and helps to account for our 65 to 80 percent response rate.  But as Vedantam notes, our power of influence comes with responsibility, which I think is particularly true in our COVID-19 world.  It is likely that when museums open, people will still agree to participate in studies based on the power of influence.  They want to help.  So it is up to us as researchers and evaluators to make sure we are inviting them into encounters that are safe for us and them.  It is our ethical responsibility to understand and wield our power of influence with care.

The post Will they say yes (pre-pandemic and now)? appeared first on RK&A.

When museums started closing to prevent the spread of COVID-19 in early March, there were surges of digital initiatives across all institutions meant to reach new and existing audiences while doors were closed. It’s an amazing and overwhelming time to be involved in the digital side of museums. However, I propose that online resources are not the only way to connect with people in these times.  While looking back through RK&A’s research over the years, I came across a couple of projects in which we evaluated museums “on the go” or programs that brought the museum outside its walls to a variety of venues.

One such example is Inside|Out, a program that began at the Detroit Institute of Arts (DIA) and has been replicated at other museums with funding from the Knight Foundation.  The Inside|Out program brings reproductions of works from the museum’s collection to outdoor venues like parks and storefronts in communities surrounding the museum. In two studies, one for the DIA and one for the Knight Foundation, RK&A looked at the impact of the program on communities. In interviews, community partners and business owners in the neighborhoods that hosted art installations indicated the program fostered pride in the community, stimulated conversation about art among community members, cultivated the community’s interest in the arts, and positively enhanced the identity of the community. Community members who were interviewed onsite at an existing outdoor installation indicated they had positive attitudes towards the Inside|Out program and the museum, and some expressed delight at the sight of art in an unusual location.

Similarly, the Baltimore Museum of Art (BMA) had a program called BMA Outpost (the Outpost), which was a mobile art museum that was set up at a variety of temporary sites.  The Outpost aimed to serve as a forum for conversations about place, home, and why the city of Baltimore matters to its residents. Visitors to the Outpost could see replicas of works from the BMA’s collection and participate in a variety of programming opportunities (i.e. creating works of art about home).  After conducting case studies for ten of the former sites, we found that the Outpost was a positive experience for participants. They found it both empowering and therapeutic. Most notably, the Outpost functioned in a healing role at many of the sites. For example, during the Outpost’s tenure at Healthcare for the Homeless, participants said the installation helped them cope with their anger stemming from their homelessness. When the Outpost installation was at YO! Baltimore, students talked about the Outpost as a means of “escape” from difficult home situations.

What we can learn from these programs in light of today’s new realities is that there is a benefit to taking the museum outside of its walls (both physical and digital). Bringing art to local communities can reinforce awareness of the institution and strengthen communities’ sense of art and self. Programs that bring art reproductions to outdoor community spaces like parks or commercial districts are especially important in these times of digital content proliferation, as not all audiences are aware of online resources or even have easy access to the internet.

Some museums and artists have already started incorporating artwork into new venues. The Crystal Bridges Museum in Arkansas is displaying artwork outside hospitals and senior living facilities to help those who may be suffering from the effects of isolation. Additionally, an outdoor exhibition in Long Island featured works by 52 artists in a drive-by exhibition where spontaneous interactions, such as artists waving to visitors and viewing the art where artists live, led to a unique experience.

Personally, I’ve been spending an increased amount of time outdoors despite allergy season. I find taking walks with my dog to be one of my main stress relievers. I typically walk along a popular trail in Alexandria. I can’t help but think that that spaces like this trail would be livened up with a little art.

You can read more about the Inside|Out program evaluation here: https://www.informalscience.org/summative-evaluation-inside-out-program

You can read more about the Outpost case studies here: https://www.informalscience.org/case-studies-bma-outpost

The post Reaching Audiences Beyond the Internet During a Pandemic appeared first on RK&A.

Since the reality of coronavirus set in back in March, our RK&A team has been having a lot of conversations about study design.  Museum closures and social distancing have greatly impacted the way we do our work as evaluators.  They have affected our clients, project timelines, data collection methods, and access to study respondents in one of our most frequent settings—the museum floor.  Sampling has always been one of the top questions we are asked about, and it is something we very carefully consider when designing our studies, no matter if the study is small or large (see, for example, our previous posts on sampling transparency and sample sizes for qualitative and quantitative studies). One question I have been wrestling with lately in light of coronavirus is the idea of capturing a “representative sample”—that is, a sample that shares the same characteristics of the museum’s visiting population (or whatever population we are seeking for a particular study).

Often, when we recruit visitors for a study at a museum, we use a random sampling approach.  The data collector imagines an invisible line on the floor, intercepts the first visitor to cross over that line, and asks them to participate in the study.  After completing the interview or questionnaire with the visitor, the data collector returns to their recruitment location and selects the very next person to cross their imaginary line.  The rationale for random sampling is that it is more likely to result in a sample that mirrors the museum’s visiting population (for more on sampling protocols, see Amanda’s post here). We use additional measures like comparing observable characteristics (i.e., estimated age and group composition) of visitors who decline to participate (our refusal sample) in the study with the sample characteristics to understand potential gaps in our sample.  All of this information can be placed within the context of a museum’s known visitor demographics (from audience research or other sources) to understand whether a study sample is representative of the museum’s visiting population.

Under pre-pandemic circumstances, this is all well and good.  But now, with the uncertainty of what visitation will look like over the coming months and potentially years as museums phase into reopening with limitations on visitor capacity and new social distancing measures, I wonder what does a “representative sample” mean now?  Are we aiming for our study samples to be representative of the visitor population before the coronavirus?  I’m not sure how useful that is considering visitation will probably not return to what it was pre-pandemic, at least not for quite a while.  In addition to reduced numbers, it would not be surprising to see demographic shifts in visitation in response to the pandemic (e.g., fewer vulnerable groups, like adults over 60).

We always strive for rigor in our evaluations, and responsiveness and transparency in study design are equally important as we learn to adapt to our ever-changing world.  As Heather Krause of Towards Data Science wrote in a recent blog post, “The goal is to retain as much value in the data you currently have and analyze and understand it in ways that make sense now.”  I don’t yet have an answer for what a “representative sample” will mean for our upcoming studies, and I think the answer may vary based on the museum, exhibition, or program.  Still, I can be responsive to both the circumstances of the pandemic and the needs of our clients by having frank conversations about sampling and what information will be most meaningful and actionable.  And, I can make decisions and approaches clear in our evaluation plan and reporting so that we are all on the same page about what the data does and does not represent.  I look forward to working toward a clearer understanding of what “representative” means for sampling in the coming months.

The post Sampling: What does “representative” mean during and after coronavirus? appeared first on RK&A.