Ok, Nonprofit leaders, now that I have your attention……Shameless use of a cat and Star Trek GIF I know.

In fact, I bet you are actually drowning in data, but it’s just not organized and used!

I once worked with a large nonprofit and when I asked them if they collected data, they enthusiastically said, yes! Then promptly pulled out a yellow legal pad with, I kid you not, tick marks. Their Annual Report was very slick, but I can only imagine the time it took someone to add up and present all those tick marks! Let’s just assume it was accurate.

You, my nonprofit friend can do better. The best part is you don’t need any fancy or expensive software to get started!

I know you are probably most concerned with fundraising for your nonprofit right now, but stick with me here! To attract donors and funders, you have to be able to tell your story. In order to do that, you need information that shows your nonprofit’s value and impact.

Here is a simple way to begin. Start by listing all of the information you collect about the people your nonprofit serves. Whether you run a program or service or a community coalition, this is your first step.

All you need is a simple Excel spreadsheet (or Google Sheets if you prefer) that lists these 7 categories in separate columns:

• First Name
• Last Name
• Community sector they represent (if applicable)
• The organization they represent (if applicable)
• The Program they participated in
• Contact information (email)
• Dates of attendance

You can use simple formulas to sum the total number of people attending, the number of sectors, organizations, or programs and attendance by sectors.

Still confused? Let’s fix that.  

Ann

aprice@communityevaluationsolutions.com

Have you ever planned a meeting complete with a well thought-out, organized agenda, only to toss it because your group had more important needs? That was exactly what I experienced last week.

In a call I was leading this past Friday, one of the participants, a community collaborative leader, went right to what is on a lot of minds these days, especially community collaborative and coalition leaders. He asked, how he as a community collaborative leader could respond to his community’s need to talk about the racial tension in his community?

We had a really great conversation.

One of the leaders on the call shared his pastor’s challenge to his congregation to understand the difference between remedies and solutions. Remedies are short term fixes. For example, a backpack program that sends food home to families experiencing food insecurity is a remedy. Remedies might be necessary, but they aren’t solutions because they don’t address root causes.

If we really want to make a difference, we need upstream solutions.  We need to understand why people are falling in the river in the first place. In our example, why are so many families experiencing food insecurity?

In my last blog, I suggested that we all have the tendency to look at simple solutions. Usually we identify individuals as the sole cause of their problems. I spend a lot of time working with community coalitions. When I ask what change they want to see in their community, people- often default to finding fault with individuals without considering the systems that make their life harder.

Root Cause Analysis is a method that you can use to get your organization to examine root causes and these my friend, will help you identify the solutions that can lead to lasting change.

Here are 3 root cause analysis methods you might want to use with your organization. Note that the examples I use here are not extensive and are just meant to get you thinking.

The Five Whys

This technique asks the question “Why?” five times. Each time you should ask your group to think more deeply about the social problem you are focused on and drill down into it.

The Tree Diagram

The tree diagram helps you identify a problem and the symptoms of that problem, as well as the root causes. The diagram displays the structure of causes for a problem and provides insight into possible solutions.

The Fishbone Diagram

The Fishbone Diagram (aka Ishikawa Cause and Effect Diagram) helps visualize a problem (the fish head) and then drills down via categories in this cause and effect model, forming what resembles a fishbone appearance.

Take a moment and consider the social problem or public health issue your community is grappling with. What root causes affect the people you most want to serve?

Your turn.

It’s so easy to think social problems are simple.

Take interpersonal violence for example. When it is discussed, most people ask, “Why does she stay?”

For a few years in graduate school, I ran a randomized community survey that asked people to tell us what they thought of why violence against women happens and what should be done about it. It’s kind of amazing to hear what people think. Or maybe it’s not.

It’s her fault.

She probably did something to make him mad.

Why doesn’t she just leave?

Rarely, if ever, do people ask, “What is it about our society that allows violence to happen?”

As a community psychologist, I am more interested in this question. (Not sure what a community psychologist does? (Click here to learn more).

As we struggle on through the COVID-19 pandemic and a national identity crisis following the recent (and not so recent) deaths of unarmed black men and women, we literally have to get our head out of the trees. It’s way past time to look at environmental context and root causes. I am not sure about you, but I am tired of plugging holes in the dam. There are just too many of them.

What would it look like if we really lived up to our tag line, Partnering for Social Change? That is the question I asked myself a few weeks ago when I took the week off to rewrite my business plan.

Social change may sound scary, but it really involves questioning our assumptions and recognizing and considering complexity. People live in families, within neighborhoods, within cities, within states, within a country. All of these systems shape us.

I once evaluated a community-based elementary afterschool program whose participants attended a school with a history of low scores on state achievement tests. The afterschool program met for about 2 ½ hours after school. The funder of the afterschool program had an advisory board. During a meeting, the chair of the advisory board said she wanted to set a benchmark that all students would be 100% competent on state achievement tests. That goal, while aspirational, failed to account for the multiple systems that surrounded these students. It put all of the pressure and responsibility on the children, and none on the systems surrounding them. Clearly programs can’t do all things for all people, but if we truly want to make a difference, we have to work on the right levers in the right systems.

Think about a game of Jenga or Pick Up Sticks. Some things are foundational.

So, this is what we will begin to explore. Because if we really want to make a difference, we have to get to the root issues.

Take a moment and consider the program you are running right now. What are the family, community and systems pressures that affect the people you most want to serve?

Your turn.

Hi everyone-

Today I am joined by Jenn Ballentine of Highland Nonprofit Consulting to talk about, what else, evaluation in the time of Covid-19. Granted, my last blog was a bit of a rant, so today, I would like to strike a  more positive and helpful tone.  

To tell you the truth, some of the conversation around data collection during the pandemic has me a little squirmy because it has felt kind of opportunistic. I don’t think rushing out to survey people when they are really worried and anxious feels helpful, or frankly ethical.

But we are evaluators, so we do believe evaluation is important and we just can’t stop doing what we do. I am a community psychologist and Jenn, a public health professional. We believe in a public health approach to prevention and in systems-level change. And if anything, this pandemic should teach us is that we are all connected. Systems-level change is needed now more than ever to correct the inequities in our society so evident in disproportionate impact of COVID-19 in communities of color.

Adaptions in the Time of Crisis

Sanjeev Sridharan recently wrote a thoughtful and poignant piece called Adaptions and Nimbleness in the Time of Crisis: Some Questions for Evaluators. In it, he observes that both program implementers and evaluators must now think about how to adapt. He raises a set of questions for evaluators to consider, and I urge you to read the article for yourself.

Today we would like to address the nonprofit and program implementers and provide some practical and feasible tips, inspired by some of the issues he raises.

Jenn and I are evaluating a federally-funded teen pregnancy prevention program that for the last year, has been implemented at 5 community-based centers for teenage boys and girls. I also am the evaluator for several Drug-free Coalitions and Alcohol substance Abuse Prevention Programs, all of whom have a school component. Jenn serves as the evaluator for school-based sexual health education programs facilitated by a statewide training and advocacy organization.

As to be expected, nearly all programming and thus, data collection stopped mid-March. This left Jenn and I wondering, what the heck we were going to evaluate beyond the data we already collected this year?

The technical assistance from funders for the most part, included four specific questions:

1. What were your intended enrollment numbers and what are your actual numbers, and what is are the reasons for these differences?
2. What is the status of your programming and how has that changed?
3. How has data collection changed (number pretests/number posttests) and how were participants affected- e.g. missed content, sessions provided out of order, etc.).
4. How will the program use Continuous Quality Improvement (CQI) strategies to document and learn from the events?

What is missing is the story here, as Sridharan points out in his first question: “What are exemplars of good evaluation stories related to the adaptiveness/nimbleness of specific interventions.” Yes, we need to understand changes regarding what was planned versus what was done, but we need the why, in order to tell the entire story. When did they have to close their doors and why did they make that decision? What happened to staff and why? And as a result of the situation, were program staff able to pivot and if so, in what way?  For example, did program staff decide to shift from in-person to online delivery?

For one of my clients, they have shifted, rather nimbly I might say, to online meetings with their youth advisory committee. They are taking notes about their discussions and developing interventions that they can do online via social media. Similarly, another client that trains health and physical education teachers to implement comprehensive sex education offered to facilitate virtual lessons for one new district in an effort to ensure that students received this valuable information.                                                                                                                                      

Another of Sridharan’s questions is “Are there examples of evaluations that have taken a developmental approach to enhance the coordination at this time of the crisis?”  He observes that the “pandemic has highlighted the need to better understand the connections between the intervention and its underlying systemic contexts/supportive structures.” During a time of crisis, coordination can be improved, perhaps accelerated, or could also break down altogether.

Some school systems for example, have enlisted bus drivers, community volunteers and even local law enforcement to deliver food to students eligible through the National School Lunch Program. Some have expanded food distribution beyond those eligible through these federal programs. Other school systems have maintained the status quo, requiring families and guardians to drive to school with the eligible children present to collect the food distribution. Those without transportation, or without a large enough car to transport the whole family, were out of luck.                                                                                                                                 

There are a lot more gems to unpack (like the dynamics of vulnerability), but we will end with this question posed by Sridharan: “Can a focus on a minimal set of components needed to produce change help enhance a focus on meeting the needs of the disadvantaged given limited resources?”

For our teen pregnancy prevention program, we can’t even imagine where to start on a minimal set of components for this implementation fidelity evaluation. How do you deliver an evidenced-based, comprehensive teen pregnancy prevention program virtually? Is it even ethical to do so with parents, siblings in the next or same room? What about students without internet access, laptops or other devices or when access to devices must be shared by multiple youth?                                                                                             

Looking Forward

We are pretty sure that six months from now, funders will be asking nonprofits what happened? What is the program implementer to do? We think it’s critically important to document the changes programs made and the various ways in which the disruption impacted their organization and the people they serve. But program staff are busy people, especially during times of crisis. Evaluators can help the nonprofits they serve by helping staff document the changes they made and why they made them. Evaluators need to stress how this information will be useful when reporting to funders, partners, board members and others. The learning that comes from this process can help the organization plan for future disruptions.

We developed a guide to help in this process. Just let me know you want the guide and I will send it to you. Depending the needs of your client and their situation, these questions can be adapted in a variety of ways. You might want to change the order, eliminate some questions and add others. Do let us know what you think and if you find it useful. Stay safe and well!