communityevaluationsolutions

#1. Leadership

In the many years that I have been working with community coalitions and collaboratives, choosing the right leader maybe the cornerstone that determines a coalition’s success. An effective coalition leader requires a combination of charisma, attention to detail, and someone who is a savvy politician. It takes charisma to attract and keep the community members needed to do the work. Because the coalition is likely funded by a cobbling of state and federal grants, all with their own rules and regulations, the coalition leader needs to be a detailed administrator. It takes a lot of work in order to meet all of the varied grant requirements and financial obligations. Finally, managing community stakeholders, all of whom have varied opinions, motivations and perhaps agendas, is challenging and requires political savvy.

#2. Make a plan, then stick to it (and change it if needed)

Ideally a community coalition’s change efforts will start with a comprehensive community assessment and data review. The coalition should conduct qualitative interviews with community members and leaders. The coalition should also take a deep dive into local data. Then, once the coalition (not just the staff) reviews and digests the data, they need to design a plan that truly reflects the needs and desires of the community. Then the coalition needs to stick with the plan. Ideas will come and go and crises happen. Sometimes, you need to tweak your plan in order to respond to the community’s needs. However, it’s important for the coalition to stay focused and avoid the shiny objects in the water. For example, if alcohol is the drug of choice among youth, the coalition should focus on that and understand the local conditions that support underage drinking. If the coalition is tasked by a funder to prevent substance misuse and abuse, they should not focus on treatment. I am not saying that other substances or the need for treatment are not important issues. But other community stakeholders likely lead in these areas. The coalition has limited time and resources and needs to think about where they spend their time and energy. We can think of similar examples for other types of coalitions. For example, a coalition that focuses on 3rd grade literacy but spends all of their time implementing programs that have nothing to do with literacy likely won’t move an indicator of improved 3rd grade reading levels. A community coalition needs to use good judgement before choosing what action to take and a focused leader to keep the coalition on track (See point 1).

#3. Nothing about us, without us

Way too many community coalitions commit the unforgiveable sin of not including community members. Involving professionals in the community is important. But involving the people the coalition intends to serve is critically important. So, for a substance abuse prevention coalition that focusses on youth, that means making sure students are engaged fully in the work of the coalition. Likewise, if the coalition is supposed to address low birth weight babies, then young women must be engaged.

#4. Focus on systems change

Too often prevention approaches and early intervention programs focus on the individual person. Sometimes, programs target community groups, by offering a community forum or parent education night. Although community education is important, changing knowledge is not sufficient for the kind of change we all want to see in our community.

A public health approach to prevention is one that considers the context of the problem and reaches as many people as possible. It also considers the social determinants of health, variables that are associated with where people live, work, play and pray.

At the heart of it, a public health approach requires systems change. Systems change means focusing on the structures, laws, policies and community attitudes that allow a social problem to happen. For example, let’s look at domestic violence. People often wonder why a woman stays in a violent relationship, but rarely do people ask what is it about our society that allows violence to flourish? How does how law enforcement and the judicial system respond to violence against women? How easy is it for women to access support? How do people feel about victims of interpersonal violence? Do they feel that the woman is to blame?

#5. The boring stuff – structure

Community members join, then stay in coalitions in order to make a difference in their community. People attracted to a community coalition tend to be passionate and anxious contribute to the work of bettering their community. But a coalition without a structure (that is, job descriptions, written by-laws, and an active steering committee that provides budget oversight and supervision for the coalition staff) is destined for trouble. We have seen work plans not implemented because a coalition director tried to do it all or just would not delegate the work to committees. We have seen coalitions nearly derailed because of disagreements between the fiscal agent and staff and financial mistakes made by staff that threatened funding. Butterfoss and Kegler’s (2002) Community Coalition Action Theory (CCAT) is helpful in guiding coalitions and helping them to avoid these and other pitfalls. We use the CCAT as a framework for understanding our coalition clients and assessing the health of the coalition. For one client, we created a coalition logic model using the CCAT model and used it to educate the coalition on what they need to do in order to position themselves for success. Have a community coalition and need some guidance? We’d love to help. You can reach us at aprice@communityevaluationsolutions.com.

Butterfoss, FD, Kegler, MC. | In DiClemente, R, Crosby, L, Kegler, MC. (Eds.) Emerging Theories in Health Promotion Practice and Research. San Francisco, CA: Jossey-Bass Publishers, 2002, 157-193.

Last month, I had the opportunity to attend the Summer Evaluation Institute hosted by AEA. I met some great people and learned a lot about evaluation. I’m what my family likes to call a “professional student” – I would be in school for the rest of my life if I could. I took pages and pages of notes, organized the resources I was given, and came up with this list of the lessons I learned while there.

1. Evaluation isn’t just for evaluators.

Going into the institute, I was so nervous. With just under a year of professional evaluation experience under my belt, I was sure I would be behind in some of the workshops. That definitely wasn’t the case. Now, I’m not claiming to be an evaluation master, but there were fewer evaluators attending than I anticipated. Project developers and managers, health promotion specialists, policy analysts, and more came from all over the country to learn about evaluation.

This isn’t a new concept for me, but it was strongly reinforced at the Institute. Working with Ann at Community Evaluation Solutions has taught me that building an organization’s evaluation capacity is an important part of being an evaluation consultant. We want to make sure that our clients can and will continue to evaluate their programs, even after our contracts end.

Fun Fact: Ann actually taught a workshop at the Summer Institute this year on this very topic: Tools and Techniques for Assessing and Strengthening Nonprofits’ Evaluation Capacity.

2. The content is the paint, but the presentation is the canvas.

Bear with me. What I mean here is that the canvas is what brings the art together. If you’ve got a ton of information, but no clear way to deliver it, it’s as good as buckets of paint on the ground. The final product is what gets remembered, and if you can’t bring all of your paint together to tell a story, then what’s the point?

Out of the five workshops I attended, there are two that I will likely remember for a while. I am painfully introverted, and as a result of that I have a strong aversion to public speaking. One of the workshops I attended was about strategies to engage your audience when presenting. Not only did I learn about the strategies, but I saw them in practice. The speaker, Sheila Robinson of Greece Central School District, was captivating and informative, which can be difficult. Kylie Hutchinson, the second speaker I will not soon forget, was personable and fun. Usually when a presenter throws facts at you, they are forgotten as soon as you walk out the door. These two speakers presented the information in a way that will make it sticky and easy to recall/repeat. I’m hesitant to say that I am excited to put what I’ve learned to use, because that would mean I have to stand up in front of a group and present, but I already feel more prepared for my next presentation.

3. It really does matter where you work.

As I mentioned in #1, I have been working for Ann for almost a year now. One of the best things about CES is that I’m able to put my hands on so many different projects and learn a little about a lot. Well, the main thing that the Institute taught me is that I haven’t learned a little about a lot, I’ve learned a lot about a lot. I honestly did not expect to learn as much as I did in my first year, and I did not realize how much I had learned until last month.

In my last workshop, we were split up into groups and given a few tasks. In my group there were a couple of young evaluators, an epidemiologist, a program developer, and a few disease prevention specialists. Working through the tasks made me realize how much I already knew about evaluation. I could answer questions that my group had, give examples of work we had done at CES, and explain why certain techniques were used in different situations. I’ve never been very good at networking but being knowledgeable about evaluation helped in that other people wanted to start conversations with me about my experience at CES. Knowledge is power, and Ann sure has given me a lot of it.

Overall, the Summer Institute was an amazing learning and networking opportunity. There are classes for all levels, and I really think non-evaluators would benefit from attending. Hopefully I am able to attend next year, but until then I will learn every day with Ann.

The AEA Summer Institute: A First-Timer’s Impression Daniel Snook is currently working at Community Evaluation Solutions doing a practicum in program evaluation.

My first trip to the American Evaluation Association’s (AEA) summer institute was eye-opening. I was aware of evaluation as a practice and as a useful tool for program development and improvement before I arrived, but I was not aware of the full breadth of Evaluation with a capital ‘E’ (i.e., evaluation as a field). Suffice to say, I now know just how much I don’t know about the incredibly multi-faceted field of evaluation.

My background is in psychology, specifically I’m a PhD student studying Community Psychology at Georgia State University. At the beginning of the conference I was feeling a bit like an incognito psychologist—I didn’t want anyone to realize that I hadn’t been doing evaluation work for years and that I didn’t identify per se as an ‘Evaluator’, at least, not yet. Community Psychology is essentially the study of how communities impact individual, and most programs (including my own) provide some training in social program evaluation. However, learning about a suite of techniques in theory and learning about them in practice are very different things. Occasionally the academic world and the real world come crashing together– that was my experience at the AEA summer institute. It was simultaneously awesome and discomfiting, and it was a great place to learn a lot very quickly! Here are some highlights of what I learned:

1. Theory matters– no matter how applied your work is.

I came into the first session I attended, Program Theory, led by Dr. Stuart Donaldson thinking I would find myself in comfortable (read: ‘academic’) territory. One of the first activities we did challenged that idea almost immediately. Dr. Donaldson asked each of the groups of audience members to evaluate the room in which we were sitting. Each table and its members were then asked to assume roles, for instance, as teams of interior decorators, information technology specialists, or, in our case, firefighters. The number of different ideas about what makes a room ‘good’ or ‘successful’ from these various perspectives was staggering. As you might expect, the lesson became quite clear: your approach to and expectations of a situation (i.e. theory) significantly impacts your practice, even if you don’t explicitly realize it.

2. Avoid “The Curse of Knowledge.”

In her keynote presentation, veteran evaluator Kylie Hutchinson described some of the basics of effectively communicating evaluation results to stakeholders. A prime mistake, she says, that presenters make, whether in evaluation or otherwise, is assuming their audience knows what they know. This is, of course, more of an implicit than explicit assumption; evaluators are consciously aware that their clients do not know every detail of the evaluation, yet their presentations often don’t reflect that. Understanding your audience’s starting point by doing things as simple as laying off the jargon or explaining acronyms can keep your audience from zoning out or rolling their eyes. It’s also tempting to include every detail about your evaluation just because each detail is important to you. But your audience hasn’t been working on the evaluation at that level of detail, so spare them all of your knowledge and focus only on what you know is important to them. If you can say it more simply, then do so.

3. Good presentations are good; bad presentations are terrible.

The AEA summer institute left me with the distinct impression that evaluators are very good presenters. They’re practically oriented, which means they want to get to the bottom of what makes a program successful (or unsuccessful) and then get to the point in telling the client about it. Evaluators also have a very high bar for data visualization, tools for incorporating theory into practice (e.g., logic models), and techniques for transforming vague goals into tangible and measurable specifics. However, I also learned a bit about how NOT to present. I’ve mentioned jargon once already, but I have to reiterate that purposefully using jargon, whether it’s to obscure the fact that you’ve got nothing substantive to say or to create a veneer of professionalism, wastes everyone’s time. Finally, to be frank, not all content is worth presenting. If you’re on the fence about whether the ideas in your presentation are worth sharing, they’re probably not.

4. Invest in your evaluations early.

The readiness is all, and that holds especially true for evaluation. Several of the sessions I attended discussed the importance of being prepared for your evaluation from start to finish to set yourself up for success. Sheila Robinson’s excellent session on strategies for evaluation planning encouraged me to ask the right questions– the why, what, and how of conducting an evaluation– early on (i.e., before beginning an evaluation). In other sessions, presenters made it clear that whether you’re using pilot studies or cognitive interviewing, it’s well worth your time to explore your proposed theory of change, your indicators, and whatever else you can before they’re set in stone. Acting with intentionality at the outset of an evaluation pays big dividends when it’s time to present findings.

5. Always mix your methods.

One thing that I already knew as a researcher, but that was reinforced powerfully at the AEA summer institute, is that it’s always best to use both qualitative and quantitative methods of measurement. Listening to speaker after speaker, it became increasingly apparent that quantitative or qualitative measures alone are simply inadequate for telling the whole truth. Quantitative methods are critical for providing the evidence in ‘evidence-based’. That’s not to say qualitative data cannot be considered evidence, but it is by its very nature more subjective, and requires ‘quantifying’ to become less open to interpretation. Quantitative data will satisfy the number crunchers in the room as well as make your findings look and feel more robust. But it isn’t quite enough. Qualitative data brings the ‘human’ element to the human sciences in ways that quantitative data cannot, because it enables researchers to (often literally) take the perspective of the participant. That makes for insightful narratives and stories that not only bring the results of an evaluation to life, but also give a human voice and face to the ‘hard’ data that’s been brought to the table. At the AEA summer institute, all the best presentations of evaluation results discussed both quantitative and qualitative elements.

How to Kill Your Community Coalitions and Collaboratives: Meet and Talk Meetings

Because CES works with a lot of community collaboratives and coalitions, I attend a lot of community meetings. So, I see the good, the bad, and just plain ineffective. What I have observed is that there are some sure-fire things you should NOT do if you want to fire up your community for change. (Note: for the purpose of this blog, I use the terms coalitions and collaboratives interchangeably).

Meet and Talk meetings are probably the most common meeting killer that I see. In this type of meeting, attendees go around the room and share updates of their organizations, one after another. There is no work done, no direction, and participants don’t really connect in any meaningful way. What participants share in the meeting really could be done through other means like email updates, newsletters, or social media posts. There isn’t really a purpose that brings participants together and if you use this type of structure consistently, you shouldn’t be surprised if the energy, and eventually your attendance, decreases.

Another guaranteed meeting killer is to have guest speakers present at each and every meeting. Although your participants may learn something valuable, this type of meeting has the same effect as the Meet and Talk type of meeting. Participants don’t really connect in any meaningful way, and there is no work done.

After all, isn’t collaboration is THE whole point of a community coalition or collaborative?

Recently, Gabrielle Hawkins-Stewart, a Prevention Support Specialist for Georgia Family Connection Partnership, led a discussion for a group of community collaborative coordinators on how to have effective meetings. This list will highlight some of these ideas along with some of my own suggestions about ways to infuse energy in community collaborative meetings. If you have been having Meet and Talk meetings for a while, you may have noticed that your attendance is down. If that is the case, you definitely want to act quickly and shake things up.

1. Get energy started before people enter the room. Have music playing. Have a bright color table cloth at the sign in table. Place candy and table toys around the room. Start with a quick, fun ice breaker. Introduce new members and help everyone connect, but don’t waste valuable time having everyone introduce themselves.
2. Remember – the whole point of gathering your community leaders and members together is to collaborate. So, make the meeting about collaboration! If you have time, make part of the meeting a work session, perhaps letting committees and workgroups meet briefly within the meeting to have a work session. At the very least, committee and workgroups should provide updates tied to your work plan. This of course, requires you to assign responsibility for your action plan to the committees/workgroups in the first place! If your time is limited, you might want to have longer work sessions every other meeting or perhaps quarterly.
3. If you are going to have guest speakers, require them to attend a certain number of meetings before they get the floor. This will help you limit people who want to get their message out but are not tied to the work of the community collaborative/coalition. If you do have guest speakers, require them to complete an information form that structures their talk so that you can ensure that what they share ties back to the work of your coalition. Then as the leader, when the speaker is done, make sure YOU tie it back to the work. If you do have speakers, you might announce that there is a speaker and tease the topic, but don’t announce the name of the speaker.
4. Every once in a while, it’s a good idea that you as the coalition or the collaborative leader lead the meeting. Use this opportunity to remind everyone why they are there. Share the purpose of the collaborative, share the goals and objectives you are trying to accomplish, the strategies you are implementing to accomplish those goals, and the data you are using to track your progress. Make sure you assign responsibility for the work to work groups and committees and use this time to show them why their work is important to achieving the collaboratives goals.
5. Round table discussions are another way to involve collaborative members in the work. Identify some good discussion questions and group members discuss and report back. Concentrate on questions that are related to the work: the strategies and changes that the collaborative and coalition are putting into place.
6. In order to provide the communication that your participants desire and expect, use different ways to facilitate communication other than taking valuable collaborative meeting time for report outs. For example, once a year, hold a resource fair so that members can share with each other what they do. You might consider limiting the first hour of the resource fair to members, then opening the resource fair to community members.
7. Instead of members sharing details of their upcoming events during the meeting, create a newsletter with a Community Event corner.
8. Encourage communication by sending out an email to those who missed the meeting with the message, “Missed the meeting? Call me (the collaborative leader) or a friend and find out what you missed!”
9. Focus on the community outcomes your collaborative is trying to address (e.g. poverty, graduation rates, foster care, unemployment). Looking at your data only once a year when you update your work plan or write your year-end report isn’t effective. Use your indicator data often, perhaps quarterly, to inform your members about local conditions and help motivate them to stay/get engaged.
10. You might consider asking groups to pay a small member fee to become a member of the coalition; they will have a little skin in the game and perhaps be more likely to attend collaborative meetings.
11. Find resources to help you improve your meetings. Look for professionals in your community who are good facilitators and ask them to facilitate a meeting. SCORE is a nonprofit association and a resource partner of the Small Business Association (SBA) whose members are retired professionals dedicated to helping small businesses through education and mentorship. They have a wide variety of skills that may help your collaborative and your members.

We’d love to hear from you! What tips have you used for effective meetings?

A Heart Full of Grace By Emily French

A Heart Full of Grace By Emily French With Martin Luther King, Jr. Day coming up, I thought we could talk a little bit about social change and “making a difference.” What does that even mean, you might be thinking to yourself. I’m not sure that’s an answerable question – I think making a difference looks different for everyone.

I went to Mercer University for both my undergraduate and graduate degrees, worked as a graduate assistant, then became a full-time employee of the school. If you’re familiar with Mercer, you know that one of the most common university mottos is “At Mercer, everyone majors in changing the world.”

I spent a lot of my academic career wondering how I could make a difference. Should I apply for medical school or pursue public health? Should I go on mission trips with my church or volunteer locally? Should I advocate for legislative change or try to support disenfranchised individuals in a more personal way?

Something I recently discovered is that there isn’t one right way to make a difference. Whether you have mobilized thousands of people for a cause, mentored a young child, or given your time to a local non-profit organization, you are significant, and the impact you are having matters. One of my favorite things about working for CES is our tagline: Partnering for Social Change. Our clients are all working to better their communities, and I am so grateful to be a part of that process.

If you want to make a difference, the first step is to decide what is important to you. There are endless options: arts, the environment, children, health and wellness, the elderly, civil rights, etc. Look for opportunities to volunteer or donate to these causes. Volunteer Match is an online service that allows you to select service opportunities based on location and area of interest. Talking to friends, family, coworkers, pastors, etc. can give you more ideas of where to serve and how to get started. Ask your connections on social media if they know anyone who is in need of some help. My challenge to you is to find what you are passionate about, and find a way to serve that passion. Volunteer. Donate. Mentor. As Dr. King famously said in 1957, “Life’s most persistent and urgent question is, ‘What are you doing for others?’”