evalacademy

 This article is rated as:

 

 

“What other training should I take?” This question can be tricky to answer for both emerging and experienced evaluators.  

---

Emerging evaluators can get a good sense of what complements their evaluation training by looking at job postings and talking to more senior colleagues. Are those postings mentioning non-evaluation skills or experience that you can seek out? You may need to read between the lines to understand what those organizations are truly looking for in an evaluator.  

More experienced evaluators may be faced with a more complex process of identifying relevant professional development. When you’ve been in an evaluation role for many years, even decades, you are probably quite confident in your evaluation tasks. Looking outside of the evaluation field for training can be a great way to complement your technical expertise.  

Most guidelines for what an evaluator should do well include skills outside of developing surveys and analyzing data. Those guidelines (such as the Canadian Evaluation Society’s Competencies for Canadian Evaluation Practice) include domains like management and interpersonal practice, and can be a great way to help identify where you can grow.  

As a mid-career evaluator and someone who both hires and supports the development of new evaluators, I’ve listed some of my ideas for non-evaluation skills to work on as an evaluator.  

---

1. Project management 

   Most evaluation work is project-based. Even ongoing evaluation activities benefit from a project management approach. I don’t think evaluators need to hold a PMP designation, but seeking project management training that’s not all about construction projects is a great idea. Kick off your project management learning with our 5 key points here, and browse our resources to find project management tools to help you right away. 

2. Writing 

   Academic writing is out of place in evaluation. Remember, we’re not doing research; rather, we’re serving organizations, programs, or communities that need the information we produce to make decisions. Some of the practices you may have implemented to meet minimum word counts on academic papers are a hindrance when writing evaluation reports. Seek business writing training if this transition doesn’t come naturally, and you’ll experience the benefits for the rest of your career. Our 9 common evaluation writing mistakes can get you started.  

3. Graphic design 

   Similar to good writing, good design makes a huge difference in how your evaluation product is received and used. You don’t need to get a diploma in graphic design, but taking a day or two to understand how design principles can support knowledge sharing and practical application is a worthy use of an evaluator’s time. You can read our series on making better evaluation reports through design starting here. 

4. Data visualization 

   How are people still making bad charts? There is so much good dataviz training readily available that there’s no excuse for evaluators to rely on Excel defaults, or use the wrong chart to tell the story. If you’ll be doing any quantitative work, dataviz training is a must and is usually not part of graduate programs.  

5. Presentation design 

   Just like dataviz, there are so many fantastic resources on good presentation design that I am always floored when I see people reading slides full of text at me. Good presentations help you keep your audience interested, and help your audience walk away with the key points you need them to hold onto. Adding presentation design skills to your evaluation toolkit will definitely make you better at your job – our analyst has 7 tips to share here. 

6. Facilitation 

   Evaluators rely on facilitation skills throughout the evaluation cycle, from planning an evaluation project, to collecting data through focus groups, to hosting sense-making and action-planning sessions after findings are ready. Learning how to lead support groups through these processes is so important for evaluators. (We’ve shared our tips on hosting good virtual sessions.) 

    

7. Conflict resolution 

   Evaluators often work in tense spaces. Big, important decisions can result from our work, and different partners will have different interests in the evaluation project. Conflict is necessary and healthy, and a skilled evaluator will understand how to benefit from productive conflict, and resolve unproductive conflict. Conflict resolution training is widely available, and may even be offered for free to employees working in large organizations. 

8. Public participation and community engagement 

   While some evaluation projects are internal to an organization, others require public outreach. Learning from the field of community engagement and public participation can be super helpful for evaluators. An IAP2 certificate can look great on an evaluator’s resume, but shorter professional development offerings can give you what you need to incorporate some new practices.  

    

9. Public speaking 

   The need for an evaluator to speak in front of large audiences may be surprising. Evaluators should be prepared for and comfortable with speaking to organizations’ members, staff, or the people they serve – perhaps to share plans for evaluation activities, or maybe to share back learnings. Training in public speaking can help you craft your speaking notes, project your voice, and feel more confident in front of people, whether it’s a small group, dozens, or even hundreds. 

10. Grant writing 

    Evaluators are sometimes called upon to support grant applications. I’m not suggesting that evaluators should be leading grant applications, but understanding what makes a good grant application can be complementary to your skills, especially if you’ll be called upon to contribute to grant applications. 

11. Cultural competence and diversity training 

    Evaluation work takes place in so many different contexts. Some newer evaluators may have had these concepts included in their graduate studies, but others may not have. Taking additional cultural competence and diversity training can help you ensure that your projects are ethical, responsive, valid, and fair – and can demonstrate to a potential employer that these are things you value.  

     

12. Software training 

    There are always new tools that can support our work. Digging into statistical software, quantitative analysis products, and productivity suites like Microsoft Office 365 or Google Workspace will probably yield new practices you weren’t aware of and increase your efficiency. While keeping up with new AI products that support evaluators feels like a part-time job these days, there are dedicated training sessions that can help you filter down to what’s most relevant for your evaluation work.  

---

Evaluation is a broad field with lots of room to find your niche. There are so many options for complementary training that choosing can be overwhelming – try to think about what you like doing, and what you can see yourself being interested in for a few more years. Lean into what you like, and supplement with what potential employers need, and you won’t be at a loss to finding professional development opportunities.  

This article is rated as:

 

 

A real-world example

I am of settler-Métis descent and a member of the Métis Nation of Alberta. Currently, I live in Medicine Hat which falls in Treaty 7 and Treaty 4 territory. My family comes from the Battlefords in northern Saskatchewan, but I mostly grew up in Calgary. I worked in non-profit organizations in Calgary and Medicine Hat, primarily in roles that focused on operations, data management, and reporting. I currently work as an evaluation assistant at Three Hive Consulting. I also volunteer within my community.

The organization I volunteer with received funds to host a round dance in our local community. The round dance was open to the public. I thought it would be a good opportunity to share with you how we evaluated this event and reported back to the funder.

---

This non-profit exists for Indigenous women and two-spirit people to both learn and teach Indigenous culture. In everything we do, we look for ways to apply Indigenous ways of knowing, being, and doing. So, how do we evaluate the outcomes and successes of an Indigenous ceremony using an Indigenous worldview, while honouring protocol and oral tradition?

 Most of us involved in evaluation have commonly used these methods:

• counts of attendees,

• satisfaction surveys, and

• the occasional pre- and post-event surveys around knowledge gained.

These methods didn’t feel like they would tell the full story or be the right tool for this evaluation. From an Indigenous worldview, the evaluation should tell the whole story. So, what is the whole story? Where does the story of this ceremony begin?

---

Where did we start?

Another volunteer and I met to discuss ways to evaluate this Indigenous ceremony. What was it that we wanted to know? Of course, we wanted to report on the funds in a meaningful way, but how could this evaluation also help us as an organization?

 We identified the main questions we wanted to answer:

• How can we improve our planning and preparation for ceremonies and craft circles?

• What impact do these ceremonies and craft circles have on our community?

• What other gatherings would our Indigenous community like to see?

Our questions involved two different levels of community. When it comes to impact, we not only want to get the impact on the Indigenous community but all who participate. However, when it comes to planning future activities, we want this information from our Indigenous community only.

---

Considerations

When we were planning our evaluation, we did not include the pipe ceremony or its accompanying feast that were held before the round dance. We didn’t feel evaluating these ceremonies was appropriate due to their sacredness. A pipe ceremony tends to remain more in oral learning and not in writing. However, we did note in our report to the funder that these ceremonies were held on the same day as the round dance.

We focused our evaluation on what participants got out of the ceremony and how we could improve our preparations for implementing the round dance. We did not centre any of the evaluation on improving the ceremony itself due to the traditions and protocols that need to be followed. Our organization can change how we plan or prepare for a ceremony, but we wouldn’t make changes to the ceremony itself.

---

What are we evaluating?

Round dance requires extensive preparation. It is part event planning and part cultural teachings. Not everyone in our community had the opportunity to learn their culture, so every time a ceremony is planned, teachings must be given in preparation. In addition, we needed to prepare some ribbon skirt craft circles as this was the regalia required to be worn for the round dance.

On average, in a ribbon skirt craft circle, it takes about 5 hours for someone to make their first skirt, and each participant needs access to their own sewing machine. The space required for this and the number of people who needed access to these teachings meant our members needed to run three ribbon skirt workshops within a month.

 

 

So now our evaluation will need to include ribbon skirt craft circles and the round dance ceremony for which we were funded.

---

How we conducted our evaluation

We knew that the evaluation would need to be done by our volunteers, so we needed to keep the data collection fairly simple.

1. Quantitative data

   • We counted the number of people who attended the ribbon skirt craft circle. We commonly count attendees at our craft circles, so this would be easy to gather. Reflecting back now, it would have been useful to also count how many skirts were made, as that metric would inform how we might run a ribbon skirt making differently. We did find that not everyone was able to complete their skirt, so it would have been helpful to have a count of how many were completed, how many were incomplete, and how many people took kits to finish at home as they were busy helping others.

   • We counted attendees at the round dance. Students volunteered at the round dance, so the count of attendees became their metric to track. As this is an annual event, we find this measure useful to see if we have growth year over year. We also use this count to assess if the space we are using is big enough.

2. Qualitative reflections

   • Through informal interviews during one of the craft circles and at the round dance, we were able to gather attendee feedback on what they gained through their participation. Those who participated were informed that the information shared would improve our service offerings and help us report to funders. We also collected participant feedback to show the impact of holding a round dance in our community.

   • After the ribbon skirt circle, I spoke with one of the facilitators about how it went from their perspective. It was recommended that when teaching people to make their first ribbon skirt it is probably best to stick to a simpler skirt pattern, as pockets were a little too hard for those with less sewing experience and a simpler pattern may have increased the number of people who were able to complete their skirts in the craft circle.

   • We were also able to gather group feedback at a members’ meeting shortly after the round dance. This group was asked to provide their reflections and feedback about the round dance. I opened this conversation broadly with, “Does anyone have thoughts or feedback to share about the round dance?” I commonly ask for thoughts and feedback from this group after we host events or anything that requires major planning. So, they are used to me asking a similar question and are ready to give their thoughts on what worked well and where we could improve. I did probe a bit more about what their highlights were from the round dance. Some of this feedback gets recorded in our meeting notes and I also take notes to supplement the meeting notes.

Our qualitative data gathering was informal. We didn’t use formal guides or scripts. Our volunteers were given a couple of questions they could use in these conversations. We aimed to use a conversation style, and much of it was based on familiarity or relationship building.

All these data sources formed the basis of the evaluation of the round dance. The quantitative data and qualitative reflections gathered would be used to report back to our funder. We would also use these evaluation findings to inform improvements for the craft circle experience and annual ceremony planning.

Given this is a new and growing organization and this funder was the largest single funding source so far, any reporting would be new to its membership. 

---

Lessons learned

Some of the lessons we learned through this evaluation were:

• Discussions with participants are time-consuming. Each conversation takes time because relationship-building is the priority. Even as an active member of this community, we first need to catch up on everything since we last spoke. Because everything is about relationships, I can’t just start asking questions about participation. Relationships are key to meaningful conversations.

• Ensure there are more volunteers available to talk to participants about their experiences and keep their goal for the number of conversations low. We were too ambitious for how many discussions we thought we could have due to both the time these conversations take and the desire for all who are at the round dance to be able to participate in a conversation.

• Prior to the ceremony and craft circles, gather the volunteers together who are going to have these discussions to practice working the questions into conversations. Most of the volunteers had difficulty or felt uncomfortable asking participants about their experiences. I think it would have been helpful to have the volunteers do some role-playing to gain confidence.

• It would have been better to focus on collecting participant feedback immediately after the ceremony. I had more meaningful conversations when we were cleaning up and people were starting to head home, as people were more eager to share their experiences and feedback was easier to hear. Carrying a pen and paper around for the entire ceremony is not practical and would hinder my participation in the round dance. By only gathering feedback as we were wrapping up, simplified how long I had to keep these supplies on me. I made notes after conversations as recording in a loud gymnasium would not have provided useful recordings.

• Discussions after the round dance were just as important. Some people need to be asked a couple of days later to get some richer responses. These later conversations were where discussions about not only what people got out of their participation came through, but also their ideas as to how to better prepare for the next round dance.

• Allow space for the unexpected. Through some of the conversations I was able to have, I learned about people’s thoughts on the non-profit’s other programs and the strategic plan. I was not expecting that type of feedback from these conversations, but that input may help lead future discussions for members about the organization’s future directions.

---

For future evaluations, I would like to involve other volunteers sooner in the planning and preparation of the evaluation. Their earlier involvement would help them to understand how we developed our evaluation questions to support buy-in and let them practice the skills required to lessen any uneasiness in our data collection methods. For a future round dance, I would concentrate on connecting with participants immediately after the ceremony. This might not apply to all ceremonies, but it was suitable for this one as this was a healing ceremony, most participants are energized after. It was also beneficial to gather additional feedback a few days later, as this time allowed for some participants to further reflect on their experience.

This article is rated as:

Early in your career, it can be hard to know what field you want to work in, let alone feel confident in finding a job. If you’re thinking that you might want to become an evaluator, or you’re a newish evaluator looking for a job, read on to see what I look for when I’m hiring.

I lead Three Hive Consulting, a small evaluation consulting firm based in Alberta, Canada. As a small business owner, hiring a new team member requires a significant amount of my time, so I need to know that I’m finding the person with the best fit. I haven’t always gotten it right – reflecting on what didn’t work, in combination with what did, has led me to a place where I feel confident making some suggestions for evaluation job seekers.

---

1. Passion for Evaluation

Maybe “passion” is a bit strong, but most important to me is that our candidate wants to be in evaluation. They’re not looking for an academic career or a research position – they want to do real-world evaluation, serving communities and programs directly rather than building up their list of publications. I’m looking to see that the candidate knows what evaluation is, how it’s different from other knowledge-generating practices, and that they believe evaluation is the best fit for them. I love to hear about how people discovered evaluation, and what draws them to it. A certificate or other professional development is a great supplement for those without evaluation degrees – that extra training shows me you want to pursue evaluation.

---

 2. Versatility in Methods

Evaluation requires holistic thinking. We need numbers and stories, science and art. I definitely hire folks who are more specialized in qual over quant, or prefer statistics to thematic analysis, but an evaluator needs to understand that both are vital to our work. Even though our team includes quantitative analysts, our evaluators in a more generalist role need to be able to bring quant and qual together. So, when I’m reading a cover letter (I won’t hire anybody without one), I’m looking for an appreciation of the complexity of evaluation methods and evidence of education or experience in both qualitative and quantitative techniques.

---

 3. An Adventurous Spirit: Willingness to Try New Things

Our work is in projects, and every project is different. I look for candidates who are excited by new opportunities and challenges, and who want to stretch their skills. Evaluation is an ever-changing field, and my team needs to be ready to embrace new things and explore what can work in a new situation. I want to hire a candidate who likes to learn and is comfortable getting into spaces where failure is a real possibility.

---

4. Team Player Mentality: Embracing Everyone’s Strengths

Three Hive is all about collaboration, so I’m on the lookout for team players. One of the things I love about evaluation is that people have come to the field from so many backgrounds; in our team, we have folks with degrees in public health, business, education, psychiatry, anthropology, and biological sciences. Adding to those academic backgrounds are our different experiences and aptitudes, things like project management, facilitation, and design. If our team members don’t seek out their colleagues’ strengths, that’s wasted talent. So, I look for a candidate who wants to learn from their team members, who respects and seeks out advice from different perspectives, and who can carve out a complementary role for themselves.

---

5. Values Alignment

Our company operates, hires, and fires based on our core values. It’s imperative to me that I can see our values reflected in a candidate’s cover letter. Every organization operates differently, and I hire people who want to work with us in a good way – they don’t just want a job, they want a job with us, and they want to live those values. I don’t think that’s asking too much, and I don’t think people last very long in an environment of mismatched values.

---

6. Demonstrating Evaluative Thinking

Evaluation is more than a basket of skills. Evaluation is about thinking. I look for candidates who showcase their ability to approach problems critically, analyze situations strategically, and demonstrate evaluative thinking. I like to see how the candidate has brought evaluative thinking to their life and work – if you’re inclined to think deeply about complex situations and consider multiple sources of evidence, you might just make a good evaluator. You don’t need to have had “evaluation” in your previous job title if you can show how you’ve brought evaluative thinking to your roles. Let me see how you’ve questioned assumptions, considered perspectives, and adapted your approach based on feedback.

---

7. Good Writing

Evaluation is so often about communication. Most of our deliverables are written products, and we want them to be engaging, compelling, and persuasive. When I see a cover letter with big walls of text, or sentences that are poorly constructed, I don’t want to keep reading – and I certainly don’t want the writer to contribute to our client-facing products. I want to see that you can craft a strong argument, so in your cover letter, tell me why you’d make a good evaluator at my company. Like I mentioned before, I don’t interview a candidate without a cover letter; that letter is their chance to demonstrate a key skill for evaluators.

---

Other leaders may be looking for different things in a new evaluator, but hitting these eight points is likely to land an interview at Three Hive, and I suspect other organizations would feel the same.

This article is rated as:

 

 

Eval Academy just released a new infographic: Monitoring vs. Evaluation

---

Who’s it for?

This infographic is for anyone looking to learn more about the difference between monitoring and evaluation.

---

What’s the purpose?

The Monitoring vs. Evaluation infographic will help you to:

• Learn the definitions of monitoring and evaluation.

• Identify the primary purpose of monitoring compared to evaluation.

• Identify key differences in data collection between monitoring and evaluation.

• Learn about the primary use of monitoring compared to evaluation.

---

What’s included?

A one-page, downloadable infographic as a png file.

 

 

Download the Monitoring vs. Evaluation infographic now!

---

Learn more: related articles and links:

• What is Evaluation? A Review of AEA’s Recent Post

• 10 great reasons to evaluate!

• Research and Evaluation

• A beginner’s guide to evaluation

This article is rated as:

 

 

Different professions have unique strengths and skills! Recognizing these provides us as evaluators with a great opportunity to learn from other professions to enhance our own work.

February is National Psychology Month here in Canada, and in honour of our counselling, clinical, and educational psychologist friends, I explored what we as evaluators can learn from psychologists to improve our practice! While evaluation and psychology practices are different in many ways, they also share quite a few similarities. For instance, both professions often involve working with people, using evidence to support our work, using tools like interviews and questionnaires, writing reports, and continuing professional development.

By looking closer at some of the skills that are essential to psychology practice, we can borrow some ideas to grow our own skills as evaluators. Using this lens can help us discover new strategies to try or remind us to be more intentional about things we may already be doing without thinking!

In this article, I’ll share three top tips we can learn from psychology practice to support our work as evaluators.

---

1. Communication and active listening

The first tip is about communication. Psychologists spend much of their time communicating with clients, families, and other professionals. To make these interactions go smoothly, psychologists learn how to communicate effectively. For example, they often practice active listening in their interactions.

Active listening is a communication strategy that goes beyond simply hearing someone’s words. It involves showing understanding through both verbal and non-verbal cues. To practice active listening, psychologists might use techniques like the following:

• Attending (being present, showing interest by nodding or saying, “I see,” for example)

• Paraphrasing (restating what they hear in other words to show understanding)

• Reflecting (expressing an understanding of the other person’s feelings)

• Silence (encouraging the other person to continue or providing an opportunity for reflection)

Psychologists also pay attention to their own communication styles and might choose to adapt how they speak or interact to match a client’s style. For example, with a client who uses plenty of jokes and avoids eye contact, a psychologist might also intentionally incorporate some humour and limit their eye contact too. This helps the client feel more comfortable and builds the relationship between two people.

---

What can evaluators learn?

Evaluators also spend a great deal of time communicating in our work! Whether it’s conversations with coworkers, clients, project staff, or collecting data from interviewees, evaluators rely on the information we gain from our interactions. We can practice active listening to make sure we understand what someone is sharing by listening carefully, asking clarifying questions, and allowing silences. Active listening also supports relationship-building by letting others feel heard and seen in conversations.

To make our communication as effective as possible, we can also tune in to the other person’s communication style by noticing their body language, volume, tone of voice, use of humour and metaphors, or the specific terminology they use (for example, “partner” vs. “spouse” or “patient” vs. “client”). We can then consider mirroring certain elements to create shared understandings, such as by matching their volume or referring to a metaphor they used.

---

2. Interdisciplinary and collaborative work

Most psychologists don’t work in isolation. Psychologists who work in agencies or healthcare settings often need to collaborate with other professionals, like social workers, nurses, or housing workers to support their clients. Psychologists in private practices may consult with other psychologists on their cases to seek advice and discuss strategies.

Collaboration and working with professionals from other disciplines can be valuable for psychologists. It can help them understand a client’s experiences (e.g., dealing with a chronic illness or navigating the legal system) and offers them diverse perspectives to inform their approach. Psychologists can also discover new resources or learn new skills and techniques to incorporate into their work.

---

What can evaluators learn?

Collaboration can benefit evaluation practice too! Whether it’s with other evaluators or people from other disciplines, collaboration provides a great opportunity to problem-solve, learn new ideas, and build relationships. Asking someone to help you talk through an issue (I like the phrase “being a soundboard”) and paying attention to what others share about their work are two easy ways to start regularly incorporating collaboration into your work.

Even people who are the only evaluators on their team can incorporate this tip! Get involved in professional organizations like your local Canadian Evaluation Society chapter or check out interdisciplinary advocacy or learning events in your area. You can also stay connected to past fellow students or colleagues on a site like LinkedIn  to expose yourself to new developments in other fields.

It can also be useful to seek out learnings in casual interdisciplinary interactions. Any interaction with someone from another field or background, such as a project manager, an academic researcher, a nurse, or the Executive Director of a non-profit, can be a built-in interdisciplinary learning opportunity! Here are a few simple ways to make the most of these opportunities.

• Reflect on any differences in how they approach their work compared to how you approach yours. Could any of these differences potentially be beneficial to your work?

• Pay attention to the kinds of tools they use in their work, such as tools for collaboration, decision-making, or productivity. Could any of them be useful for you?

• Ask questions when you feel curious or when you don’t understand something.

• Think about how they do training and professional development in their role. Are there any resources you could learn from too?

---

3. Documentation

Psychologists need to document everything! From intake forms to case notes to communication logs to insurance documents, psychology practice requires strong procedures for documentation while protecting client confidentiality and privacy. Because they have so many records, many of which contain confidential information, psychologists typically follow clear processes for creating, sorting, keeping, and destroying documents. They often schedule blocks during the week to create case notes or update other files, for instance, and they adhere to record-retention procedures set by their professional organizations.

---

What can evaluators learn?

While evaluators may not use the exact same kinds of documents as psychologists, it is still important to be intentional about the ways in which we create, organize, and retain documents.

It’s a good idea to create a strong digital file organization system to help keep track of project documents, notes, and deliverables. Here at Three Hive & Eval Academy, we use a standard folder setup for each project (such as folders for Planning, Data, and Reporting) so everyone knows where to find a document. The organization system you use is up to you—create whatever categories make the most sense for your work—but just make sure you have a system! It’s also important to have clear record-retention and destruction policies and ensure everyone on your team is aware of them.

Evaluators can also take inspiration from the communication logs that psychologists might use to keep thorough records of any communication you have during a project. It’s helpful to schedule regular time to track these communications to avoid losing anything essential, make it easy to refer back to previous interactions, and create transparency in case you need to demonstrate what has been carried out. You could track emails with clients, survey invitations, meetings, phone or face-to-face interviews, or documents shared with you—and that’s just a start! What types of communication do you think could be helpful to track in your work?

---

Do these tips apply to your own work? Is there anything else you have learned from psychologists? Share with us in the comments!