The process of writing an evaluation report is like cooking. It can be a joyful and meditative process for some and an annoying necessity for others. Both cooking and report writing take practice; the more you do them, the more you refine your processes and find your own groove. While there is no formula to create a perfect reporting process, there are some key steps that can set you up for success.

Get to know your audience

Pick your recipe

The process of cooking starts long before ingredients hit the pan. For most, cooking starts with picking a recipe. When you are cooking for others it’s important to figure out what they are hungry for. Is your audience hungry for a full meal, or do they only have time for snacks? Are they meat and potatoes kind of people or does risotto and lobster tickle their fancy?

Ideally, your evaluation plan indicates which information your audience needs to know and your evaluation framework closely ties the data sources to the evaluation questions that need to be answered. Knowing what your audience is expecting helps you narrow your focus in the report writing process. In addition, like cooking, reporting writing is influenced by time, budget, quality and availability of data (ingredients), audience preference, and your comfort and skill level. Develop an evaluation plan (recipe) that will optimize these factors and be prepared to make adjustments as you go along.

Get to know your data

Grocery shopping

Getting to know your data is like going grocery shopping. In the data collection period, things don’t always go to plan. The quantity and quality of your data can vary, just like the availability and quality of produce in the grocery store. In some cases, you can even outsource your data collection, just like online grocery shopping. In this case, it’s important to examine the data you received for quality and ensure you have the information you need to answer your evaluation questions. Take some time to understand what ‘ingredients’ you have and how they fit with your original recipe.

Analysis

Prepping your ingredients

Analysis has been referred to as ‘slicing and dicing’ data, a cringe-worthy term that fits delightfully into this report writing metaphor. In cooking, how fine or course your chop up your ingredients is directed by your recipe. How you prepare your ingredients is going to affect the final product. Again, like grocery shopping, prepping your ingredients can be done by a sous chef, but how it’s done will affect how you, as the head chef, produce a final product. The recipe you choose will give you a plan and starting place.

In report writing, your analysis should be guided by your evaluation plan. Your evaluation plan lays out the questions you aim to answer and the indicators you have collected to do so. As you analyze the data, keep in mind how you plan on presenting your data. While a data analyst may be analyzing the data for you, a clear path helps lead to clear results.

Regardless of when you consider your report structure, it’s important to consider how the information fits together as you analyze the data.

Outline, outline, outline

Make a Plan

Next up, make your plan. You have all the ingredients, they are prepped, but what order do you tackle things in? I must confess, this is where the cooking metaphor breaks down a little. In cooking, you aim to have all the components of your dish ready at roughly the same time so that everything is served hot. Because of this, you need to be aware of timings and have a game plan in mind of what to tackle, when. In report writing, the parallel isn’t so strong. The pieces of data don’t all need to come together in a time-ordered way. However, outlining your report (developing your game plan) is a valuable step to help your findings flow together. The processes of outlining your report helps you develop a strong narrative in your writing. A strong outline will help you to remember your guiding questions and key findings as you write.

Key considerations when developing a report outline includes examining how your findings fit together. Does your data flow well together, or do you need a way to cohesively present distinct pieces of information? How will your audience best digest the information? Sketching out the structure of your report provides the structure that allows your audience to follow your logic.

Write

Cook

I’ve spent a long time discussing the cooking steps that don’t involve the actual transformation of ingredients into a dish, and this is on purpose. Preparation is key. Having a recipe, the ingredients to make the recipe, prepping the ingredients, and having a game plan of how they are all going to come together is half the battle. These are all steps you can do in advance to make the cooking process less arduous. The same holds true for writing your evaluation report. Once you understand what your audience wants, what the data shows, and how you are going to piece it all together into a cohesive narrative, writing the sentence and paragraphs is much easier.

Just like everyone’s cooking process is different — some use every pot and pan in the kitchen and leave the cupboards open as they go, while others work meticulously and stepwise — everyone’s writing process is different. There are lots of articles and pieces of advice on how to write; common tips include “eat the frog first”, or “always stop writing for the day when you have more to say.” Engage in your reflexive practice and identify how and where you write best. If you get stuck, got back to your evaluation questions. These are your touchstones.

Cooking can be a constant flow back and forth between prepping your ingredients and cooking. Sometimes you may even find you need to run out to the grocery store for more of something. When you are cooking, it can be helpful to grab a second opinion to taste as you go along. Have you been too heavy-handed with some flavours? Have you drifted too far from the recipe? Is the product well balanced? These principles also apply to evaluation report writing. Sometimes as you write you may notice gaps in the data and need to collect more, or you may find you rely too heavily on one set of findings and neglect to give the other findings time to shine. Getting feedback from peers about what they take away from your report can help you make sure you are getting your key messages across.

Editing

Trim the Fat

Most people tend to dread the editing process. You have poured your energy into crafting the pages in front of you, but that effort won’t necessarily be appreciated as-is. It’s time to trim the fat. Be ruthless. Do you need each sentence, each piece of information? How much can your audience digest? How much detail do they really need? Turn back to your evaluation questions — how does the data you present answer the questions and how much detail does your audience need?

In the world of cooking, consider how hungry your audience is. Are they full and can only handle small tidbits of digestible information? Or are they ravenous and will eat a 7-course meal without complaint? Your evaluation report is no good if you are providing a 7-course meal and your audience is only interested in the main dish.

Visualization

Food Styling

I have left visualization for last, but I don’t believe it comes last, chronologically. Rather, it’s a topic all to itself that many others have touched on and can and should come throughout the process. Sometimes visualizing the data can be just as powerful as writing about it. A good visualization helps you to make your point and a bad one muddies the water and puts your audience to sleep.

At this point, you may be overwhelmed at the number of hats you are supposed to wear—from recipe developer, to shopper, sous chef, head chef, and now food stylist. Don’t be alarmed. You don’t have to be a pro chef to learn some simple tricks that spruce up your presentation, like twirling the plate, not the spaghetti, to create an Instagram-ready plate of spaghetti. There are lots of simple visualization tricks you can employ to make your presentation more appealing (like Evergreen Data and Depict Data Studio). People eat with their eyes first and in the world of evaluation reports, appearance can make a big difference.

Let’s be honest, this part is often a little rushed and is not the time to be developing the stylistic elements of your report. It is, however, the time to be tweaking them. Small things that make a big impact include: using colour selectively, adjusting your heading styles, and adding white space so your words can breathe. Use stylistic elements like colour, icons, and fonts to link ideas and make key points stand out.

The visualizing and stylistic components are the icing on the cake — considered frivolous or extra by some, the thing you usually run out of time for, and something that can take ages if you aren’t careful — but they are essential and what makes people interested in trying what you’ve made. No matter how good your meal tastes, if it doesn’t look good, people aren’t going to want to eat it. No matter how good the findings in your evaluation report are, if they are not presented in a way that’s intuitive to understand and appealing, they are unlikely to gain much traction.

Conclusion

It’s time for an apology. I promised to show you how writing an evaluation report is like cooking, and I spent the majority of the time explaining how to do everything else before and after the writing part. Hopefully you can see that writing evaluation reports is not just about writing the words that make up the report. It’s a process that starts as the evaluation plan is developed and continues as you collect and analyze data. A strong evaluation report starts with a strong evaluation plan. Like cooking, there are steps you can follow to make the report writing process easier. Developing your reporting style and process takes practice. Don’t be afraid to get feedback from your peers and clients and try to have fun with it!

Email Address
Sign Up

We respect your privacy.

Thank you!

Working in the evaluation field is appealing in that it can take place across various sectors, systems and geographic locations. One area of particular interest for some evaluators, like myself, is program evaluation in low-resource settings (LRS). LRS, sometimes referred to as “resource poor” or “resource strained,” indicates countries or regions that lack the financial means to cover the costs associated with infrastructure, healthcare and/or trained professionals – as well as other system or societal needs.  These evaluations are often requested by funders or granting agencies as a means of providing evidence for effective use of funds. Likewise, evaluations are particularly important in LRS as programs need to operate as efficiently as possible due to limited human and material resources.

Program evaluations in a LRS can be challenging in that the program staff may not have the skills or capacity to see it through. However, agencies often require formal evaluation of their funded programs which may lead to the hiring of a contract evaluator. Take USAID for example, they provide funding to LRS programs all over the world and emphasize the need to embed evaluation in program planning. USAID values evaluation so much that they have developed policies to enforce and support agencies in meeting evaluation requirements (see USAID Evaluation Policies).

In my (very biased) opinion, I think that LRS evaluations can be complex, but in the long run will help programs as well as the population and country in which they take place. Moreover, evaluation is often viewed as something that must be done for funders, but in many/most cases is just as valuable to the program itself and it provides a source of evidence to inform program planning. As evaluators we need to advocate for the use of evaluations for both funders and their funded programs – the best way is to show what we can do by conducting meaningful evaluations. We already have the skillset to conduct evaluations of all shapes and sizes (impact, development, process, etc.) – but in a LRS we have to be a bit more creative and thrifty in our approach.

As such, I offer four strategies for success when evaluating within a LRS: Be Tech Savvy, Consider Capacity, Be Ready to Adjust, and Use Yourself as a Resource. As you explore these strategies, I am confident that many will already be a part of your practice in all settings (not just LRS). However, I would encourage you to lean in to the strategies even more when you find yourself in a LRS. Lastly, all evaluations in both high- and low-resource regions are susceptible to being thrown curveballs (take COVID-19 as an example) – meaning that as evaluators we need to be ready to engage new strategies on short notice.

Be Tech Savvy

So you find yourself in a LRS or perhaps a very low budget evaluation, but you still need to exchange information and data with stakeholders. In this case, you could even sub out LRS for “pandemic” and the following advice will still apply.

Even if the program has been using the same spreadsheet since 1995 or has a VPN that takes what seems like 2 days to connect – try to utilize existing systems. When capacity is limited, implementing new systems may result in lost time or frustration.

If you find yourself in a setting where staff are eager to have better software or have requested new systems for their information – start by being proud of your forward-thinking team/client but be cautious of available capacity for both finances and time. What I mean by this is to utilize free (or low cost) and easily accessible software before signing the organization up for a pricey software package.

• Sharing Information: Consider free tools such as G Suite or DropBox to share files and work on documents (reports, spreadsheets, presentations) simultaneously. These are less expensive (or free for basic accounts) compared to platforms such as Office 365. Note that many of these tools have offline capability meaning that you can work on the documents even when internet connection is limited.

• Communicating: The COVID-19 pandemic showed us the many available resources for connecting. In my experience, many of the free services are just as good. Zoom or GoToMeeting are popular, but only offer a free trial period or require a subscription for longer meetings. Alternatively, well-known platforms such as Skype, Google Hangouts (part of G Suite), or even Facebook through their Messenger Rooms can be used for free video calls. WhatsApp and Slack are also great apps for communicating with teams or clients. These are just a few examples of the many communication tools available to evaluators.

Consider Capacity

Evaluations can often be perceived as a capacity strain, whether it be through using staff time or the cost of hiring an external consultant. Given the capacity limitations inherent to LRS, evaluators should keep capacity at the forefront of their minds from the first meeting to the final iteration of the report. Some examples of capacity considerations during an evaluation include:

• Developing the work plan and timeline: Try to plan meetings so that they occur only when necessary and include relevant stakeholders. Regular check-in meetings may not be helpful and may take staff away from program delivery – consider 1-2 page status reports instead.

• Creating evaluation plan/questions: We often evaluate efficiency of programs, but it would also be helpful if evaluators put more of an emphasis on capacity or even a sub-section focused on capacity. Develop questions that are feasible in a LRS and likely to uncover actionable findings (not just funder-mandated metrics). Check out Eval Academy’s How to Write Good Evaluation Questions for guidance on writing evaluation questions.

• Collecting data: Try to pull from existing data, add questions to regularly administered surveys, or plan focus groups for days where stakeholders are already in the same location. If appropriate, consider methodologies such as Participatory Action Research (PAR) to build evaluation/research capacity of staff (Baum et al., 2006).

• Presenting findings & recommendations: Include capacity building or capacity considerations for all recommendations – after all, recommendations are not likely to be adopted or sustained without capacity.

Whether you are an internal or external evaluator in a LRS, the program or agency should feel like you contributed capacity whether it be through the new knowledge, sound recommendations or development of internal evaluation capacity.

Be Ready to Adjust

As evaluators we like plans and go into projects with a clear timeline and guide of what we would like to accomplish. Does it always go as planned? Definitely not! In all engagements we need to be ready to pivot or adjust our plan – this is even more true for evaluations in LRSs.

As I write this article, we are still deep in the uncertainty of COVID-19. Although many evaluators are already comfortable with remote working, many aspects of conducting an evaluation have had to change. Couple this with the complexities of a LRS and there is no choice but to be flexible and ready to adjust evaluations accordingly.

I expect that evaluators will have many successes (and some failures) to share about their evaluation practice during these times. For now, I will offer a couple suggestions that I have found to be helpful working on a LRS project during a pandemic.

• Information gathering: Focus groups, interviews and surveys may need to be facilitated online. Likewise, program staff may need to extract data and share with the evaluator remotely. Workshop facilitation may require the evaluator to search for facilitation tools embedded in video chat platforms (such as surveys) or collaborative tools (such as a shared whiteboard).

• Focus on the most important aspects: Some key evaluation topics or focus areas may need to change. Consider resource constraints or competing priorities (pandemic related or not) and the impact on the program you are evaluating. Ask questions like:

  • Are the evaluation topics/questions still valid?

  • Are the various evaluation phases feasible remotely and/or with less access to staff and the population being served?

  • How can evaluation evidence support the program as it adapts for COVID-19 or other resource constraints?

• Revisit the timeline: Expect the timeline to change – whether it is rescheduling some of the meetings, changing them to virtual meetings, or completely revising the project work plan. For example, if the evaluation capacity is severely limited due to a pandemic or other LRS obstacles, look at postponing until there is more capacity. It may be better to postpone or extend the timeline rather than sacrificing the quality of the evaluation.

Use Yourself as a Resource

In a LRS, answers to your questions or material to inform both the planning and execution of an evaluation may not be easily accessible (or in some cases may not be known). As an evaluator for a LRS program, internal or external, you will likely need to be your own resource when it comes to finding data, identifying stakeholders or to develop a general understanding of the program. Hopefully the program or agency at the center of the evaluation will be willing to share all relevant documents and offer some context – but in most cases you will need to be ready to dive in to uncover more. Here are a couple of tips to further explain this strategy:

• Hands-on learning: Rather than taking the capacity (resources or staff) to learn about the program or agency being evaluated, consider shadowing or observing the program (activities and meetings). This will prevent the evaluation from eating up too much staff time and I would argue that it will also provide you with a richer understanding of program being evaluated.

• Self-led professional development: There may be minimal options for professional development in LRS, especially if you are working as an internal evaluator. Connecting with other evaluators or professionals working on LRS projects is a great place to start. There may be existing Communities of Practice (in-person or virtual) or in my experience individuals working in the same region are more than happy to share their experiences. For evaluation specific education or new methods (even seasoned evaluators need some inspiration every once in a while!) consider online resources, like EvalAcademy.

These strategies were summarized with LRS evaluations in mind, but I am confident they can be adopted for evaluation projects in all settings. If you have any other resources or strategies for evaluating in a LRS, please comment below.

Resource

Baum F, MacDougall C, Smith D. Participatory action research. J Epidemiol Community Health. 2006;60(10):854–857. doi:10.1136/jech.2004.028662

Email Address
Sign Up

We respect your privacy.

Thank you!