Welcome to our monthly roundup of new and noteworthy evaluation news and resources – here is the latest.

Have something you’d like to see here? Tweet us @EvalAcademy!

New and Noteworthy — Reads

How Far Dare an Evaluator Go Toward Saving the World?

The American Journal of Evaluation recently published an article by Michael Quinn Patton (MQP) that outlines value statements from a diverse group of 40 evaluators working in philanthropic foundations. These value statements are an update to Robert Stake’s 2004 publication “How Far Dare an Evaluator Go Toward Saving the World?” The article includes a list of what evaluators care about and emphasize in their practice. MQP also includes his additions to the list, along with the facilitation steps and a process for illuminating evaluators’ values.

Checking our bias in “unbiased” research instruments: applying a diversity, equity and inclusion lens to instrument design

This recent blog post by Evidence for Action discusses how Diversity Equity and Inclusion (DEI) principles and practices can be applied in the design or evaluation of research instruments, such as survey questionnaires, cognitive or psychosocial assessments, tests, or checklists. This post also offers a supplemental methods note where the authors offer a snapshot of the instrument design process.

Creating Effectiveness Principles for Principles-Focused Developmental Evaluation in Health-Care Initiatives

The Canadian Journal of Program Evaluation’s spring issue contains an article that dives into the practical application of principle-focused developmental evaluation. The article focuses on three evaluations conducted in British Columbia and highlights lessons learned through the process of creating effectiveness principles.

New and Noteworthy — Tools

Useful Questions for Analysis and Reporting Challenges and Changes  

Evaluation Support Scotland recently shared a two-page document that lists key questions you can ask as you analyze your evaluation data and get ready to report on challenges and changes.

New and Noteworthy — Events

gLOCAL Evaluation Week 

Organized by: Global Evaluation Initiative

Dates: May 31 – June 4 

Building Forward Better: Towards Renewed Global Evaluation Agenda 

Organized by: Global Evaluation Agenda

Date: June 1; 5:30 PM (Rome) 

GEI – Evaluation Can Spark Change & Better Decision Making! 

Organized by: Global Evaluation Initiative

Dates: June 9 & 10; 10:00AM (EDT) 

Engaging Your Collaborators with Data Parties 

Organized by: Community Solutions Planning & Evaluation

Date: June 17; 9:30AM – 12:00PM (PT) 

Cost: $100.00 (USD) 

Using Data Analysis & Visualization to Drive Social Impact 

Organized by: Clear Horizon Academy

Dates: June 24 

Duration: 6 weeks 

Cost: $1200.00 (AUD)

 

In an evaluation consultant’s ideal world, potential clients would show up in our inboxes and ask us to be their evaluators because they’ve heard about how great we are or have worked with us before.

But in the real world, organizations must follow procurement policies or are seeking the best expertise or best value for money they can find, so they issue a public Request for Proposals. 

Evaluation consultants and firms who respond to RFPs know that some are good, and some are …not good.

A good RFP will yield good proposals. Unclear RFPs with overly burdensome submission requirements may drive away highly qualified firms or consultants, and what submissions do come may miss your intended mark.

If you’re creating an evaluation RFP, include these elements to get better submissions.

---

1. Information about your initiative

• Background: Tell proponents what your program does, when it started, who you serve, and where. 

• Existing evaluation documents: Share whether you already have a logic model or theory of change, an evaluation plan, or organizational policies about evaluation. You can share these documents with your RFP, you can make them available upon request only, or just let proponents know that they exist. 

• Available data: If you already have some data from registration or intake, or you’ve done some surveys, tell your proponents about that data. Knowing if you’ve already been tracking outcomes, for example, will help your proponent to design their approach. It’s helpful to share any details about the quantity and quality of the data, and the file format.

2. About the evaluation

• Evaluation purpose and questions: Tell proponents why you want to evaluate. Are you still planning your program, and want the evaluation to help develop it? Do you want to make decisions in the future? To learn as you go and inform program development? To meet your funder’s requirements? To share learnings broadly? If you have some evaluation questions in mind, describe them. 

• Project scope: Describe what you want to evaluate. Is it your whole program? A specific component? How many sites? How many clients/participants/patients?

• Timelines: Timelines should include when the work should start, when it should end, and any other important dates or seasons that matter. Think about when you need a report, or whether summer vacations or grant submission timelines might play a role in the project.

• Deliverables: You may not know all the details at this point, but share your thoughts on what you want the proponent to produce. A written report is a common deliverable, but there may be other information products needed along the way, such as interim reports, data summaries or progress updates, or you might want to produce a journal article or white paper.

• Budget: Let’s be clear: you need to include a budget in your RFP, whether that’s a ceiling or a range. If you only have $15,000 to spend, a submission for $275,000 isn’t going to help. “But won’t everyone aim for that top dollar amount? How do we get the best value for our money?” You can still score on pricing and value for money, but if proponents know how much available, they will design a much more feasible proposal. If you don’t know how much you should allocate, you may want to have a quick chat with an evaluator or a colleague to get some general estimates. Remember, deciding whether to write a proposal is a business decision for evaluation firms and consultants—they don’t want to waste their time or yours. 

• Methods: You might not need to list your methods, but if you MUST do a Social Return on Investment, or if you strongly prefer NOT to run focus groups, put that in your RFP. Consider leaving the decisions about specific methods to the planning process, after you’ve hired your fantastic evaluator. 

To see how evaluators scope out a project, see our Program Evaluation Scoping Guide.

3. About who you seek

• Evaluator/firm requirements: Share what you’re looking for in your successful proponent. Does your firm need to be local? Are you looking for a team or an individual? Do they need to have any specific credentials? How much, and what kind of experience do they need? While some content experience is helpful, it’s not always necessary; evaluation is a discipline with tools and approaches that can be applied to a wide variety of content areas. 

• Insurance requirements: Evaluators should hold professional liability insurance. You can clarify what amount of insurance your evaluator should carry – some common amounts are $1 million, $2 million or more, depending on your organization and the nature of the project.  

4. The proposal

• What to include in the proposal: Please. Please, don’t make your proponents work in templates or online forms. The proposal is a chance for you to see how your proponents write and design. Your final report probably isn’t going to be written in a clunky Word template, right? Listing some common elements that you want each proponent to speak to is helpful – content experience, methods expertise, capacity, proposed evaluation approach, project management, budget, team composition, subcontracting arrangements, references, resumes, etc. 

• Page limits: If you are limiting pages (and we don’t think you need to – if you’re looking for a concise proposal, just say so!), consider whether what you’re asking for can actually fit in the page limit. You’re probably not printing this document anyway, so a proposal with more pages but more white space will probably be easier to read.

5. The submission process

• Submission process: Hopefully, the days of couriering multiple copies of proposals and supporting documents are behind us. But if you require proposals to be named in a certain way, sent to a particular email address or to contain a specific subject line, list those details. Make sure the deadline for submissions is clear, including your time zone.

• Work samples: It can be difficult for evaluation firms to submit samples of their projects because many reports are prepared in confidence and not made publicly available. If you do require sample reports to be submitted, please clarify that you will limit sharing to the review committee, and delete the files after you’ve made your decision.

6. The selection process

• Proposal scoring: What will proponents be ranked on? Will a total score be calculated based on budget, approach, expertise? Your criteria might look something like this: 

• Shortlisting: Let your proponents know what happens after you score all submissions. Some organizations select their evaluation firm based on the written proposal only, while others prefer to invite a few for interviews before making their final decision.

• Communication timelines: Share your expected timelines for communicating with proponents. If your selection process is delayed, it is courteous to send a brief update to proponents—each of whom probably really wants to work with you. 

Following these tips should help you to attract high-quality proposals. Our final suggestion is to spend some time before writing your RFP to ask yourself the following questions:

• Who needs to know what? Who are our partners? How will they want to be involved in the evaluation? What do we expect to learn, and what do our partners need to know?

• What really matters? Is it more important that you find the least expensive option, or is deep content experience a priority? Are you open to creative approaches, or do you have a firm methodology in mind? Do you want a very structured workplan, or are you comfortable with the ambiguity inherent in a collaborative planning process?

• What is our budget? If you don’t have a precise budget in mind, you must have some range. Is the budget flexible? If you’re incredibly impressed with an approach, can you find extra budget to accommodate?

• Who decides? Do you have a review committee? Are they available to meet and make decisions together? How will you reach consensus?

• Is this RFP process genuine? If you already know who you want to hire, why not save everyone a lot of time and effort and just hire them? There are a few clues that proponents read as indicators that the winner has already been pre-determined:

  • A very short turnaround time

  • A request for very specific experience

  • Limited circulation of posting

---

Want to take these tips with you? Download our Evaluation RFP Checklist. 

Download our Evaluation RFP Checklist

Not sure where to post your RFP? Processes vary by region and by sector, but the Canadian Evaluation Society, American Evaluation Association, Australian Evaluation Society, European Evaluation Society, and other regional evaluation bodies offer RFP or tender hosting and distribution. 

 

To learn more about applying evaluation in practice, check out more of our articles, or connect with us over on Twitter (@EvalAcademy) or LinkedIn.

---

Sign up for our newsletter

We’ll let you know about our new content, and curate the best new evaluation resources from around the web!

Email Address
Sign Up

We respect your privacy.

Thank you!

---