The May 13 Group

the next day for evaluation

You are here: Home / Archives for allblogs

allblogs

What to Put in Your Evaluation Contract

 

Congratulations, evaluator, you’ve landed a client! Or maybe your organization has selected an evaluator to complete an exciting project. Either way, your next step is a contract for evaluation services. 

It’s always safest to have a contract in place before any work starts. As the contractor, it helps ensure that you will be compensated for your time and efforts, and as the commissioner, the contract helps you protect your information and confirm the services and deliverables you expect.

Agreeing on contract terms can be complicated. For more complex evaluation projects, you may want to consider two separate contracts: one for evaluation planning, and a subsequent contract outlining the scope of data collection, analysis, and reporting.

Contracts may be drafted by either the evaluation commissioner or the evaluator/firm hired. Smaller organizations may be relieved to have the evaluation firm initiate the contract, while larger organizations may have internal procurement processes that must be followed. 

Monochrome Mountain Landscape Photo Travel Postcard.png

Below are some components you should include, or at least consider, in your evaluation contract.

Please don’t mistake this article for sound legal advice, though! If you have the resources, it’s best to have a lawyer review your contract template or individual contracts.

1. Names

Well, this one is probably obvious. But be sure to include each entity’s legal name, and possibly their “operating as” name if applicable.

2. Term

Define the dates the contract begins and ends. An end date may be difficult to assign for complex projects. It may be prudent to make the contract end date a few weeks later than you currently expect to hand over the final deliverable.

It can also be helpful to include a statement indicating that the contract term will be revisited at a certain date (perhaps one month or two weeks before the end date) and possibly extended. Many events beyond your control may change the timeline, so giving yourself some additional time in the contract may save some paperwork.

3. Compensation

Set out how payment is determined, whether by an hourly rate or a cost per deliverable. Indicate when invoices will be delivered (Monthly? Quarterly? Upon completion of certain deliverables?) and to whom.

You can also describe what needs to be included on those invoices; some clients prefer something akin to a timesheet, while others may be satisfied with a list of activities or products delivered. Compensation details are sometimes included as a separate schedule to follow the main body of the contract.

4. Expenses

Clarify who is responsible for any expenses such as travel, focus group incentives or printing, and whether expenses should be approved in advance.

5. Deliverables or Services Provided

The details of deliverables or scope of services are sometimes included in a schedule appended to the main contract. This section requires both parties to assess their level of comfort with ambiguity. Do you want to be very specific, outlining each step of the process and the details of the deliverables, or more general? Using general language introduces more room for interpretation – and therefore more risk – while very specific language may limit the ability to adapt to emergent situations or needs.

Some contracts may detail the headings or sections required in a final report, while others may simply state that a final report is required. Think carefully about whether any raw data files are to be provided as deliverables; clients sometimes want this information, but your professional ethics as an evaluator may prevent you from providing this kind of deliverable.

6. Acceptance/Approval of Deliverables

Who determines when something is done? It can be very helpful to clarify this responsibility upfront. Is a formal signoff required, or is verbal confirmation acceptable?

You may also want to clarify the process for reviewing document drafts; for example, a request to incorporate potentially conflicting feedback from eight separate committee members means something very different to the evaluator’s time than addressing feedback collated by the contract manager.

7. Termination

All contracts should have some sort of exit clause. Not all projects run smoothly, so it’s best to have guidance for how either party can end the contract. Define how much notice must be given, and how the evaluator will be compensated for work completed.

Termination clauses often list several different scenarios that are grounds for termination, such as contractor bankruptcy, non-compliance with contract terms, refusal to perform services, or even contractor death.

8. Non-competition and Non-Solicitation

The evaluator will be in a position of confidence, accessing information that is not publicly available and potentially sensitive. This clause can define the contractor’s obligation to refrain from using the organization’s confidential information to further their own business interests.

9. Confidentiality

Be sure to outline the evaluator’s responsibility to keep the organization’s information confidential and secure. This clause may reference a separate confidentiality agreement or constitute an agreement to protect confidentiality on its own.

10. Intellectual Property/Ownership

The evaluator will likely be producing data collection tools and knowledge products as part of the contract – who owns these materials? Who can use them, and under what circumstances? Survey tools, for example, may be developed exclusively for this contract – do they belong to the evaluator or the organization that hired them?

In this section, you can also indicate how evaluation reports should be referenced, or whether the evaluator can share reports with potential new clients as work samples.

11. Dispute Resolution

Describe the process to be followed in case of any disputes. Will a third party be engaged to remediate? Or are there existing internal processes to be followed? In our experience, formal disputes are rare, but this clause will provide guidance should any arise.

12. Assignment and Subcontracting

Assignability clauses outline whether and when a contract can be assigned to another entity. If the evaluator determines they are unable to undertake the work as described, can they hire another firm or individual to complete it? Many evaluation contracts are non-assignable – after all, the organization has selected the evaluator they thought best for the job. Based on past experiences, they may be wary of a “bait and switch” situation.

Subcontracting a portion of the work (for example, transcription, data entry, or data collection at a remote site) may be perfectly reasonable – just be sure to clarify when subcontracting arrangements may be made, and whether the evaluation commissioner has a role in approving the firm or individual being subcontracted.  

13. Other Legal Bits

Most contracts will have several other clauses that are relatively immaterial to the evaluation work, but important for legality. These include standard clauses on interpretation, “entire agreement,” enurement, unenforceable terms, and limitation of liability. For these sections, be sure to have your lawyer provide or review some text. 

14. Signatures

The final piece! Provide space for each party to sign, along with their name, role, and date. 

15. Final Copies

In our experience, most contracts are signed digitally rather than on paper. But if your contract requires physical copies, clarify which party will hold the original, or whether two original copies must be signed and held by each part.

PRO TIP: If you don’t already have a relationship with a lawyer, ask your business-minded friends for a recommendation, or try a service such as Upwork or UpCounsel  to find short-term support.

To learn more about applying evaluation in practice, check out more of our articles, or connect with us over on Twitter (@EvalAcademy) or LinkedIn.


We respect your privacy.

Thank you!

 

Why Nonprofits Shouldn’t Use Statistics

Today’s article comes from Maryfrances Porter, Ph.D. & Alison Nagel, Ph.D of Partnerships for Strategic Impact. They were recently guest speakers in our Simple Spreadsheets course and had so many great insights! – Ann

—

Thank you to Ann Emery, Depict Data Studio, and her Simple Spreadsheets class for inviting us to talk to them about the use of statistics in nonprofit program evaluation! If there’s one thing that we never want to miss, it’s an opportunity to tell people their job is easier than they think!

This is why we created ImpactStory™ Coaching – because it’s actually within reach of small- and medium-sized nonprofits to be clear, confident, and convincing when talking about their impact!

It was also just nice to talk to a group of smart, creative, data-minded professionals who also (yes – that’s right – also) feel overwhelmed by the prospect of having to use statistics.

We used to think our feelings of statistical overwhelm in the nonprofit space was because we were wimps (even though we have literally taken a combined SIX YEARS of graduate-level statistics*).

But then we realized that much of the time, statistics just don’t have much of a role in nonprofit work. And here’s why!

Why Nonprofits Shouldn’t Use Statistics

When working in the nonprofit world sample size is usually very small (i.e., the number of clients served in any given year is usually 40 to 400 people).

Even if you have more people than that (e.g., a school district’s worth of students), it’s still unlikely you need statistics, unless you are trying to answer a scientific-type question (and what scientific-type questions nonprofits with a lot of data might ask is for another blog post on another day).

Simply having a statistically significant group of survey respondents for such a small number of people means you have to get surveys from A LOT of people: 37 of 40 and 196 of 400! This is really hard to do (although we do have tricks for making sure you get surveys from almost 100% of your clients)!

Statistical significance is often mistaken to mean a big difference, but what it really means is a not random difference (e.g., if you looked at a different group of people, you’d find that difference again… it’s a reliable difference). When you have a small number of people, it also means you must have a GREAT BIG DIFFERENCE to get statistical significance.

The math just works out that if you look at 1 million people, just about any finding is statistically significant (e.g., a tiny difference in a big group is almost always not random), but when you’re looking at 100 people, you must have a really big difference to get statistical significance. In the science world, if you have a small group of people and do not find statistical significance, one thing you can do is test a much bigger group!

An older man and a younger man facing each other and smiling.

A Nonprofit’s Mission is to Serve as Many People as Possible to Address an Identified Need

In order to use statistics to identify the impact of a program, you usually need a comparison group (e.g., a random group of people who do not get the program) to which you compare your clients. Ideally, both groups are selected randomly: the people getting the program, and the people not getting the program. (We all know that’s not happening!)

Realistically, the comparison groups available to a nonprofit are either people the nonprofit randomly refused serve – or – a very nonrandom group of people who didn’t want the services the nonprofit was offering. Scientifically, neither of these are good options for comparison groups.

And we’ve never met a nonprofit so flush that they had money to track people they don’t serve. Even if a nonprofit had money to spare, spending this way would not be aligned with its mission to serve as many people as possible.

A sidewalk that has written on it, "PASSION LED US HERE" as two people stand looking down at it and their feet.

Nonprofits are Not Set Up to Follow People for a Long Time after Service Provision (e.g., 6 Months to 40 Years)

Most nonprofits provide a service for a specific amount of time, people graduate from that service, and then they go on with their lives (hopefully with more strategies to reach their goals). Staying in touch with people over time is very time consuming and very, very expensive – especially if you want to stay in touch with at least 80% of people (which is a minimum in the scientific world).

If you’re a smaller nonprofit that means you have to track 32 to 320 people over time (to follow up on those 40 to 400 clients you served). Frankly, this is both impossible and still too small to analyze with statistics (see point #1 about statistics with small groups of people).

Using scientific methods to test hypotheses (which are what statistics test) are what scientists do; delivering programming and tracking client progress is what nonprofit practitioners do. 

We have used this example before:  Scientists discover and test medicines to make sure they work. Doctors deliver what’s been shown to work and make sure the people they treat get better.

Two. Separate. Jobs. 

You – our nonprofit friends – are doctors.

A healthcare professional is taking the blood pressure of a seated patient.

Nonprofits Focus on Working with Individual People and Complexity Not Populations and Averages (Which is the Realm of Science)

It’s a cognitive error to assume that statistics (which typically focus on averages) applies to individuals.

For example, the average number of car crashes a person gets into in a lifetime is four (this is scientific knowledge derived from statistics) – but we all know people who get in many more crashes and people who never get in a crash (this is the reality of being an individual in the complexity of life).

And, if you – as an individual – get into four crashes that does not mean that you are now immune to getting in crashes (this is the cognitive error of applying statistics to individuals)!

So, drive safe and buckle up!

Also, if you are concerned about diversity and equity then you need to have more people from marginalized groups from whom you gather data so you can really hear what they have to say. Period.

You do not want to just have a representative number (e.g., a number equal to the proportion in the larger population) because their voices get washed out in the average.

When doing nonprofit work: Each. Individual. Voice. Matters.

What Can Nonprofits Do?

Nonprofits Should Think of Themselves as Conducting Qualitative Analysis with Numbers and Stories

Qualitative analysis basically means you are looking for patterns and changes in patterns in both your numbers data (what people report on surveys) and your stories data (what people tell you in words).

You’re examining how the data look – the shape, the themes, the patterns that emerge, and when the patterns change.

Your Data Team is the litmus test for determining which things are important and meaningful and which things are not.  Data Teams are for answering questions in real life; experimental design and statistics are for answering scientific questions. (Ask us more about Data Teams!  We love to talk about them!)

Board showing notes and pictures trying to determine which is the most important information.

You HAVE TO GRAPH Your Data to See How it Looks

If you do use math at all it’s probably only to calculate the percent difference, the percent change, and maybe a risk ratio. This means you count how many people say something and how many people didn’t say that.

Graph all the answers in both groups. Then break the groups up differently to better understand the patterns of responses (e.g., males and females, comparisons based on race or income or zip code or classroom or age. . . you get the picture). If you don’t graph your data, you’re sunk. 

You simply have to graph it to see what it’s doing. Mostly bar charts (to compare groups) and line graphs (to look at stuff over time).

Bar chart showing ages from 0 to 100 broken down.

Here are some examples:

  • Count how many people said “Strongly Agree” and “Agree” compared to “Disagree” and “Strongly Disagree.” What’s the percent difference between the two groups?
  • Decide how different these counts are: meaningfully different or slightly different? The best way to make the most valid assessments of how meaningful the differences are is to use a Data Team. (We love a good Data Team!)
  • Based on what you know about the people you serve, as well as changes in the community and at your organization, what do you think might be the reasons for those differences? (These are follow-up questions your Data Team can ask during their data review meetings.)
  • Think about how you might divide the groups into different groups, or subgroups, to explore deeper questions (e.g., males and females, wealthy and financially struggling, graduated and not graduated, etc.). If you are looking at disparities, what’s the risk ratio of one group having a poor outcome compared to the other?
  • If you have data over time (i.e., surveys from the same people at different times) – you may want to look at percent change happened over time?
Line chart with headline that reads, "What happened to women in computer science?"

What Software Should Nonprofits Use?

99% of the time. . . Excel.

If you have data with lots of complex relationships (e.g., data from parents and children, over time, in different programs) you probably want to be using a database like Apricot.

Then you can create and run reports that graph your data with the touch of a button!  And you can still can create downloads of the data behind those reports and create your own graphs in Excel.

If you have many hundreds or thousands of people you are serving, then it’s just easier to clean and sort that data in a statistical package like SAS, SPSS, or R. 

In these cases, we choose to hire someone who’s very good at these programs (like a graduate student taking a stats class) and pay them like $30-$40/hour to clean the data, maybe do some descriptive statistics and show us some averages. Then we have them download the clean dataset (a delimited CSV) and we pull that into Excel for graphing!

Tableau is great for being able to create dashboards you can manipulate and post on the web. But you actually have to know what graphs you want before creating them.

So, we’ll create the graphs we want (in Excel!), and then hire someone to transform the data and recreate those graphs in Tableau so nonprofit leadership can manipulate them or post them on the web.

We hope all this is some weight off your shoulders!  Sign up here to stay connected with us and follow us on all the social media! We have lots more to share!

Connect with MaryFrances Porter & Alison Nagel

Partnerships for Strategic Impact: https://impactstorycoaching.com/

Maryfrances Porter, Ph.D.  – LinkedIn: @maryfrances-porter-psi/

Alison Nagel, Ph.D. – LinkedIn: @alison-nagel-41493a125

How to create a basic logic model [activity book]

Working with a group to create a logic model? I created a little activity book to help you on your way.

Prerequisite (Before you Download)

This activity book assumes that you are trying to take action and do something. Or that you’ve already taken action.

If you haven’t, you’ll have to imagine yourself taking action on something to get any use out of this book.

What is in the activity book?

Basically just a series of question based activities. Instead of getting bogged down in jargon, the activities start with simple exercises.

Excerpt from the Activity Book (Activity 4)

It starts with these.

  • What are you trying to do here?
  • Name three people who a stake in what you are trying to do.
  • What if those three people answered that first question?

Then we dive into thinking through actions and consequences.

  • If I do this…this happens.
  • From actions to outputs.
  • From consequences to outcomes

Finally we pull it all together.

  • Assembling a basic logic model.
  • Intentionally complicating your logic model.
  • Bonus activity: coming up with a counterfactual.

How much and where to download.

The personal non-commercial version is free but tips are appreciated.
(Just put in $0 to download for free, no judgement). There is also a print-away version that requires at least $10.

Here is a button:

Download the Activity Book

If that doesn’t work, you can download it directly from Gumroad. It’s also on my downloads page.

How many can I print?

Totally okay to print the free version for personal use.

If you want to print several for a get together with your small non-profit, also okay to use the non-commercial version.

I know some of you teach classes, feel free to print it for your students (or send them a copy you downloaded) of the non-commercial version.

Do you make money as a consultant or trainer doing this kind of thing? That’s why I created a commercial version. But if you buy that, feel free to print away as much as you would like.

But I just want logic model cartoons…

For that I suggest either buying my book.

Supporting me on Patreon (my Patrons get access to all my cartoons. They will also get the above commercial version activity book downloadable on Patreon for nothing extra).

Or just pulling them from my big post of evaluation cartoons (I have a bunch there).

Designing for Humans: Considerations for Innovators

Over at our sister site Censemaking we wrapped up a series called Design for Humans. The focus of the series was on different considerations that we must make in the process of designing useful, impactful things for the world around us.

Strangely enough, we often don’t do a good job at designing for humans with our services and products. In the process of innovating, we make many assumptions about how the innovation will be received and the impact it will make.

There are reasons why real substantive change is actually very difficult to generate. Humans are difficult to work with and to influence and the more we can design for how humans actually live, think, work, and play the more likely our change-making efforts will yield positive results.

Deeper Research

One of the ways to do better design for humans is to conduct detailed research. This goes beyond a few focus groups or observations toward more fuller engagement with different audiences. Strangely, many of us actually don’t really know what we want. We think we know what we want or that we know how to articulate what we want.

Research in this context involves the time, care, and attention to who people see themselves and who they act as. This is about our imagined reality and our lived one. (The two intersect and diverge through our days and life).

Many of our purchases are based on decisions that involve aspiration, comfort, fear, convenience, ethics, and utility in different combinations. These are both our own and the ones that we adopt from others depending on our situation. Individual decisions are very often tied to collective ones. If we do not understand the different groups we are affiliated with or aspire toward being a part of, we lose much of what is known about what drives a decision.

If we do not take the time to understand who it is that we are speaking with or seeking to influence — their individual set of values, beliefs, and experiences — then we will create a ‘mass’ for a ‘mass market’ that is unlikely to be receptive.

Luck and Science

A great deal of impactful design is based on a dance between luck and science. Viral marketing campaigns are more brilliant in retrospect than in advance. As Tendayi Viki and Mitch Joel recently spoke about in their conversation on innovation, most great innovations look absurd in advance and clever in retrospect.

This means great design is about framing the ridiculous.

It also means ensuring that, practically speaking, we must consider the very way we humans think, feel, and live — which is often ridiculous to the outsider. Great campaigns and products tap into this.

They also are about luck. Designing for luck through great observation, research, testing (prototyping), and ongoing evaluation ensures that what we put into the world has the potential to succeed in making some positive difference, not necessarily that it will no matter how good it is.

Whether its luck, perception or something else we find ourselves recognizing that the gap between what we think is valuable and useful and what others think (or are willing to consider) is large. The Chasm illustrated above is real and rather than engage in a futile effort to design around it, we need to recognize that there are sharks in those waters.

That’s what they did in the film Jaws. When we design for that, we design for humans (and sharks).

If you’re looking to create something more human in your work and need help, contact us and we can help make sure you don’t get eaten by the sharks you’re swimming with.

Images by Business Illustrator used under license.

The post Designing for Humans: Considerations for Innovators appeared first on Cense.

en English
af Afrikaanssq Shqipam አማርኛar العربيةhy Հայերենaz Azərbaycan dilieu Euskarabe Беларуская моваbn বাংলাbs Bosanskibg Българскиca Catalàceb Cebuanony Chichewazh-CN 简体中文zh-TW 繁體中文co Corsuhr Hrvatskics Čeština‎da Dansknl Nederlandsen Englisheo Esperantoet Eestitl Filipinofi Suomifr Françaisfy Fryskgl Galegoka ქართულიde Deutschel Ελληνικάgu ગુજરાતીht Kreyol ayisyenha Harshen Hausahaw Ōlelo Hawaiʻiiw עִבְרִיתhi हिन्दीhmn Hmonghu Magyaris Íslenskaig Igboid Bahasa Indonesiaga Gaeilgeit Italianoja 日本語jw Basa Jawakn ಕನ್ನಡkk Қазақ тіліkm ភាសាខ្មែរko 한국어ku كوردی‎ky Кыргызчаlo ພາສາລາວla Latinlv Latviešu valodalt Lietuvių kalbalb Lëtzebuergeschmk Македонски јазикmg Malagasyms Bahasa Melayuml മലയാളംmt Maltesemi Te Reo Māorimr मराठीmn Монголmy ဗမာစာne नेपालीno Norsk bokmålps پښتوfa فارسیpl Polskipt Portuguêspa ਪੰਜਾਬੀro Românăru Русскийsm Samoangd Gàidhligsr Српски језикst Sesothosn Shonasd سنڌيsi සිංහලsk Slovenčinasl Slovenščinaso Afsoomaalies Españolsu Basa Sundasw Kiswahilisv Svenskatg Тоҷикӣta தமிழ்te తెలుగుth ไทยtr Türkçeuk Українськаur اردوuz O‘zbekchavi Tiếng Việtcy Cymraegxh isiXhosayi יידישyo Yorùbázu Zulu