In most evaluation plans, we try to define scope. Defining evaluation scope means putting boundaries around things like:  

• Time – defining what timeframe of the initiative you will include in your evaluation, or how long you will be undertaking evaluation work 

• People – deciding which clients’ or stakeholders’ perspectives will be included, and which won’t 

• Components – clarifying which parts of a program or initiative to evaluate, and which to exclude 

The scope of your evaluation project should be clearly documented in your evaluation plan. That documentation will be important throughout the life of the evaluation project. As an example, here is a scope statement from one of our recent evaluation projects: 

The scope of this evaluation will vary between operational phases. Generally, the scope will be limited to activities related to this initiative from Red Deer and north in Alberta, although some alignment of evaluation activities with activities in southern Alberta may be incorporated. While past programs and systems inform the initiative, this evaluation will not investigate any of those previous programs or services. This evaluation plan addresses Phases II (April 2021 – September 2022) and III (October 2022 – March 2024). 

 And another:  

 All evaluation projects face limitations in resourcing and feasibility. This evaluation will include all patients and providers participating in the initiative. Data elements will include what can be reliably reported by patients and providers through surveys and through patient interviews, as well as data captured in the pharmacy information platforms. All activity between project initiation (March 2021) and project close (March 2022) is within scope for this evaluation. The perspective of others in the healthcare system, including family physicians and health authority staff, are considered out of scope for this evaluation. Patient health outcomes are not in scope for this evaluation. 

Ideally, our evaluation projects would proceed as planned. But as all project managers know, sometimes things change. Actually, most of the time, things change! 

In some situations, our evaluation approach can be modified to adapt to the changing context, but in others, we have to say no to scope creep. 

What is scope creep?

The term “scope creep” comes to us from the project management world. Definitions about, but essentially scope creep is when the project work begins to extend, or “creep,” beyond what was originally agreed.  

 Many evaluators will have received requests from clients to add in a thing or two here and there after the evaluation project is already underway – maybe another data source has surfaced, such as a survey run by a partner organization that seems relevant. Maybe they would like to add another stakeholder group to be interviewed. Or maybe they would like to include activity data from the last fiscal year, despite originally wanting to focus only on the current year. Some scope creep requests relate to reporting – an extra presentation here, a customized sub-report there.  

 Or perhaps it’s the evaluator that wants to expand the scope. You may have identified a new opportunity to strengthen your evaluation project by incorporating a new analytic approach, or early findings suggest there is merit in pursuing an additional line of questioning. There are times when expanding the original scope of an evaluation project can open up important insights that may have seemed inaccessible during the planning phase.  

 But beware! Scope creep isn’t always so obvious. It can creep up slowly, and before you know it, you’ve added a couple of survey questions, provided a third draft of a slide deck, and interviewed management staff that you hadn’t planned to.  

What to ask yourself when you discover scope creep

When you see scope creep happening, take a quick pause to reflect. You might need to say to your client or team: “Let’s just take a few minutes to see where this fits in our project scope.” Referring to the scope statement defined in your evaluation plan will support the conversation or introspection that follows. Whether you are acting as an external evaluation consultant or as an internal evaluator, there are a few questions you can ask to help make the “to scope creep or not to scope creep” decision.  

• What is motivating this request? 

• Are there ethical implications to this new task? 

• Does this new request add value?  

• Do we have access to the data source required for this request? 

• Do we have the time to complete this extra task? 

• Is there room in the budget? Or, can additional funds be found? 

There’s no easy algorithm for making your choice about project scope. Here are a few considerations that can help you make the best decision.  

When should you indulge in scope creep?

We can’t anticipate everything when we’re designing an evaluation plan, and there is a very real possibility that a meaningful opportunity to create a better evaluation will arise. If the option to strengthen your evaluation approach arises and:  

• it is ethical,  

• it doesn’t unfairly burden any individual or group, 

• it is in alignment with the overall purpose of the evaluation,  

• it can be done well, and  

• it is feasible within the timelines and budget allocated… 

then go ahead and creep! Your original evaluation plan doesn’t need to restrict innovation—one of the points we’ve made about evaluation project management is that you shouldn’t live and die by the plan. 

When to avoid scope creep

On the other hand, some project expansions are best avoided. If the potential expansion appears to: 

• be motivated by a desire to bring a more “positive” conclusion, 

• be too far outside the original purpose, 

• have no ready, reliable data source, 

• place too much burden on a particular individual or group, or 

• require more investment than it’s worth, 

then maybe you need to push back. And to be very practical, sometimes you just can’t accommodate new requests because of important deadlines or budget limitations.  

 If you indulge scope creep when any of those red flags are present, you run the risk of damaging relationships or delivering a late or sub-par evaluation product.  

If you do decide to expand your scope, make sure you document that decision (Remember the JCSEE Program Evaluation Standard on Evaluation Documentation?). Savvy project managers will have a change log to note any changes and their reasoning. You should probably update the evaluation plan, noting the date of the scope change, and share with the team. There may even be a place to include the rationale in your evaluation report, perhaps in an appendix.  

We hope these points are helpful the next time you need to make a decision about scope creep. 

Email Address
Sign Up

We respect your privacy.

Thank you!

As an evaluator, a big part of our role is to collect data. Sometimes that data comes from administrative databases that require data sharing agreements, but sometimes it comes from people, who need to consent to sharing their information (or data) with you. So let’s talk about what consent is (and isn’t).

I’ll share the punch line with you up front: consent doesn’t have to be a scary, overly burdensome process. If you keep some key principles in mind, obtaining the right consent can be pretty straightforward.

There are different types of consent, many of which don’t apply in an evaluation context: consent for medical procedures, legal consent, etc. It is this legal consent, and the “signed consent form” that most people think of when they think of consent.

Consent, broadly, is giving permission for, or agreeing to, something happening. Informed consent means individuals consent with full knowledge of the nature of participation, alternative options, and potential benefits, risks, or consequences to them (more on this below!). As the evaluator, you need to respect an individual’s choice to participate in your evaluation. For an individual to participate, they need to know what those pros and cons or benefits and risks are! Finally, assent is given on by a person unable to provide their own consent (e.g., children or adults with disabilities).

As an evaluator, it is best practice to consider the consent process through an ethical lens, and then to document that process and the decision-making rationale. The documentation serves to provide evidence of the process.

What is Informed Consent?

Informed consent requires that the participant be informed of: 

• all the known or suspected risks and benefits of participation 

• the complete nature of participation (that is, what are they being asked to do or share) 

• the withdrawal process (if there is one) 

• their rights to access their own information 

• benefits to the project 

• what will be done with the information 

• who will have access to their information and confidentiality will be maintained 

• compensation or expenses 

However, writing this out in a consent form is not the end of informed consent. An important part of consent is that the participant is provided the opportunity to ask questions or clarify information. This dialogue is important to ensure the potential participant has all the information they need to make an informed decision, but also for the project lead to ensure that the participant properly understands the information.  

A review of consent literature has shown that there are major opportunities for improving the consent process! Falagas et al1 found that only 54% of consenting participants understood the study’s aim, 47% understood the voluntary nature of participation, 44% understood the right/ability to withdraw, 50% understood the associated risks, and 57% understood the benefits to participating. These results highlight the importance of an open discussion where it is the aim of the evaluator not only to inform, but to ensure understanding. You are not collecting informed consent if your participant is not informed!

Methods of Obtaining Consent

1. Consent forms (aka written consent)

The purpose of a written consent form is to provide all the information regarding participation to the participant in a clear, systematic way. The written consent, whether signed or not, can provide evidence that there was, at minimum, a presentation of information to the participant.

Reminder: Having a consent form does not necessarily mean you have informed consesnt, and informed consent does not necesarily require a consesnt form.

A systematic review by Nishimura et al2 found that an enhanced consent form or an extended discussion around consent were the most effective methods at improving participant understanding. An enhanced consent form aims to simplify consent to optimize understanding. It can include increased font size, enhanced literacy level, or additional materials including leaflets. Microsoft Word has features that can help you assess readability and literacy levels or look into an app like Hemingway.

If a paper consent form is used, it is important to draft the consent with the target audience in mind. Health Canada recommends a reading level of grades 6 – 8 for the general population3. Here’s a quick checklist for what to include in a consent form:

• purpose for the data collection 

• description of the complete nature of participation, which may include time    commitment 

• risks 

• benefits 

• assurance of the voluntary nature and the right to withdraw 

• use of information 

• confidentiality 

• contact person 

• any appropriate legal disclaimers, or reference to research grants or ethic approvals 

Reading Level Tips: To reduce reading level, try to make sentences shorter by removing unnecessary words or breaking a long sentence into 2-3 separate sentences. Choose words with as few syllables as possible.

Try to avoid jargon and acronyms and assess the literacy level (aim for Grades 6 – 8).

It is common for the project lead to keep a copy of the consent, and also provide a duplicate copy for the participant to keep.

A word on signatures: A signature on a consent form has its roots in legal consent. A signature may act as evidence not only that the person acknowledges the information in the consent form but that they have, in fact, provided consent. However, unless you have funding, legal, or organizational policies around signatures, they are not necessary to obtain informed consent. In fact, if you are collecting data from vulnerable populations, there can be risk in keeping a record of a participant’s name, especially if it can be linked to their data. An ethical principle of evaluation is to collect the bare minimum data required, so consider carefully if you absolutely need a signature. 

2. Verbal consent

Verbal consent can be an ethically sound method of obtaining consent. Scripting a consent process will ensure that each participant is given the same information about the project. Or, a consent form may be used as an aid or resource. As in a written consent, the language used in a verbal consent process should use lay language and keep the target audience in mind. A script or consent form can act as evidence that a consent process was documented and that each participant engaged in a conversation before giving or denying consent.

3. Implied consent

Implied consent is when a participant does not explicitly express consent but consents via action or behaviour. It can also be a valid option in appropriate circumstances. 

In some evaluation projects, the project leads do not interact with participants. For example, a lead would like a survey handed out by front desk staff at a service centre to each customer. In this case, the project lead is not available to obtain consent and placing the onus of obtaining consent on the front desk staff may be inappropriate. Once a project has been deemed ethically sound, it is possible that a participant can give informed consent by action. For this to occur, the participant must have access to the same information as one would receive in a written or verbal consent process. This information may come as a detached written document or the information may be included at the start of any paper data collection (e.g., a survey). 

Example Text: By completing this survey you are consenting to have your responses used in aggregate and shared back with program staff to be used in quality improvement.

There should be a statement that indicates that by moving forward with participation (e.g., by completing this survey), that the participant understands and accepts the nature of participation and provides consent. In this case, the completed participation, along with documentation of the consenting process, would act as evidence of consent. Because capacity to consent cannot be assessed in this way, ethical review of the project is recommended.

In all cases, the project leads should have a clear, documented process for how consent will be obtained, and how (if necessary) it will be reassessed throughout the project. If you are unsure if there are any policies around how you obtain consent, check in with your local health authorities, associations/colleges, or governmental acts that may have policies around consent. 

If a consent process is used, the participant should be informed of the intended use of the data, including an intent to publish data. The intent to publish, however, does not necessitate requiring consent. Publishing data while maintaining confidentiality is ethically acceptable. 

Stay tuned for Part 2 of this discussion that dives more into how to obtain consent and a list of principles to consider! 

In the meantime, check out Eval Academy tools for some consent form templates we have for interviews and focus groups.

Email Address
Sign Up

We respect your privacy.

Thank you!