A blank page can intimidate any writer. The same goes for evaluators – that “evaluation questions” column in your evaluation plan may be daunting. Looking at examples of real-world evaluation questions just might inspire your own.  

 We’ve already covered how to write good evaluation questions. As a quick reminder, your evaluation questions should be developed collaboratively and be open-ended. You can look to strategic plans or relevant quality frameworks for thematic guidance. Most importantly, be sure to edit until you’re sure the language is just right. 

 The examples below are grouped by content area, but with slight changes, can apply to many different evaluation projects. If you’re new to evaluation, we hope these examples can help you write questions that apply to the initiative you’re evaluating. If you have some evaluation experience already, watch for an upcoming post sharing evaluation question examples at different stages of the evaluation and learning cycle. 

Youth Programming

• What makes this program unique? 

• Who are the program participants? 

• To what extent are children, youth and caregivers participating in programming? 

• To what extent is programming making a difference in the lives of children, youth and families? 

• Which groups benefitted most from the program? 

• To what extent do program staff hold required safety certifications? 

• In what ways did program operations differ from the original plan? 

Healthcare Initiatives

• In what ways did program planners consider equitable access to care? 

• How has the program impacted pathways to and from primary care? 

• To what extent is the organization demonstrating leadership in community health planning? 

• To what extent do patients experience an easy referral process? 

• How do quality of life outcomes compare to norms in the region? 

• To what extent does the program affect mobility? 

• How efficient is the patient flow through arrival to admission? 

• How quickly can patients access the service? 

• To what extent are patients receiving appropriate preventative screening? 

• How effective are interdisciplinary teams? 

• In what ways has the project affected patient safety? 

• What contribution has the program made to meeting the targets for the Sustainable Development Goal “ensure healthy lives and wellbeing at all ages?” 

Environmental Initiatives

• How has the campaign impacted local awareness of the incentive program? 

• What impact has this process had on emissions in the northeast region? 

• How well aligned are the program activities with the regional strategy? 

• In what ways have residents changed their use of plastics? 

• To what extent has recycling capacity increased? 

• How has the program expanded access to affordable sustainable energy? 

• How well do the program’s messages resonate with local understanding of forest use? 

• For how long do sponsored companies maintain these new practices? 

• Which of the three program options was the most cost-effective? 

• To what extent did the initiative create unintended outcomes? 
  Collecting Data

Education & Training Programs

• How well prepared were students upon program entry? 

• To what extent did the course deliver content students expected? 

• How successful are program graduates in achieving certification? 

• How have trainees applied the lessons from the course? 

• How accurate do students feel their assessments were? 

• What factors most drove learner success? 

• To what extent do trainees report changes in confidence? 

• What changes in organizational performance can be attributed to the training program? 

Collective Impact

• To what extent are partners collaborating? 

• How engaged are local leaders in contributing to the network? 

• How well do chosen outcomes resonate with each partner? 

• How confident are partners that they can achieve the intended outcomes? 

• To what extent has the collective impact been measured? 

• How have the initiatives affected quality of life? 

• How long do agencies’ collaborative relationships last? 

We hope these evaluation question examples have inspired some of your own. 

 Wondering how to document your evaluation questions? Download our free Evaluation Plan Template. 

To learn more about applying evaluation in practice, check out more of our articles, or connect with us over on Twitter (@EvalAcadmey) or LinkedIn.

Email Address
Sign Up

We respect your privacy.

Thank you!

Understanding what you need to know. Designing your approach to finding the answers. Easy enough in theory, but real-world examples can show you have to apply theory to practice. That’s why we’re sharing this series on how we evaluated. 

 In this post, I describe how Three Hive Consulting supported the evaluation of a home health monitoring project for patients with chronic conditions. 

The Project: Home Health Monitoring in Primary Care

This project rolled out in the summer of 2020, in the throes of the Covid-19 pandemic. At that time, there was enough experience of the pandemic to know that limiting in-person interactions was the key to reducing the spread. The world was working to find ways to deliver services remotely, and healthcare was no exception. 

 Patients with chronic conditions often require more frequent medical care and are at greater risk of developing severe complications or needing hospitalization with Covid-19 infection. Monitoring symptoms and delivering care virtually was an obvious need.  

 In this project, patients with chronic conditions were provided with free monitoring kits for 90 days. The kits included blood pressure cuffs, pulse oximeters, thermometers, scales, and tablets. Patients input their information into tablets loaded with a platform enabling the reporting of results to primary care nurses. Primary care nurses then checked the platform regularly for alerts for clinically significant results. They called the patients directly or consulted with family physicians when appropriate. Many concerns could be addressed by nurses over the phone, while others required the patient to attend either a virtual or in-person visit with their family physician.  

Intended Outcomes

This home health monitoring project aimed to: 

• Improve patients’ access to care 

• Decrease patients’ risk of exposure to Covid-19 

• Detect worsening symptoms earlier 

• Optimize physicians’ time 

• Improve patients’ ability to self-manage 

• Reduce hospital admissions and emergency department visit 

• Maintain or improve patients’ health-related quality of life 

Developing the Evaluation Plan

This project involved many partners. Primary care in this region (central Alberta, Canada) is delivered by family physicians and supported by Primary Care Networks (PCNs). Three Central Zone PCNs received funding and in-kind resources from several partners, including Health City, Alberta Innovates, and Boehringer-Ingelheim. Each of these partners had input into the evaluation plan. 

 With limited resources available to evaluate the first phase of this potentially multi-year project, partners agreed to four focus areas: 

1. Project reach (using document review and administrative data) 

• Did the project achieve its aim of enrolling at least 30 patients? 

• Who are the participating patients? 

• What challenges were encountered in patient recruitment? 

• What was helpful or successful in patient recruitment? 

• Was PCN and provider participation maintained through Phase 1? 

2. Provider experience (using surveys at two points in time)

• What challenges did providers encounter in working with the HHM model? 

• What worked well for providers? 

3. Patient experience (using surveys at program discharge) 

• What challenges did patients encounter in working with the HHM model? 

• What worked well for patients? 

• What suggestions did patients have for improvement? 

4. Patient health outcomes (using the EQ-5D-5L at intake and discharge, patient surveys at discharge, and clinical data) 

• How, if at all, did patient-reported quality of life change over the duration of the project? 

• To what extent were clinically significant results identified through the HHM platform? 

• How quickly were patient results reviewed? 

• To what extent did patients utilize other health services during their participation in the project? 

We designed the evaluation plan to minimize the workload on busy providers and take advantage of information already documented for clinical care; asking nurses or physicians to track additional data elements was not feasible, particularly during the pandemic. Patient health outcome data was limited to what patients could reliably report themselves and the clinical information captured in the home health monitoring platform.  

The evaluation plan included mid-term reporting of provider experiences and final reporting of all available data. The evaluation findings would need to provide information to inform decisions about spread and scale, as well as point to a more robust evaluation approach supporting the expansion of this model of care after the initial phase.  

“It’s safe to say this is a new way of doing business,” says Central Zone PCN Committee Operations Lead Jodi Thesenvitz. “We need to see if this is going to be a legitimate model of doing work going forward.”  

Collecting Data

In a year of shifting priorities, collecting data went fairly well but did not yield the sample sizes we intended. All data collection was undertaken online to ensure safety and convenience, but we struggled with uptake.  

 Invitations and links to complete the EQ-5D-5L and patient experience survey were embedded within the home health monitoring platform, and so were seen by all participating patients. Of the 37 patients enrolled, only 11 completed the EQ-5D-5L at both intake and discharge, leaving us with results that were not entirely reliable. Similarly, only 17 completed the patient experience survey at discharge. 

Our Learnings

Low response rates are always a cue for evaluators to re-examine their recruitment processes. Upon reflection, the language we used in the home health monitoring platform to invite patients to complete both survey tools was not as compelling as it could be. In our subsequent second phase, we’ve adjusted the invite to provide more information about the importance of patient feedback and used plainer language to describe the EQ-5D-5L—it’s a very simple tool with a rather technical name. 

We’ve also increased engagement with providers, letting them know early about the role of evaluation in the project and their role in supporting that evaluation.  

The evaluation findings have been already been helpful to the project partners. “It helped validate the expansion of the initiative,” according to Health City CEO Reg Joseph. “One of our key goals is to scale the initiatives that make sense to scale, in that they drive health adoption of innovation, improve health outcomes and drive economic growth in the health sector. To do this often requires policy change that itself requires evaluation and data.” 

The findings have been helpful for PCNs and physicians considering joining the second phase of the project. The evaluation report was “used as a communication tool,” said Thesenvitz. “It was a source of evidence for those considering participation in the next wave. It legitimized and explained the effort.” 

Future Evaluation

The second phase of the project allows us to expand our evaluation approach. With a larger group of patients and access to system-level health data such as emergency department visits, hospital admissions, and physician visits, we will be able to build the body of evidence needed to make decisions not just at the operational level, but at policy and funding levels, too.  

We hope this example has helped illustrate what evaluation of a virtual health initiative, or even just program evaluation in general, looks like in practice. For another tool to apply evaluation theory to practice, see our free resource on applying the JCSEE Program Evaluation Standards in practice. 

To learn more about applying evaluation in practice, check out more of our articles, or connect with us over on Twitter (@EvalAcadmey) or LinkedIn.

Email Address
Sign Up

We respect your privacy.

Thank you!