As technology improves, more and more people are hosting virtual meetings and facilitating sessions remotely. Whether due to global pandemics, restricted travel budgets, or stakeholders flung across the world, evaluators should be prepared to facilitate virtual meetings or sessions.  

Most of us have probably been in remote meetings that have dragged on where we have checked out, checked our emails or surfed the web, barely paying attention. Facilitating remote sessions is a skill that takes practice to hone. We’ve taken the guesswork out of how to facilitate a great virtual session and have rounded up our best tips and tricks below. 

Before the meeting

Invest in good technology. Buy a headset and microphone with good audio quality as speakerphone can be echo-y and cause feedback. Before the meeting, it’s also a good idea to test your technology. Make sure the tools work with your operating system and that you know how to use the features. Can you mute all participants? Do you know how to enable a chat box? These are the kind of things you want to know in advance. Provide your audience with an audio dial-in option where possible.  

Send out an agenda and set expectations. Give participants an idea of what to expect and provide them with documents in advance. Sometimes having a brief half-page summary prepared in advance can help your participants come prepared and ready to engage. Let participants know if you expect them to use video. 

Log in early to allow space for people to join the meeting and to ensure it starts on time. Some meetings won’t allow participants to join until the host or moderator has joined. Logging in early gives time for people to get set up and connect.   

During the meeting

During your virtual meeting or facilitation, provide an agenda and refer to it throughout the meeting. This can help you stay on track and give your participants a frame of reference. 

Lay down the ground rules, or better yet, build them together! Do you want people to type questions in the chat box, or use a virtual hand raise to ask questions? Do you want participants to mute their microphones, or do you want to have a more informal meeting where microphones are unmuted to build a sense of community (note: this only works with a small group). Build explicit group norms to ensure conversation flows smoothly. 

Begin your session with an activity that promotes connection (a.k.a. an icebreaker). Ask a question and have the group type answers in the chat box or ask a question and have all participants answer out loud in turn. 

Enable the chat box and encourage people to use it! If participants are only typing in the chat box, you can use their responses to engage them and encourage them to speak out. 

Leave a little extra time for people to answer questions and contribute. Normally when facilitating discussion, you should pause for 8 seconds before assuming everyone is done contributing. When hosting virtual sessions, leave extra time (about 12 seconds) for participants to unmute themselves and ensure no one else is talking at the same time. It feels very awkward to wait so long, but count the seconds in your head. 

Try not to talk for more than 5 minutes at a time. Break up information dissemination with tasks or structured requests for feedback. Give your audience a problem to solve. If you are using a meeting to simply share information without requiring feedback, send an email instead! 

Create a shared responsibility. So often in virtual spaces, participants can become passive observers. Encourage involvement by giving participants a short time frame to complete a very specific task. Have participants share their answers out loud or in the chat box. 

After the meeting

Technically this advice is for the end of your meeting and it holds for in-person meetings as well, but make sure to set the date for your next meeting, or set expectations for participants to respond to a poll about the next meeting before the meeting ends. Hopefully people haven’t been paying attention to their email while in the meeting, so your request to set the next meeting time may get lost in their inbox. Be proactive and get consensus while you have everyone on the line. 

Send a short meeting summary, including any links that were mentioned. Summarize key questions or take-aways as participants may have been focused on entering items into chat boxes or keeping up with the flow of conversation instead of taking notes (ok, they might have also zoned out). 

Ask for feedback. Chances are you aren’t an expert at hosting virtual sessions (and that’s ok!). Ask your participants how it went; were they engaged, did the technology work, was there anything you could do differently? 

---

Tools 

• Google Docs are great for getting people to brainstorm together. You can also use them to have the group break out into smaller groups, with each group writing on their own Google Doc.  

• Consider exploring augmented and virtual reality options (check out our article on AR and VR for ideas).

• Use the polling function in your meeting technology to get real time feedback and earn your participant’s attention 

• Get creative! Ask people to draw something and post a picture of it or make a Web Quest

• Use visuals to break up conversation and get people engaged. 

• Use online whiteboard tools to inspire creative thinking 

---

Tips:  

Keep your meeting brief. If it’s going to be over an hour, consider a small break mid-way through to let participants get up and refocus. 

Don’t panic if something goes wrong. Sometimes, technology glitches and it’s out of our control. Be calm and if there was another speaker planned, pass the baton to them while you try to troubleshoot. If you can’t solve the problem within 5 minutes, consider postponing the meeting. 

---

Sign up for our newsletter

We’ll let you know about our new content, and curate the best new evaluation resources from around the web!

Email Address
Sign Up

We respect your privacy.

Thank you!

---

 

 

---

New and Noteworthy

---

The new and noteworthy reads for March 2020 are focused on the new and noteworthy virus – COVID-19. “Noteworthy” is an understatement that does not describe the gravity this virus has and will continue to have on our lives for years to come. As such, here are the latest evaluation resources focused on COVID-19 that we hope will help guide your evaluation practice.  

Evaluation’s Yoda, Michael Quinn Patton, shows us the way in a pandemic

Leave it to MQP (@MQuinnP) to be one of the first to show us how this virus is/will impact our evaluation practice. Check out his blog post, Evaluation Implications of the Coronavirus Global Health Pandemic Emergency where he lays out, in a simple and engaging way, 15 implications we need to keep in mind. My favourite is to “Be Proactive”. It seems the tendency right now is to sit, wait and give our clients the time and space they need to deal with this crisis. While I agree evaluation is not top of mind for people right now, MQP hits the nail on the head when he says: 

“Don’t wait for them to contact you. Evaluation is the last thing on the minds of people who aren’t evaluators. They won’t be thinking about how the crisis affects evaluations. That’s your job as an evaluator. Get to work doing that job. Adjustments need to be made now, sooner rather than later. Offer help in updating your evaluation. This doesn’t necessarily mean delaying data collection. It may mean accelerating it to get up-to-date information about the effects of the crisis.”

Of course, he also talks a lot about principles relating to his Developmental and Blue Marble evaluation work and how it applies in this situation. In fact, Blue Marble Evaluation held a webinar titled, Blue Marble Evaluation: Evaluation Criteria for Transformation, that discusses evaluating transformation and six criteria for evaluating it. If that is a bit too much for your brain to handle right now then he has also put a video on YouTube for “evaluators’ pandemic comfort music” for you to sing along to. 

UNDP Independent Evaluation Office: Six tips for evaluation during crisis

The IEO of UNDP (@UNDP_Evaluation) recently produced and shared a one-page infographic that highlights six tips on evaluating programmes during crises. Like MQP, their first tip is to “rethink evaluation plans” with a focus on getting data to stakeholders as soon as possible. Of course, many of the tips talk about working remotely, be it working as a team or collecting data remotely. They recommend Skype interviews, mobile questionnaires, online surveys, collaboration platforms (Slack and Yammer) and satellite imagery to gather data.

Doing fieldwork in a pandemic

If you really want to dig into collecting date remotely, you need to check out Deborah Lupton’s crowd-sourced Google Doc, Doing Fieldwork in a Pandemic. The document was originally created to turn traditional fieldwork (i.e. face-to-face) methods into a more ‘hands-off’ mode; since, many people have added numerous resources.

Deborah Luton also wrote a blog recently titled Social Research for a COVID and Post-COVID World: An Initial Agenda. In this blog post she talks about her experience doing social research on the HIV/AIDS pandemic and the obesity epidemic; she shares insights on similarities and differences with the current COVID crisis. From there she proposes a list of key social research questions that will generate a better understanding of the current and future impacts of COVID. 

Evaluation in a Crisis 

Chris Lysy of Fresh Spectrum (@clysy) is an evaluator and evaluation cartoonist and blogger. Recently he wrote about The Evaluation Mindset: Evaluation in a Crisis. Like MQP, Chris talks about the important role evaluators play ensuring decision makers have access to timely information. As he says, “evaluation is an anytime activity”. 

Checking in with evaluators

Carolyn Camman (@c_camman) and Brian Hoessler’s (@StrongRoots_SK) produced a podcast (Episode 33: Eval Café Check-in) where they asked sixteen evaluators from around the world (including our own Shelby Corley)  how the COVID-19 pandemic is affecting them, what they want people to know, and what is giving them hope during this time. The results – it turns out evaluators are human and not just methodological robots! One of the big themes is how this crisis has brought people together (not physically of course) and ignited our need for connection. The guest evaluators talked about how people are reaching out more now than before when social distancing wasn’t the norm. Of course, like all good evaluators they talked about learning and what learning would come from this crisis. They wondered if the increased emphasis on human connection would continue and hoped that it has given us pause to re-assess our values and priorities going forward. Some also talked about the implications for how we work as evaluators and questioned if the new remote ways of working will push us to continue down that path in the future.

Tips for virtual gatherings

Speaking of remote ways of working, evaluators are adapting and incorporating virtual gatherings into how we do our work. For ideas on how best to do that, refer to Elizabeth Diluzio and Laura Zatlin’s AEA blog post Tips + Resources for Virtual Gatherings During This Global Pandemic.

The Spinoff cartoonist is a data wiz with data viz

There have been lots of charts flying around the Internet trying to tell the COVID pandemic story. My favourite data viz so far was posted on a New Zealand online magazine called The Spinoff; it was illustrated by their cartoonist Toby Morris. It is an animated graph that shows how individual discipline really can stop transmission. It is so good I sent it to my baby boomer parents who seem to think staying home does not apply to them.

---

Sign up to our newsletter

We’ll let you know about our new content, and curate the best new evaluation resources from around the web!

Email Address
Sign Up

We respect your privacy.

Thank you!

---

 

 

Up until one month ago my evaluation experience was firmly rooted in Canada. From my academic endeavours to my career as an evaluator, I carried a perspective very specific to Canada and the developed world.

Like many young (ish) evaluators, I wanted to get international experience – more specifically I wanted to take on the challenge of doing evaluation work in a developing country. I’m a Credentialed Evaluator, how hard could it be? Would it really be that different? Would I apply the same approach and methods? Would evaluation be welcomed with open arms? These were the types of questions I optimistically asked myself as I packed what I thought would be practical clothes and savvy travel gadgets before my long flight to Siem Reap, Cambodia. I would soon find out that I was incorrect in my optimism to copy and paste my evaluation perspective as well as my packing list.

One month later, as I cross my fingers and hope that the power stays on long enough for me to send some emails, I am finally able to reflect on my experience so far. In no particular order, I am sharing six of my learnings to date. Although some may come across as cynical or potentially naïve, I share them as I am confident they will make me a better evaluator in my current role as well as when I am back home in Canada.

---

Lesson #1

Context is Everything

Cambodia and the Khmer people captured my heart as a traveller, so naturally I jumped at the opportunity to volunteer at a healthcare NGO based in Siem Reap. Learning about a country and its history has always been of interest to me, but now I understand that it is also imperative to be able to do any meaningful evaluation work.

Cambodia is a country with a very dark, recent history. One doesn’t have to go too far back in time to hear the horrific stories of the Khmer Rouge, genocide, and war. And there are constant reminders of the obstacles that this country continues to face – from landmines to the loss of a generation. For example, the country was left with only 32 physicians for a population of approximately 6.6 million after the genocide (Santini, 2002) essentially forcing them to restart their medical education system.

The impact of these events is evident in the way people communicate, build relationships and set priorities. For example, health care appointments are often done with an open door and multiple family members in attendance – a sign of the slow process to rebuilding trust with both individuals and the organizations they represent. Likewise, when conducting interviews in villages the majority of time is spent being welcomed into a home or establishment and offered food, then once a relationship is established the actual gathering of information can begin. Again, I can’t emphasize enough that as an evaluator it is vital to understand the context. I started to realize this on day one and feel that it may be the biggest “take home” from this entire experience.

As evaluators we are trained to ask questions – we need to do this before we even get close to starting our work. Don’t be afraid to ask questions and continue to learn. In my case I was rewarded with a country that is happy to share their stories and perspectives which has given me the foundation to now ask better evaluation questions.

---

Lesson #2

Feasible, Not Perfect

Capacity, does it even exist? This question is not unique to the developing world, I think it is inherent to most organizations – but I am reminded of it as I navigate my way through my current project. For example, it is great to recommend a new data collection tool, but is it actually possible to implement? Or worse, if implemented will it take up more time than it should for frontline staff?

Coming from a Consultant position prior to this project, I aim to submit watertight deliverables. I still have this perfectionist mentality, but it isn’t always in the best interest of low resource projects. I am learning that in a low resource setting a finished product is often better than a perfect product.

Lastly, it is important to create or implement changes that are not only feasible but sustainable. NGO’s, particularly in developing countries, have quite a bit of turnover due to short term expat contracts. A question I am trying to ask myself regularly is “Will the documents or processes I implement be easily understood by future staff and volunteers?”

---

Lesson #3

Champion Evaluation

In my experience so far I rarely hear about evaluation without it being tied to monitoring. Monitoring & Evaluation (M&E) is often thrown into funding agreements or annual reports but I would argue that it isn’t really evaluation, but rather the presentation of inputs and outputs.

There is a desire to present the impact of the work NGOs are doing, but in reality they rarely get past transactional or process data. Again, this is an issue in both developed and developing countries but one I experience daily in my current role.

Rather than correcting definitions (output versus outcome, anyone?), I have decided to adopt an educational approach. I think that when evaluators find themselves in a setting where evaluation literacy is low, it is our job to explain the difference between research, monitoring and evaluation. We need to champion the use of our profession to help developing countries communicate what they are doing (and at my NGO it really is impressive!) and then hopefully this will help them land more funding and do more good.

---

Lesson #4

Be Patient (but seriously)

I already mentioned the inconsistent electricity – one prime example of something I took for granted. I was not the best version of myself the first few times I sat in the dark wondering if it would be hours or days until I could charge my laptop. Fortunately, I work with people that learn to use that time to grab a coffee, stretch and catch up with other staff. They are rock stars when it comes to having patience and I really hope it will rub off on me.

Patience isn’t just required to live in a developing country, it is needed for every project and every task. I have learned that often it is two steps back before ever moving forward. NGOs in the developing world have big goals and are eager to make a difference (a lot like NGOs back home). However, it doesn’t always line up with what processes are in place. For example, I was asked to help my NGO better understand their donors. They have a donor database so I figured it would be a ‘quick win’. Wrong – data entry processes appeared to have changed every year and excel spreadsheets became preferable to the actual database. So it was two steps back, which in this case was creating a process map to understand data entry processes.

---

Lesson #5

Unexpected Influence

I’d like to think that as an Evaluator I am pretty aware of bias. However, I have been introduced to potential biases that I have never considered in the past. For example, Westerners or ‘Barang’ as Cambodian’s say, introduce a lot of bias when engaging in research or M&E. We can be mistaken as a potential donor or funder – this might put people on edge or cause them to only show the good stuff (which isn’t great when you are trying to improve a program). 

Unsurprisingly, we also come with our own set of ideas and beliefs. Again, not a revelation but I am more aware of this than I have ever been in the past. I have to actively try and ignore the KPIs and evaluation questions I became accustomed to at home as chances are they are not nearly as meaningful for this population.  For example, most health care measurement in developed countries revolves around topics like quality, efficiency, and safety. I initially took this same approach but quickly learned that access and availability of expertise were more important to understand before diving in to the more traditional KPIs. Moreover, the larger concern is getting health services to rural communities and training local staff to be able to sustain access.

I am fortunate in that I am being exposed to methods that help minimize our bias. For example, Participatory Action Research or PAR is actively used at my NGO (Baum et al., 2006). It may not be a new method, but it is definitely one that I am eager to learn more about.

---

Lesson #6

Take It All In!

I saved my favourite lesson for last. If you are lucky enough to work with an NGO in a developing country – take it all in (seriously). Do as the locals do, try the food, get lost in markets, visit local art galleries, and try to pick up a few phrases (even if it is embarrassing like my Khmer). It can be overwhelming, such as dodging scooters in 38°C whenever I need to get food, but it is worth it. I have never been so out of my comfort zone, but I’m learning to love it.

I would argue that engaging in the culture will not only give you some great party stories but it will make you a better evaluator.

Written by: Samantha Larose

---

Sources

Baum, F., MacDougall, C., & Smith, D. (2006). Participatory action research. Journal of epidemiology and community health, 60(10), 854–857. https://doi.org/10.1136/jech.2004.028662

Santini, H. (2002). Rebirth of the health-care system in Cambodia. The lancet, 360, s57-s58. Retrieved from: https://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736(02)11824-1.pdf

---

Sign up for our newsletter

We’ll let you know about our new content, and curate the best new evaluation resources from around the web!

Email Address
Sign Up

We respect your privacy.

Thank you!

---