cplysy

Apr 01 2020

complexity and equity

I promised Jara I wouldn’t overthink this one and just get this off Twitter and into a blog post, and I’m really trying hard to live up to that. (I might have overthought “not overthinking it” though.) I also know that with how fast things are moving now, there’s a balance to be struck between deliberateness and irrelevance. The post I’m writing now isn’t the one I would have written last week, nor is it the one I’d write a month from now. It’s the one I’m writing in this moment, sitting on my couch in my apartment where I’ve been mostly alone for the two and a half weeks, since the global pandemic came crashing down on these shores.

It’s an interesting time to be an evaluator. There’s a real and present (and needed?) challenge to the relevance of our work. I was speaking to a friend the other day who is working with teams to develop logic models for their programs. I asked her if they were including COVID-19 in their logic models and she said yes, under the “assumptions” section (i.e., “things we assume to be true about our context that would affect our program). I asked her if they were including potential global disasters like COVID-19 but whose manifestations are as yet unknown. She said no, they were just trying to wrap their heads around what was already happening.

Honestly, can’t blame them. I’m trying to do that too. Program logic models are about diagramming knowns, not unknowns. They are representations of what we believe is happening and why. But they hinge on the assumption that the relevant causality of our contexts is predictable, describable, and repeatable, and that’s just not always true. And, worse than untrue, it’s often not useful. Evaluating in complexity means evaluating in uncertainty. In a space where we fundamentally can’t be sure exactly what is happening, why, or what will happen next. Where knowing what has already happened does not give us predictive confidence over the future. This is the nature of complexity—irreducible uncertainty. Assumptions of certainty don’t apply.

Uncertainty can give rise to anxiety and fear, but also surprise and delight. We wouldn’t gather with friends if we knew exactly what was going to happen each time, what conversations would emerge, what unexpected jokes would make us laugh, what poignant turns there might be. We gather with friends to be immersed in the complexity of human dynamics, confident in our collective ability to navigate whatever happens in service of greater joy and connection. We look forward to what is unknown.

Certainty and uncertainty also came in our recent Eval Cafe episode with Nora Murphy Johnson and Andy Johnson of Inspire to Change, and Chris Corrigan of Harvest Moon Consultants. We talked about how evaluators are often hired and expected to bring certainty, but in actuality when we are at our best, we bring more uncertainty. We bring it with the question that we ask, with the invitation to look more deeply into what is being done, why, and to what effect. Our role then is not to eliminate the uncertainty, but to accompany people well as they move into and through it.

Why is it so hard though? That’s a question I ask a lot. I don’t believe it’s just that uncertainty and complexity is inherently a more difficult place to work in. It has challenges, but it also has a rhythm and principles. “Human beings are built for complexity” (a phrase I can attribute to Chris). We navigate complexity all the time—our lives are improvisations from the moment we wake up to the moment we fall asleep each day. We raise children. We form societies. We host gatherings. We learn and create languages. We make art. We are literally adapted for these things.

So why is it so hard to work this way? Why is it that folks who teach developmental evaluation (which is just a form of evaluating according to the principles of complexity) have to warn us (and truthfully) that we will need to constantly be helping our clients and stakeholders “stay the course”, not panic and retreat back into the familiarity of formative and summative evaluation? Why is it so hard for many of us (myself included) to even understand what developmental evaluation is and do it, consistently and coherently?

There isn’t a single answer to that question (beyond “it depends”, the ultimate single answer to any question), but there’s one part of the answer that has been on my mind lately, and it speaks to the idea that “injustice is rooted in uncertainty”, a hypothesis that Chris offered on the podcast.

Here’s what I tweeted about it (slightly edited for clarity and links added for reference):

Here is my take: the conversation about complexity is inextricable from the one about justice and equity.

The point on which my whole practice has turned lately is the understanding that the aversion to complexity is not just because complexity is challenging, but because settler colonialism and white supremacist culture reject complexity. Complexity isn’t just “hard” inherently. It’s specifically hard for particular ways of being and acting. It’s hard to consolidate power in complexity, because it requires diversity and collaboration, acceptance of partial and multiple truths, and openness to ongoing change.

Applications of complexity thinking aren’t inherently just. A complexity-informed approach can be used to colonize better, exploit better, oppress better. As @jdeancoffey pointed out, complexity work must intentionally serve equity, not just be assumed to, “And I would add @MQuinnP that we need to do DE in pursuit of equity, liberation and justice. We have to be for something.”

And still I see a resonance in the obstruction and devaluation of the ways of knowing that serve in complexity and the epistemic injustice and epistemicide levied at Indigenous ways of knowing, which also tend to be relational and contextual. Insistence on certainty at a level inappropriate to context is both irrational and, when combined with institutionalized inequity capable of enacting and sustaining harm through violence or neglect on specific groups of people, unjust.

In more practical terms what this means for me as an evaluator is that I have a responsibility to be competent both in certainty and uncertainty, discerning when each is appropriate, and supporting the people I work with in navigating both. Creating certainty is not my job. AND that ongoing questions at all times in my work are, “Whose way of knowing is being centered here? Why? Is it named or presented as the default? In what contexts does that way of knowing operate?” These are @equitableeval questions, and I need them to evaluate in complexity.

And from Jara in response:

I would add not only to evaluate in complexity but to understand complexity and the ways in which it may or may not be grounded in or be in service of equity, liberation and justice – probably need to add healing.

I also followed up with a response to myself that while it might be difficult to consolidate power in complexity, it’s not impossible to do so. Lots of bad things also operate in complexity. Racism, colonialism, exploitation capitalism, these are all things that have thrived and continued to thrive within the parameters of the complexity of human social organization. Complexity can be navigated to many different ends, hence the need for an explicit ethic or axiology of justice to inform where we are headed and why and how. As Jara said, we have to be for something.

Human beings aren’t innately good or bad, we’re just human. We’re self-determining and we exist in contexts that influence us and are influenced by us. If it were just a case of deciding we want to work in complexity and skilling up in how to do so, it wouldn’t be so hard. There’s a paradigm shift involved, but it’s doable. There are complementary new practices and techniques to learn, but they’re available. Any muscle takes time and effort to be strengthened, every craft can be practiced and honed. But it’s harder than it needs to be because we’re also reinforced in so many systemic ways not to work in complexity, every time there is a demand for certainty that’s disconnected with the actual context being operated in.

There are RFPs and funder expectations that equate to “buying success” because we only want to fund what is ‘guaranteed’ to work with a ‘proven track record’ (the language of certainty). The under-resourcing and over-burdening of the social sector where the demands for responsibility and accountability are not remotely matched with the necessary support, leading to fear and aversion to risk and being punished for not delivering exactly what is asked for. The ways that the epistemologies of certainty and their accompanying methods (e.g., randomized control trials) as accepted and positioned as more credible, valid, and valuable than epistemologies and methods of complexity and uncertainty (even in casual language like, “well ideally we’d have harder data on this, but for now this is what we’re seeing on the ground”—anyone obsessing over exponential charts lately knows that the most useful data is the data you have to work with and that numbers don’t bring certainty, just a different kind of perspective best complemented with other kinds of data, like stories about what other people and countries are doing to cope and manage in highly emergent circumstances). There’s a whole fantastic article from Tanya Beer about all the institutionalized impediments to working in complexity within evaluation itself and within funding organizations. On a grander scale, there’s straight-up capitalism (economic stability, remember?). And white supremacy culture. These are all barriers to just and equitable evaluation as much as to complex evaluation.

The argument here isn’t that complexity and complexity-informed approaches are superior or universally appropriate. I don’t want a complexity-informed approach to making a vaccine. I want the vaccine figured out according to the known good practices we have in that space because that’s a context where certainty, predictability, and established practices apply. And I want to know that we are prepared to have adaptive, responsive, uncertainty-appropriate practices for how we’re going to keep ourselves together as a species over the coming months, which is a very different situation to grapple with. And I want that adaptivity and responsivity to be consciously, explicitly, and fundamentally in service of justice and equity.

And for my fellow white and settler evaluators (and anyone else who wants to), for us to keep asking, “What and whose ways of knowing are being reflected in this evaluation work? Why? Whose values and worldviews are determining what ‘counts’ as valid knowledge? And appropriate ways of generating and using that knowledge?”

And for those of us operating in complexity, to be asking those questions as well, and also what we are navigating complexity in service of, and how does that show up in our work?

And for us to look to the work that has been and is being done particularly by people of colour and Indigenous evaluators working in community, in relationship, and in complexity (and who are persistently erased in the evaluation community, particularly when they are women, as has been thoroughly researched and described by Vidhya Shanker and referenced in this recent AEA365 post), not to invent or reinvent evaluating in complexity but to see how it is already being done, listen, and learn.

Mar 31 2020

Being a Volunteer Evaluator

Have you ever been curious about what it is like to volunteer as an evaluator? Or are you considering a volunteer opportunity in the near future? This article aims to give you some insight into finding and engaging in a volunteer opportunity. Many organizations recognize the need for evaluation of their programs or services, but not all of them have the capacity to hire an evaluator for their team or to bring in an evaluation consultant. For these organizations, volunteers or pro bono evaluators may be the solution.

On the other side of the coin, an evaluator (or any professional!) may be eager to offer their skills pro bono to support an organization. The reason for seeking out a volunteer opportunity is likely to vary – an evaluator may want to help an organization that is important to them, experience a new industry, new country or culture, or to simply offer up some of their spare time to volunteering. Ideally, the organizations in need of evaluation expertise will be connected with the volunteer evaluator resulting in a mutually beneficial experience.

I am writing this article as a volunteer evaluator currently offering pro bono services to an NGO abroad in need of monitoring and evaluation support. My aim is to share some information on volunteering in general, pro bono evaluation, my volunteer experience, some key differences, and tips for success should you decide to pursue your own volunteer opportunity. If you want to learn more about my volunteer experience you can read my article about evaluating in the developing world.

Volunteerism

Skilled professionals, such as evaluators, have a lot to offer through pro bono services. Studies have found that taking time from work to volunteer is linked to overall well-being of the volunteer (Thoits & Hewitt 2001) and enhanced skills such as cross-cultural competency (Caligiuri et al. 2019).

Through discussions with skilled volunteers over the past few months, I found that a common theme or rationale for volunteering was to apply skills or expertise in a new setting or context. Anecdotally, the experience of volunteering allowed for engagement with new populations and places; which was viewed as professional development in a less traditional sense.

My experience was supposed to be a three-month engagement volunteering with an NGO in a developing country, which I am now completing remotely due to the COVID-19 pandemic. I too was eager to experience a completely new context and challenge myself to apply my evaluation skills in a completely different space.

Volunteer Opportunities

Some volunteer opportunities require a short-term commitment (under a month), while others require a longer-term engagement and/or full-time hours (over a month to a year or more). I was compelled to take on a longer-term commitment after reading about sustainable and ethical volunteerism. Moreover, in some cases short term volunteer tourism in developing nations can do more harm than good (Guiney & Mostafanezhad 2014, Guttentag 2009, Raymond & Hall 2008), so make sure you fully research the role and organization before you commit to any volunteer position.

Once I nailed down an ethical volunteer organization, I began to explore how I could tackle this opportunity financially and without turning my regular life into total chaos (not easy, but worth it!). For many people (myself included) the largest barriers to volunteering are the logistics of moving abroad and the financial cost associated with it – however there are ways to mitigate these obstacles.

Create a budget: stick to your budget and check-in as frequently as possible.
Cost of living: volunteering in a developing country, although more challenging, is much more affordable. For example, in Cambodia (where I am), rent for a high quality studio apartment goes for around $250 USD, and can be as low as $100 USD per month if you’re willing to sacrifice some creature comforts. You can also find delicious meals for as low as $2 USD.
Look at ways to press “pause” back home: utilize sabbatical opportunities, rent out your home, bank vacation days, or continue to work remotely.
Consider volunteering remotely: working virtually is becoming more common, as is volunteering. This is a great solution for evaluators who want to volunteer but are not in the position to leave home.

I recognize that leaving home for an extended amount of time is not feasible for many, but I do believe that long-term engagements are important; it takes a couple of weeks just to understand an organization, whether it is local or international. A good place to start if you want to contribute your skills locally is through resources such as Volunteer Canada (if you’re Canadian) or similar organizations set up in your country. These organizations house lists of credible volunteer organizations nation-wide. If you’re like me and interested in getting some international volunteer experience, consider organizations like Professionals Doing Good that focus on connecting experienced professionals with projects that match their particular skillset. This organization offers both in-person as well as remote opportunities for professionals unable to relocate.

The one point I feel is worth emphasizing is that skilled volunteerism is needed. Even if you can only offer a couple hours a week, many organizations can still put your skillset to work.

Evaluating as a Volunteer

Continuing with the discussion of time commitments for volunteerism, this is even more important for evaluators. If an organization requires a full evaluation of a program and you can only offer a couple weeks of time, this may not be the best fit. However, using individual skills or methods for smaller tasks/deliverables can still be meaningful in short time frames.

For example, evaluators can offer to help develop a survey, hold focus groups, develop a logic model or perhaps an evaluation plan that can be put into play when needed or if more resources become available. You do not need to conduct an entire evaluation to make an impact – just offering one or two of your evaluator skills can be a game changer for organizations.

It may sound cliché, but being a volunteer evaluator is very rewarding. We, as evaluators, have a skillset that many organizations need (whether they know it or not). In my experience, within the first couple of weeks, evaluators can bring a different lens to a program that allows for quick wins and buy-in from leadership to prioritize or at least support future evaluation work.

Key Differences

If you can’t tell by now, I am in support (and encourage) evaluators to participate in skilled volunteer opportunities. Of course, the nature of being a volunteer rather than an employee or consultant results in a very different experience. I thought I should share three main differences I experienced as I put on my volunteer hat and set aside my consultant hat.

Different expectations: Many NGOs are often expected to undertake monitoring and evaluation by their funders (or board) but do not always have a grasp of what it actually entails. As a result, expectations might not be well-defined for volunteer evaluators. I see this as both negative and positive – negative in that the scope may not be clear, but positive in that we are given the chance to really show what evaluators can do.
Getting attention: Many volunteers, depending on the role, work quietly in the background. I think that for evaluation activities to be successful we need to engage with as many relevant stakeholders as possible. It might not be normal (or appropriate) for an NGO executive to receive an email from a volunteer asking for their time. Fortunately I am with an NGO that supported my need to run ideas by program leads, but I would not expect this for every organization.
Flexibility: When volunteering, it can feel a bit daunting to enter an agreement without previously responding to an in-depth RFP or signing a pre-determined work plan. Don’t get me wrong, I love a regimented plan (I would argue that most evaluators do), but the flexibility was appreciated. It allowed for me to shift my approach with ease and provide more meaningful deliverables. My most recent example of this comes from attempting to report some administrative data only to find that most of it was missing. I decided to evaluate the process of collecting, storing and utilizing data – resulting in recommendations for process improvement.

Strategies for Success

Now that I have hopefully shed a positive light on evaluator volunteerism, I want to share five tips to help you succeed in your future volunteer opportunity.

Be intentional: You are giving up your time (and likely money) to volunteer. Approach the project or tasks with intent to ensure change is made or meaningful deliverables are left.
Have patience: They may have welcomed you with open arms and are excited to have an evaluator on board, but this does not mean they are equipped to handle an evaluation. Be patient as you ask for data or learn what previous evaluation efforts (if any) exist within the organization.
Make a plan: I mentioned earlier that it is nice to not have a set work plan or contract – this doesn’t mean you should avoid planning entirely! Once an organization learns what skills you bring to the table, scope creep can become a very real threat. I recommend developing a plan and sharing it with your main contact to ensure everyone is in agreement of what you can offer/ have time to do.
Build a network: One of the best parts of volunteering abroad is the expat community. Even an introvert like myself found that it was easy to meet other volunteers and expats. You will find that you have a lot in common and they will naturally become your support system while abroad. Start to connect with past or current volunteers via social media before you leave, they will be eager to share their insights and tips.
Own the experience: You are there because you want to be. It may not always be exciting work, but enjoy it! Meet new people, learn about the organization as much as possible, and be proud of the work you have done.

Being a volunteer evaluator has been very rewarding and I have learned more than I had expected to. Although the path to scouting an ethical volunteer opportunity, planning for my departure and living in a new country were not easy – I can say that it was 100% worth it. If you are considering a volunteer opportunity and want to learn more, please get in touch with me on LinkedIn.

Written by: Samantha Larose

Sources

Caligiuri, P., Mencin, A., Jayne, B., & Traylor, A. (2019). Developing cross-cultural competencies through international corporate volunteerism. Journal of World Business, 54(1), 14-23. https://doi.org/10.1016/j.jwb.2018.09.002

Guiney T, Mostafanezhad, M. (2014). The political economy of orphanage tourism in Cambodia. Tourist Stud. https://doi.org/10.1177/1468797614563387

Guttentag, D. A. (2009). The possible negative impacts of volunteer tourism. International journal of tourism research, 11(6), 537-551. https://doi.org/10.1002/jtr.727

Raymond, E. M., & Hall, C. M. (2008). The development of cross-cultural (mis) understanding through volunteer tourism. Journal of sustainable tourism, 16(5), 530-543. https://doi.org/10.1080/09669580802159610

Thoits, P. A., & Hewitt, L. N. (2001). Volunteer work and well-being. Journal of health and social behavior, 115-131. https://www.jstor.org/stable/3090173

Email Address

We respect your privacy.

Thank you!

Mar 31 2020

Presenting Data While Working Remotely: Audio, Lighting, and Speaking Tips

I recently had the chance to talk with my friend Jon Schwabish of PolicyViz. You might recognize Jon from our earlier interview about tips for doing and teaching data visualization in cultures other than your own.

Our latest conversation was part of the Urban Institute’s Data@Urban Digital Discussions series. The intention of these talks are to connect with each other while we’re all social distancing. Check out the full line.

You can also watch our full discussion below Jon asked me some specific questions as well as opened up to the attendees for their questions and I’ll go over those below.

Want to skim the highlights from our discussion? Keep reading!

Which Data Visualization Trends Are Here to Stay?

Jon asked, what is one technique or trend that you’ve seen that you think is going to continue? What’s a challenge that you’ve seen that keeps popping up for the folks you’re working with?

Being a good writer is important, but being a great speaker just became more important than ever.

We have to connect with other people (over phone calls and video calls) while quarantined.

We have to stay in touch with fellow humans.

But we also have to get people to know, like, and trust us… so that they trust and use our data.

We can’t just write reports and make pretty slides. Reports and slides aren’t enough anymore… and have never been enough. Speaking skills matter more than ever.

What Do You Help Clients With?

When you’re working with clients, what are your main things that you’re trying to help them think about? How do you help them move past just creating the report?

Let me share an example from a recent client meeting.

A few weeks ago, I was in California to deliver a keynote speech at the U.C. Davis campus. After the keynote, I also met with a small group for a few hours to work on one of their data projects.

During that meeting, we focused on not putting all our eggs in one basket.
The group had put significant effort into developing a (really good) interactive dashboard for their website.

Dashboards are great for specific audiences. But they’re not enough on their own. Dashboards can reach some, but not all, of our intended audiences.

We pulled up the Urban Institute’s pyramid and talked about which dissemination formats are the best fit for certain audiences. As a group, we reached consensus that we needed more than just a report, or more than just a dashboard.

So while I do a lot of consulting on the specifics of reports, dashboards, presentations, and infographics, I also do a lot of consulting related to communication strategies more broadly. I want to make sure that organizations have selected the right mix or reports and/or dashboards and/or presentations and/or infographics before we spend our precious time, money, and mental bandwidth actually making or improving on those designs.

Tips for Presenting Data Over Video Calls

What are your top tips for setting up a good home office, especially for webinars and video calls?

I’ve been working from home for six years, and I’ve traveled full-time for the past year. I’ve got my “home” office set up really nicely! I can speak on podcasts, video calls, and record online courses from anywhere with minimal tech equipment.

Why Video Calls Matter

During this COVID-19 quarantine, we need quality human connection more than ever. We need to be able to see and hear each other on video calls (over Zoom, GoToMeeting Skype, etc.).

My public speaking coach taught me that only 40 percent of communication is the actual words that you use. The remaining 60 percent of communication is related to facial expressions, tone, pitch, using your hands to reiterate your main points, and so on.

You must look and sound professional during those video calls in order to get your point across. Sixty percent of your communication depends on it!

I suggest you focus on three areas:

Audio
Lighting
Speaking skills

Audio Quality

Want to sound professional and be an effective communicator? We all need to invest in separate microphones.

You don’t need to spend a fortune. There are plenty of microphones in the USD $50 – $100 range.

I use this USB microphone and love it. It’s small so it even fits in my backpack while traveling for work, which means I can record online courses or speak on podcasts from anywhere in the world.

If you don’t have a separate microphone, then at the very least, you’ll need to use the ear buds that came with your cell phone. Those ear buds will have a built-in microphone. They won’t give you perfect audio quality, but they’ll be better for your audience than using your laptop’s built-in mic.

Lighting

Lighting is critical!

People can’t connect with you if they can’t see your face.

The number one thing to remember is: You don’t want to be backlit. Make sure your light source isn’t behind you.

The best option is to have a natural light source (i.e., your window) in front of you. But, you also have to consider what’s behind you. For example, my current background is a spare bedroom in the Airbnb rental we’re in. Not my top choice. But we’re all making-do right now.

I’m a huge fan of separate LED lights. It doesn’t matter how amazing and life changing our dashboard, graph or presentation is, if people don’t know, like, and trust you as the presenter of that dashboard. Trust is huge with data, and that’s why I use separate LED panels. So people can see my face! I’ve used these LED panels for years and love them.

When buying a LED light panel, you want to make sure that it has:

Knobs that allow you to adjust low-high brightness level. If the light is too bright, you’ll look sweaty and shiny. You’ll need to adjust the brightness level and you’ll need to adjust your distance from the lights. I make sure to stand at least an arm’s length away from my lights.
Knobs that allow you to adjust the cool-warmth balance. You need to be able to adjust it based on your skin tone and what suits you best.

Speaking into the Camera

You have to look into the camera!!! Not at yourself on your laptop screen!!!

If you can only see my eyelids, because I’m busy looking down at my own face on my laptop, then you can’t connect with me as a human being.

Ann K Emery showing how to not speak into the camera: by looking at your laptop the entire time. You can't connect if you can only see each others eyelids.

It’s much easier for your audience to connect with you when they can actually see your eyes:

Ann K Emery showing how to speak into the camera- by looking at the camera. People can connect better when they see each others eyes.

Is it weird to look into your webcam? Of course. It takes practice. But it’s worth the effort. Small sacrifice for you. Big payoff for your audience.

My preferred setup is a tripod with this webcam and LED panel mounted onto it.

Prop Up Your Webcam

Your webcam should be slightly above eye level. Mine usually hits me in the forehead.

If you’re using your laptop’s built-in webcam, then your webcam will be waaaaay too low, and you’ll get the dreaded double chin.

You’ll need to prop up your laptop with cardboard boxes, a stack of textboxes, or even by standing in front of a makeshift “desk” like a tall dresser.

You need 100% of your mental bandwidth to be focused on discussing your data on camera. You can’t waste a moment by being self-conscious, and you’ll definitely be self-conscious on camera if you’ve got a double chin!

When you know that you look your best on camera, then you don’t have to worry about how you look.

Zoom Out & Show Your Hands

Zoom your camera out and include your hands in the shot.

Or, stand far enough away from your webcam so that your hands are included in the shot.

There’s lots of promising research about how your audience rates you as being more trustworthy when they can see your hands.

You don’t want to wave your hands around wildly, but you can use your hands to:

Count things off (idea 1, idea 2, etc.)
Indicate sizes (large, small, etc.)
Use your hands near your heart to express empathy
Show chronology (point down for ‘happening now’ or point behind you for ‘in the past’)

Pregnant? Show Your Beautiful Belly! (Or Not…)

If you look back at some of my older videos, they are purposefully only showing my head and shoulders. I was trying to block out my pregnant belly.

I didn’t want clients to know I was pregnant. Pregnancy discrimination in the workplace is real. I didn’t want to lose paying clients because they knew I was pregnant.

During both of my pregnancies, I got several emails from prospective clients along the lines of, “We were going to have you come speak but, heard your great news! We know you’re expecting, so we found another speaker instead.”

What a frustrating experience. To be fired before you even got the job.

If you choose to hide your belly while on camera, I get it. I’ve been there too.

My Makeshift Home Office

Here’s my latest set-up, from the Airbnb outside of Orlando that we’re calling home for a few weeks (or months?).

Picture of Ann K Emery's current home office set up that includes a webcam, LED light panels, tripods, microphone and different light sources.

Lessons Learned from Teaching Online Courses

My question is about your lessons learned from your online training courses. There are lot of platforms out there, I’m curious about your experience and what’s worked for you.

This is a day-long conversation.

I have learned so much about developing online data visualization training.

I’ve been speaking forever (well, since I started giving conference presentations about ten years ago). Then, I started working for myself six years ago. Two years ago, I finally listened to my friend Chris Lysy and started developing online courses.

Over the past two years, I’ve developed:

Simple Spreadsheets, my data analysis course;
Great Graphs: Design Principles, my software-agnostic course about data visualization best practices;
Great Graphs: Excel How-Tos, my behind-the-scenes course about advanced Excel charting tricks;
Dashboard Design, my course on making static one-pagers in spreadsheet programs like Excel;
Powerful Presentations, my deep dive on presenting visualizations in slideshows, staff meetings, and webinars;
Report Redesign, my deep dive on reporting best practices;
Soar Beyond the Dusty Shelf Report, my complimentary mini course about getting started with dataviz; and
Look Professional on Camera, our latest complimentary mini course, with behind-the-scenes tech tips for speaking on podcasts, recording online courses, and being interviewed on camera.

There are currently 540 lessons (!) inside these courses. So, it’s safe to say that I’ve learned a thing or two about what works—and what doesn’t—when it comes to teaching online.

Online Course Platform: Teachable

I use Teachable as my online course platform.

I use Teachable because I asked about 10 of my friends who make their living teaching online—meaning they make a full salary from their online course revenue alone, not just a couple hundred bucks her and there—and they all used Teachable.

That was a good enough endorsement for me!

I’m not an affiliate, just a very happy customer.

Their founder, Ankur Nagpal, also gives me a lot of confidence in their product. Ankur and his team are constantly improving. They interview instructors like me and they listen to our feedback. I love that. Sure, there are technical glitches once in a while, just like any software platform would have. But the Teachable team is constantly improving and iterating for their creators.

I also like Teachable because it’s user-friendly for students. Students log in, see the syllabus listed along the left, and see the video in the main screen. Underneath the video, you can add descriptions, images, quizzes, discussion boards, and attachments.

Screenshot of Ann K Emery's schools on the Teachable platform.

Video Player: Wistia

Teachable also uses a video player called Wistia, which is a bit cleaner than YouTube. It just has a play button and a couple of settings. I’m always aiming to make the students learning experience as easy and simple as possible.

Wistia also adjusts the resolution based on your internet speed for students and will play the best version for what your internet is like that day. I have students located in offices all over the world—I’m thinking of my international development and public health professionals in rural villages in Bangladesh—and I know they can participate even with slower internet speeds.

How to Create Content

Moving online was fairly seamless for me because I’d already been doing in-person trainings.

I see people creating courses from scratch and that’s really difficult.

But if you already have a 30- to 60-minute conference presentation that you’ve given, you can absolutely record that and turn it into a mini course.

Recording Tools: Screencast-O-Matic and Camtasia

For recording, I used to use Screencast-O-Matic . I liked it because you just hit record. It’s affordable and there’s only a little bit of minor editing to do.

A few months back I switched to using Camtasia to have more control over the editing. I’m not a master editor. I still mostly record my screen with my webcam in the corner. But now I can add text boxes, animations, and sounds.

Video Length: Varies on Purpose

Video length varies.

I used to think having a bunch of 2-minute videos was better than having a single 10-minute video. And honestly I’m still torn over this.

I’m monitoring the data behind the scenes to see what my students like the best. Right now, I purposely have a mix of anywhere from 2 minutes to 45 minutes.

Starting NOW with Online Training

I wish I started teaching online even earlier.

I started YouTubing way back in 2012, but it wasn’t until 2018 that I finally figured out that online courses were a “thing.”

Online learning doesn’t replace in-person learning. They’re apples and oranges; different formats with different strengths.

If you’re thinking about starting, then start today! You’re not behind. Now’s the time.

Using Icons Within Reports

I noticed one of the key features of your reports is to use icons to make things a little bit more intuitive. I used the nounproject.com to get icons to use in my reports. Are there any other resources you’d suggest for icons?

I’ll list out the options from worst to best:

Google Search

Don’t do a Google search for download free icons. This gets you in copyright jail. There are better options.

Microsoft Office

If you’re using Office365, then you’ve got free icons in PowerPoint, Word, and Excel. Just go to Insert àIcons. You can customize the color, the size, and so on.

The Noun Project

The Noun Project has both free and paid versions.

The free version…

Gives you access to thousands of icons
Only lets you download the icons in black
Requires that you cite your icon source

The paid version—which is a whopping $40/year, what a steal!!—is such a great resource. The paid version…

Gives you access to ALL their icons
Lets you download the icons in the custom color (RGB, Hex, etc.) of your choosing to match your brand guidelines
Doesn’t require that you cite your icon source (One less thing to remember; One less chance of going to Copyright Jail)

Custom Made Icons

The top tier is to hire a graphic designer to create custom icons for your organization.

I’m starting to see a lot more organizations do this and am always suggesting this to my clients.

A designer can come up with a couple dozen icons that your organization can use over and over again. The designer can also ensure that your icons match—that they all have the same edges (square vs. rounded), the same line thickness, and so on. These are small details that will make a report look as professional as possible.

A professional graphic designer might charge you anywhere from a few hundred to a few thousand dollars for this, depending on what you need.

Your Turn

How are you coping with presenting data while working remotely? Do you have your own audio, lighting, and speaking tips to share?

Mar 30 2020

Using Incentives for Data Collection

After a bit of a hiatus, we are back with a new post to our Evaluation Design series! This post explores the use of incentives for data collection: What are they? When might you need to use them? And why are they important? This topic is especially relevant considering the current COVID-19 pandemic because, as Amanda noted in her recent blog post, many of us are looking for creative solutions to keep our work moving forward. As we explore alternatives to in-person data collection methods, incentives can be particularly important for securing participation using online recruitment methods (more on that below).

What is an incentive?

Incentives are a token—typically a small gift or cash—given to a respondent in exchange for or appreciation of their participation in a study. They can be as small as a free pencil or bookmark from the museum’s gift shop or as large as a substantial gift card or cash. Museums can also get creative with the incentives they offer by providing free admission to a fee-based program or a ticket to a show in their theater—experiences that may come at little cost to the museum.

When might you need an incentive?

We often encourage museums to provide a small incentive to visitors for participating in a study (e.g., taking a survey or participating in a short interview) as a gesture of appreciation for the time visitors have taken out of their visit to provide their feedback to the museum. Incentives take on additional importance in the following scenarios:

The time invested by the respondent is significant (e.g., participating in a focus group)
Responses are needed over a long period of time (e.g., interviews or surveys taking place several months after participating in a program)
Respondents are difficult to secure (e.g., the desired respondent characteristics require targeted recruitment or respondents are difficult to reach because of logistical barriers)

If we can identify the possibility of the above scenarios during the planning phase of a project, we can work to build the cost of incentives into the budget to prevent any surprises later.

Why are incentives important right now?

As we move toward more remote data collection to keep ourselves and our communities as safe as possible through social distancing, we are reminded of the importance of incentives as related to the third bullet point above. While we would often recruit visitors in a museum to participate in an interview or survey, museum closures are making respondents more difficult to reach. Incentives are becoming increasingly important as we turn to online recruitment methods because we are asking for participation from people who may have no prior relationship with the museum (i.e., they are not already in the museum for a visit), and we do not have the benefit of building trust and rapport before respondents agree to participate like we do during in-person recruitment. Incentives can provide a sense of legitimacy to the study, encourage commitment to finish the interview or survey, and show genuine appreciation for the time invested by the respondent.

The post Using Incentives for Data Collection appeared first on RK&A.

Mar 27 2020

7 principles of a Theory of Change

A good theory of change (ToC) is based on seven principles: 1. Repeat doing it until you get it right, 2. links are as important as expected changes, 3. risk and assumptions are key, 4. it must be scalable, 5. it is not a results chain, 6. organisations often already have ToCs in place which can’t be ignored, and 7. it has to be simple.

The post 7 principles of a Theory of Change appeared first on Thomas Winderl.