allblogs

There have been a few times in my life when I decided to do something and then, as the thing approached I thought “What have I done? This is too big and too scary and too hard and I’m totally not going to be able to handle this!”. Moving across the country to do a PhD. Play in a hockey game that lasts for 10 days. Do an MBA part-time while still working full-time. Accepting my current job. As it turned out, all of these were things that I could handle and are things of which, as it turns out, I’m extremely proud! It’s almost like being scared that I’ve bitten off more than I can chew is a sign that I’m about to do something awesome.

My job prior to my current job was fun and I learned a lot and I met some great people, some of whom I’m still good friends with (Hi Heather!). But after 5 years in that job, I’d hit a pay ceiling, I’d learned all that I could learn, and so I wasn’t feeling challenged any more. And then a co-worker of mine told me about a job posting she’d seen that she thought I might be interested in. It was a job doing the same type of work (evaluation in healthcare), but taking it to the next level. A leadership position where I’d get to run a team of evaluators to conduct an evaluation of a massive, multi-organization, multi-year project that has the chance to change the face of healthcare in the region. I was excited by the possibilities this job entailed, so I applied and I got the job. And a few days after I handed in my resignation at my old job I thought “Oh my god, what have I done? I know how to do my old job really well. But there’s so much I don’t know about this new job – I have to learn a whole new area of healthcare AND I’ll be the boss of people and that’s a whole new ballgame for me. What if I can’t do it?” What I should have realized then was, just like the PhD, just like the Longest Game, and just like my MBA, that fear was a sign of a great challenge and I’d shown over and over again that I can rise to a challenge.

The last five years have been really interesting. I’ve learned a tonne about health informatics, about applying complexity concepts to the evaluation of an ever changing project, about governance, about managing people, about managing data when you have a large group of people creating and using a huge dataset, and that’s not even getting into what I’ve learned in terms of the findings of the evaluation so far!

I’ve had the opportunity to collect data from 13 healthcare facilities and counting, I’ve built my team up from 2 to 11 evaluators (all of whom are pretty fantastic, I must say), and I’ve presented my work across Canada, as well as in the US and Australia.

And even after five years, I’m not bored. I honestly feel like we are just getting things rolling and we are improving our processes at every step, and I’m learning so much from all the amazing people on my team, and we are producing information that is actually getting used by decision makers. And there’s so much more still to come.

This is not to say that it’s been easy, or that I will be easy going forward. In a recent presentation I gave about the project at the Canadian Evaluation Society conference, I used this image to represent my experience:

I also often reference that MC Escher painting where the stairs are going up but also going down at the same time as representing what it’s like to work on the project I’m working on. (I can’t put the image here on the blog because I don’t have copyright permission, but here’s a link to the Wikipedia page on it where you can see the image)

But honestly, it’s kind of OK with me. The real world is messy and things don’t always work out how you planned them, but you learn a lot by going along for the ride.

Image sources:

• Fantasy sea monster posted on Pixabay by Barroa_Artworkswith a royalty-free license..
• Number 5 posted on Flickr by Steve Bowbrick with a Creative Commons licnese.

Cross posted on my other blog.

Applauding your engagement in professional networks and the contributions you have made to the evaluation profession and the community of practice, Beth. You really exemplify this competency. Keep up the good work!

And the final reflective practice competency is:

1.8 Engages in professional networks and activities and contributes to the evaluation profession and its community of practice.

There are several ways that I engage in this competency:

• involvement with the Canadian Evaluation Society (CES)
• teaching evaluation
• mentoring my evaluation team
• social media involvement

Involvement with CES

I first got involved with CES back in 2010, when I was looking to find my way in this profession. The national conference was being held in Victoria, so I volunteered for the conference as I figured it would be a good way to meet other evaluators and learn about the field. And was I ever right – the evaluation community was so welcoming and I met people there that I’m happy to call friends and colleagues to this day.

For the next several years, I went to the CES national conference when I was able to attend, but then in 2015 the BC & Yukon chapter decided to host a one-day conference of its own, and that’s when my involvement really took off. I volunteered to be the conference program chair for that conference – and also volunteered to be a program co-chair for the national conference which was scheduled to be held in Vancouver in 2017. That role was a tonne of work, but it was also a lot of fun, as I got to work with two delightful fellow evaluators, Sandra Sellick and Wendy Rowe. I really enjoy and get a lot from conferences (both in the content I learn and in the networking opportunities they provide) and I know from experience that they take a lot of effort, so I think that volunteering for conferences is an important way that I can contribute to the profession and its community of practice.

Also in 2015, I joined the CESBCY council as a member at large, later transitioning into the VP role when the VP stepped down. In 2017, I became the chapter president. I’m really proud of the work the chapter is doing – we are hosting a lot of professional development events (e.g., one day conference, various workshops and webinars) and meetups that serve the evaluation community.

This year I also coached a student case competition team at the CES national conference – and that was a really rewarding way to support new evaluators in our community!

Teaching

Another way that I feel that I contribute to the evaluation profession is by teaching evaluation. I’ve taught evaluation courses at both SFU and UBC, and I’ve supervised practicum students from SFU, UBC, and UVic. And several of my students have gone on to work in evaluation (right now, I have three of my former practicum students and two of my previous evaluation course students working as evaluators on my team!)

Mentoring

And speaking of my team, I currently have 10 evaluation specialists working on my team and a big part of the work that I do as the leader of the team is to mentor and support them. This is another way that I am working to contribute to the future of our profession.

Social Media

Another way that I’m involved in evaluation professional networks is online. There’s the #EvalTwitter hashtag that a lot of us connect through. There’s even a monthly #EvalTwitter tweetup on the last Thursday of every month (at 5:30-6:30 pm Pacific time). And through#EvalTwitter I learned about Eval Central, an online forum that “aim[s] to encourage positive and fruitful discussion among culturally diverse evaluators from around the globe.” So I recently joined that and am eager to see what kind of conversations happen there.

Image credits:

• CES logo is from https://evaluationcanada.ca/
• Social media icons image posted on Flickr by Sean MacEntee with a Creative Commons license.

I didn’t write a blog posting in my series last Sunday – the weekend was busy and time got away from me! But it’s now this Sunday night and I’ve got cup of tea and I’m ready to reflect on reflective thinking!

1.7 Uses self-awareness and reflective thinking to continually improve practice.

Often I do my my reflective thinking over a a cup of tea – whether sitting on my own to do some reflective writing, or chatting with colleagues (As an aside, if you want to read some brilliant thoughts on reflective practice, check out Carolyn Camman’s fabulously titled blog posting “The coffee is largely metaphorical“). I’m an external processor and I find that I tend to come up with a lot of my great “a ha!” moments when I write my thoughts down or talk to a friend or colleague. I also don’t have a great memory, so when I have an insight, I need to write it down to cement it in my brain (or the very least, so I can look it up again later.)

Journalling

I write a lot of reflections as I go about my work. Whether I’m collecting data, analyzing data, in a meeting, or whatever activity I might be doing, if I have an “a ha!” moment, I write it in my reflective journal (which for the project I’m currently working on is typed up and saved on a shared drive with the rest of my team’s reflections, as these “a ha!” moments are about the content of the evaluation that we are working on together). A reflection might be about a pattern I’m noticing in the data, or a connection I’m making between different parts of the evaluation, or a surprise that I wasn’t expecting, or thoughts on some of our longer-term evaluation questions. My general rule is “if it’s interesting enough for me to want to tell my team about this cool thing I saw or thought of, it should write it down as a reflection). This improves my practice because it helps me to identify things that are important to the interpretation of the data, which allows me to develop accurate and comprehensive evaluation findings.

I also keep some separate reflections that are more for myself than as part of the evaluation data. For example, since I’m the team manager, if I have reflections that are about my work as a manager, and I might not want to share those with the team right away – especially if I’m trying to work through a challenge or figure out a way to be a better team manager. Some of those reflections might become things that I do want to talk about with my team later, but sometimes I need some time and space to work through stuff first. This helps improve my practice because being an effective leader will help my team be effective in its work.

Team reflections

Speaking of my team, we’ve taken to having group reflection sessions after we complete any big chunk of work where we debrief on:

• what worked well
• what didn’t work well
• how might we have done things better
• what can we glean from what worked well/didn’t work well to improve our practice for our next task

These are some pretty standard evaluation type questions, but we’ve definitely been able to continually improve our practice by doing this reflection together.

For example, in our first big round of data collection, we didn’t do nearly enough documentation of our data analysis. And with having a big team of people all working on different pieces of the data analysis, it meant that we had a lot of files that we’d all named in different ways, with our spreadsheets set up in different ways and often not very well labelled. So when it came time to write up our findings, it was quite difficult to find the data we needed, and we sometimes had to reproduce some of the analyses to ensure we had the correct data. So my big lessons learned for future rounds of data collection were:

• we needed standardized naming convention that we all used
• we needed all steps of analysis clearly documented so that another person could pick up the file and understand exactly what was done (without having to sift through formulas and pivot tables to figure out what it all meant)

These seem like pretty basic things – and they are – but this was the first time for all of us working on a big team. We each had our own individual naming conventions and ways of setting up our analyses in our spreadsheets that had served us well working as individual and what we hadn’t realized was how many different ways people could do the same task! Since the project is being implemented in a phased approached, we are now entering a period of time where our work will be a bit cyclical (collect baseline data for a site, monitoring data at the time of implementation, collect post-implementation data 3-6 months later, and repeat for the next site). And I can see that we are getting better and better each time because we’ve been reflecting on how we do our work and finding ways to be more efficient and more effective.

Another reflection that I shared in a team reflection session recently was something that I think links to the “self-awareness” part of the competency. Working in healthcare, even as a non-clinician, you get exposed to situations and information that can be quite emotional. For example, even when doing a chart audit, you get exposed to stories of serious illnesses/injuries and deaths. Or when interviewing healthcare workers who are exposed to traumatic situations, you also get exposed to those traumatic situations. As human beings, this can bring stuff up for us (like similar illness, injuries, patient journeys, and deaths of loved ones, for example) and it’s important to be kind to ourselves when stuff like this gets to us. I am extremely lucky that I work in a large team made up of kind and caring colleagues, so we know that we can go to each other if we need to debrief, or if today is just not a good day for us to do that particular observation or interview. Being aware of situations that might bring up things for me and being aware of my emotions as I’m experiencing them can help me to manage those, ask for help when I need it, and thus help to ensure that they don’t negatively effect the work. It can also help me to be empathetic to my colleagues and the people I interact with as I do my work.

In addition to reflection with my team of evaluators at work, I am also part of a co-op inquiry group that meets monthly to reflect on a particular topic (for us, it’s “boundaries in evaluation”) and that has been an amazing experience to hear the reflections of a group of evaluators from different sectors and locations – I have left every meeting having expanded on ideas I’ve been having and having learned new ideas or perspectives from my colleagues that have resonated with me.

Teaching

Teaching is a fantastic opportunity to reflect. Whenever I prepare to teach an evaluation course, I’m dedicating time to stepping back and thinking about the big picture of evaluation – what it is and how to do it well. I find it also brings me back to the basics and it gives me the opportunity to think about whether there are ways that can improve what I’m doing. I use a lot of storytelling and examples when I teach – I’ve had many students tell me that they really appreciate that I do that because I tell them “what really happens, as opposed to what the textbooks tell you it’s going to be like”. But it also helps me because, again, it gives me an opportunity to think about how I’ve done my work, how it links to concepts, theories, standards, etc. and how I might do my work in the future.

In addition to getting back to basics, I also like to tell students about whatever the “hot topics” are in the field at the time, which means that I have to keep abreast of what the hot topics are, and typically do a bit of research to be well versed enough in the topic to discuss it with the class. This is an opportunity for me to identify gaps in my knowledge and do some learning.

Another aspect of teaching that I think is reflective is that students tend to ask really great questions. And since they are coming from a different perspective, sometimes those questions are things that I haven’t thought about before, which forces to me to reflect situations from a different angle. Sometimes they ask questions that I do not know the answer to – when that happens, I tell them that I’ll go do some research and get back to them. This links to that notion of self-awareness – knowing the limits of my knowledge, having the confidence to say “I don’t know that right now, but I will find out”.

Blogging

And finally, this blog is something that I’m using as part of my reflective practice now. I’m glad that I decided to write this blog series on the evaluator competencies as a way to provide some structure and timeline to get me in the habit of reflecting here on a regular basis ¹Last Sunday notwithstanding.. I’m finding it quite useful to spend a bit reflecting on the extend to which I have each of the competencies and areas for each where I can continue to learn and grow.

Image Source:

• Pot of tea photo posted on Flickr by Jack with a Creative Commons license.