carolyncamman

“Can we even get that data?”

That question, or some version of it, is usually one of the first, if not the first, question I hear when planning or discussing a new evaluation project. People want to know if it’s possible to collect data on a particular outcome or from a particular group. There’s often an undertone of, “I bet we can’t,” in the question too.

From the outside, I think evaluation must look a lot like survey + interviews = report. Those are the parts of the process that most people actually get to interact with after all. (Which is too bad, because they are also the least impactful parts to experience in terms of process use.) So I can understand why questions about data collection (and reporting deliverables) often dominate early conversations I have.

But data collection is not the hardest part of evaluation. My answer to, “Can we even get that data?”, is “Yep, probably.” In some way, in some form, we can get data on that or from that group of people. But there are so many more essential questions to be asked first. Do we need that data? What are we going to use it for? Do we have a plan for how to use it? Do we have the people, resources, and processes in place to make sure it’s going to get used?

Because here is the thing. If you have gasoline, you generally also want—at minimum—a gas-powered vehicle, a capable driver, and a destination in mind or at least a general direction to start in, otherwise you’ve got a lot of something you can’t use very effectively. Same goes for data. And while you want to factor stopping for gas into your road trip plans, it’s probably not where you start the planning process.

A flaw in that analogy though is that it doesn’t reflect just how much of a problem it can when we focus on data collection to the detriment of data use. When we collect data we don’t need and can’t use, or *do* need but fail to use well, the consequences can range from wasteful to, in the most extreme circumstances, deadly.

Julia Coffman wrote up one of the most compelling examples of this in her article, Between the Devil and the Deep Blue Sea: The Consequences of Small Failures in Learning. In it, she explores a fatal shipwreck caused by what should have been an avoidable collision with a hurricane, had it not been for the persistent misuse of data (in this case meteorological data about the course of the incoming storm). There was enough of the necessary data to avoid the collision as well as people highly qualified to interpret it on the ship, but fatal decisions were still made. Julia explains the cognitive and interpersonal biases as well as situational factors that may have contributed to the misinterpretation and misapplication of data, though we can’t know for certain as the main decision-maker—the captain—died along with his crew.

Just having data doesn’t mean we will use it well, and having too much data or the wrong kind of data can be misleading as well as a waste of resources. As this article from the Stanford Social Innovation Review points out, with technological advances it’s getting easier and easier (and cheaper) to collect, store, and analyze data. But a tool like a dashboard only produces higher-level data of your raw data, it can’t tell you what it all means or how to use it effectively. That requires human-level interpretation and application, which isn’t getting any faster or cheaper or easier and isn’t helped by a flood of irrelevant information without a meaningful practice for making sense or use of it.

The latest example I’ve found of how easily and chronically we end up in these bad habits with data is a is New Yorker review of various books on the history of spies and intelligence agencies, which sums up the main takeaway as, “The history of espionage is a lesson in paradox: the better your intelligence, the dumber your conduct; the more you know, the less you anticipate.” Yep, that’s right—there’s a long and storied tradition of intelligence-gathering being self-defeating and counter-productive.

Espionage and evaluation differ many ways , but fundamentally we’re still talking about data-gathering to inform decision-making and we’re in the territory of human fallibility, so many of the problems surfaced in the article sounded awfully familiar. For example, “Not for the first or the last time, the point of spying—to know what the other side is likely to do—had been swallowed up by the activity of spying,” reminds me of this statement from another article on over-measurement practices, “Micro-measuring what we have done seems to be more important than what we actually do.” Process subverting purpose!

The problem of volume and data overload comes through in this comment, “if you have any secret information at all, you often have too much to know what matters”. And the issue of focusing on the wrong kind of data, “The two agencies were so busy spying on each other, it almost seems, that they forgot to spy on each other’s government. Knowing what the K.G.B. was doing wasn’t the same thing as knowing where the Soviet state was heading, and the rise of Mikhail Gorbachev and the fall of the Soviet Union came as a complete surprise to the C.I.A.” There’s also a repeated pattern of accurate, useful intelligence being suppressed or ignored because of “… confusion, political rivalry, mutual bureaucratic suspicions, intergovernmental competition, and fear of the press (as well as leaks to the press), all seasoned with dashes of sexual jealousy and adulterous intrigue.” (Okay, those last two probably come up less often in evaluation, but the rest are commonplace!)

The role of trust and transparency in the misuse of intelligence fascinated me too. “The universal law of unintended consequences rules with a special ferocity in espionage and covert action, because pervasive secrecy rules out the small, mid-course corrections that are possible in normal social pursuits. When you have to prevent people from finding out what you’re doing and telling you if you’re doing it well, you don’t find out that you didn’t do it well until you realize just how badly you did it.”

Maybe this is less relevant to the use of evaluation findings since there technically we’re not required to operate under a shroud of secrecy the same way spies are. But don’t we end up doing so much of the time anyway? Evaluation is political, and a lot of it happens quietly, in-house, with little-to-no publicly available documentation, and minimal participation by most people affected by the process in anything other than providing data and maybe having access to the final report. While participatory approaches to evaluation seem to be gaining in usage, I’m not sure they’re anywhere near being the norm. There can be a real culture of fear around data and how it might be used to reflect badly on organization or program that prevents, or at least seriously limits, actual useful discussion and application of it.

I won’t minimize the political nature of data and how it can be weaponized, but I’ll offer that blanket secrecy is not necessarily the most effective way to manage that risk, especially with the cost to utility. This is where the value of having a clear evaluation strategy—a direction and purpose behind why the evaluation is being conducted and how it will be translated into use—can benefit again. It ensures that data are collected with purpose and intention and can be interpreted and applied in light of that purpose, with a clear explanation of why that data, that interpretation, and that use. The more robust the evaluative reasoning, the harder anyone else is going to have to work to offer a counter. And, bonus, you end up with a better evaluation either way.

One of the first posts I put up here was one about advice for emerging evaluators, based on my own experiences of getting into the field. I wrote it about a year after moving to Vancouver and six months after I committed to building a consulting practice. Three years later, I’m pleased with how well that post holds up. I might word a few things differently now, but the gist would be the same. And I’m pleased to report that I took my own advice (for once) and it’s stood me well as a developing evaluator and consultant. So much so that the last year in particular has been one of transformation and levelling up!

Over those three years I’ve kept learning, though my own experiences and through talking with all sorts of wonderful folks, both inside and outside the field of evaluation, about their own career trajectories. I want to make sure all of that wisdom get a wider circulation than just my own brain and decided it was time to revisit and add to this topic. I originally planned to do just a one big follow-up post, but one thing I’ve learned through upping my blogging game this year (have y’all noticed how much better I’ve been doing? One a month at least now! Thank you, thank you, oh, the applause is too much, thank you, you’re too kind!) is that shorter and more frequent is better. So I’m going to try to make this an on-going topic that I can add to as I go and focus on just one or two things at a time.

And just to summarize, the headline versions of the original advice were:

1. Have a vision

2. Be visible and accessible

3. Get involved and meet people

4. Keep learning (about anything!)

5. Do your research (specifically on the field itself)

So here it is, one more thing you can try as an emerging professional (since a lot of this is not evaluation-specific). Once again, I offer no guarantees to this advice besides my sincere promise that it’s what I’m trying to do myself. (Also I’m starting with one of the things I find absolutely hardest to do as a way to motivate myself to do it more. Mmm, public accountability.)

6. Know your value

I picked the exact wording for this one up from a tweet about tips for successful collaborations. I endorse the entire tweet, but that part stuck with me particularly because as soon as I saw those three little words I went, “Ah ha, yes! That’s the thing I’m bad at!”

What do I mean by “know your value”? How I interpret that is try to have a reasonably accurate understanding of what you offer and how that’s received and experienced by the people you work with (whether that’s clients, colleagues, employers, etc.) and what it’s worth to them. That includes the value of your skills and experience (which will continue to grow), the ineffable value of you as a whole and unique individual, and your fundamental worth as a human being (which is, y’know, baseline and non-negotiable), and is about money-stuff as well as general perception of you and why people are interested in working with you.

There’s a couple pitfalls here. One is false confidence, or overestimating the value of your professional skills and experience in particular and assuming you bring more to the table than you do. That’s the one I’ve always been most worried about, but it turned out I was causing myself problems with the other pitfall, false humility (a.k.a., underestimating the value of your skills and experience and also who you are and your personhood too sometimes).

It feels safer to underestimate rather than overestimate, but the issue is that when you discount your own value (sometimes literally, by under-pricing your services), you’re actually cheating yourself and the people you work with. You undercut your own confidence and capacity, and, if you’re literally under-selling yourself, you’re also helping set a normative standard of what work in your field is worth and what other people who do work like yours can expect to be paid. It’s complicated because in evaluation and other similar professions, we’re often working with organizations and communities that are under-valued and under-resourced as well and we understandably don’t want to impinge too much on those resources, especially if we’re not feeling confident in the value of what we offer. But we need to be thoughtful about how we’re participating in these systems because we can easily be part of the feedback loop that keeps us all stuck in the same patterns that reinforce our collective under-valuing.

(Here’s some bonus advice that comes to me via a therapist friend who was struggling with setting his fees because money can be such a barrier to accessing mental healthcare, especially for the people he wanted to support most. The guidance he got from his mentors was that if you set your prices too low for you to sustain yourself on or only take clients at the very bottom end of your sliding scale, you will end up under-resourcing yourself and risk exhaustion and resentfulness toward your clients, which will then show up negatively in the therapeutic relationship. It’s not that we need to be content with the terrible systems we’re operating in, but our resistance must also be strategic and we can’t treat ourselves as expendable in the process.)

So how do you figure out a reasonably accurate understanding of what you have to offer and what it’s worth to other people?

On the money front, what I did for a long time (and I don’t recommend this part) was grossly undercharge (because after being a grad student, everything feels like a lot of money!) and fortunately had a lot of kind, patient people (fellow consultants and occasionally clients) who pointed out to me what I was doing and made sure that I got paid more until I eventually figured out that I needed to raise my rates myself. I panicked a whole lot about alienating people by “asking for too much”, but I also paid close attention to my budget, my overhead, and my income and figured out what kind I needed to be making to make a sustainable go at this and what seemed plausible based on what I knew about the local evaluation market (from working on projects, from learning about other consultants’ rates, and from scanning RFPs and job postings that had budgets and salary details included). And then I didn’t go with the lowest number I thought I could get away with—I went for something that I knew I would meet my needs and let me bring my best to my work rather than put me in a position of chasing contracts, stressing about hours, and feeling obliged to work on projects that feel wrong but pay well, or appear to (I’ve been advised those are usually the ones people regret most and I believe it). I went for something that gives me flexibility to keep doing volunteer work. That keeps me mindful of offering maximum value for the time I spend on my projects and not just grinding through. That communicates to prospective clients that I believe in the value of what I offer and that our work together is a commitment for both of us. The first time someone was noticeably disappointed with my stated rate was pretty nerve-wracking, but I survived.

(I realize I’m not giving specific numbers, which I know people always want, and that’s because I’ve learned that appropriate rates are very, very contextual and I can’t do the nuance justice in a blog post. So I can’t tell you what you, personally, as an emerging professional should charge or expect to see as a salary, because it depends so much on where you live, what specific work you’re doing, what kind of experience you have, and a lot of other factors. But, reiterating the advice above, look at RFPs and job postings and talk to other people doing similar work as yours. If you’re moving into consulting specifically, remember that you aren’t just charging for “the amount of work I can do in an hour” or whatever time increment—you need to factor in all of the overhead expenses that you can’t bill directly for, like your office supplies and equipment, professional development, healthcare expenses, accounting fees, etc. Consulting fees are typically 2-3 times higher than the equivalent “hourly” rate of a salaried employee because you have to cover what would otherwise be covered by your employer’s operating budget. There’s a lot of good guidance out there on this topic, like this article that also links to other resources, such as this hourly rate calculator. Gail Barrington’s Consulting Start-Up and Management is also a classic consulting reference text in the evaluation field for a reason and covers some financial basics.)

And then on the other value front, the “why in the hell do people actually want to work with me?” front (since that’s actually what I’ve had on my mind more lately despite all the money talk in this post), for that I had to do a lot of thoughtful listening and getting humbled on the false humility. Once again, other people to the rescue! Sometimes it’s hard to know the impact we have on other people (that’s literally one of the reasons evaluation jobs exists!). We’re so inside our own context that we lack context for our context. What I learned is that I don’t get much insight from focusing on what I think people get from working with me, I have to listen and pay attention to what they say and do. Am I getting repeat clients? Are people asking to work with me again? Recommending me to others? Saying nice stuff behind my back (a.k.a., reverse gossiping)? Cool, then I can assume that I’m doing something people appreciate and respect. And when I don’t see these things, I can look deeper. Was I not bringing my best? Was it a poor fit? Is this an outlier or a trend? Not everyone will love me and that’s fine, but it’s always interesting to notice patterns.

The thing about value is that it’s qualitative as well as quantitative. It’s not a great strategy to just maximize the total number of people who want to work with you (you’ll get tapped out quickly and pulled in a lot of different directions because the work is so varied), you want to be attracting the people that you want to work with and that make sense for you to work with. So look to those folks and find out what they like about you. Ask them. If they’re a client, do a post-project check-in, do some meta-evaluation on yourself and your process. If they’re a colleague, ask them what they’ve noticed about working with you. And don’t count on just one person’s perspective either. Different people will have different takes and see you from different angles (this is the logic of the “360-degree review” process, although you don’t need to go to such elaborate lengths).

The point is not just to buff up your ego (and you can and should take in the “needs work” feedback too—critique that comes from a place of mutual respect is a gift), but to get a clearer reckoning of what your contributions are so that you can work to a place of honest, realistic, and grounded confidence and humbleness. It’s got some less obvious practical implications too. A lot of what I write in bios and website copy and other forms of marketing (everyone’s favourite thing to write about!) is informed by real feedback I’ve gotten from people I’ve worked with. Start gathering up that kind of insight now, it’s a very useful indicator of the kind of professional you’re developing into.

If you listened to our recent episode of Eval Cafe with Michael Quinn Patton on principles-focused evaluation, you’ll remember him sharing his new favourite example of principles in action. It’s from the introductory article of a recent special issue of The American Statistician, which is all about moving beyond the use of p < 0.05 as the threshold for determining statistical significance. The article offers an impassioned explanation of why abandoning the entire concept of statistical significance is necessary and also outlines the beginnings of an alternative practice for valuing and interpreting statistical findings. The reason it showed up in the podcast is because the authors ground this new framework in principles, or flexible advice that can guide decisions and give direction, but must be adapted and interpreted in context. In comparison, p < 0.05 is a rule—it is applied the same way regardless of any contextual factors. (Check out the podcast and also Michael’s book, Principles-Focused Evaluation, to learn more about the implications of principles for evaluative work.) Specifically, the principles that the authors offer are, “Accept uncertainty. Be thoughtful, open, and modest” (or “ATOM”, as a mnemonic), and the remainder of the issue (43 articles worth!) goes on to offer more depth around the issues of p < 0.05 and the discussion of alternatives.

For an academic publication about statistics, it is, frankly, stirring. Read this:

“At times in this editorial and the papers you’ll hear deep dissonance, the echoes of ‘statistics wars’ still simmering today (Mayo 2018). At other times you’ll hear melodies wrapping in a rich counterpoint that may herald an increasingly harmonious new era of statistics. To us, these are all the sounds of statistical inference in the 21st century, the sounds of a world learning to venture beyond ‘p < 0.05.’ This is a world where researchers are free to treat ‘p = 0.051’ and ‘p = 0.049’ as not being categorically different, where authors no longer find themselves constrained to selectively publish their results based on a single magic number. … As we venture down this path, we will begin to see fewer false alarms, fewer overlooked discoveries, and the development of more customized statistical strategies. Researchers will be free to communicate all their findings in all their glorious uncertainty, knowing their work is to be judged by the quality and effective communication of their science, and not by their p-values.”

(Has a paper on inferential statistical testing ever brought tears to your eyes? This brought tears to mine. The freedom in it, the vision of it, the clarion call to a remembered sacred purpose of meaningful scientific discovery—it’s poetry. And, honestly, the whole article is a delight to read and it’s open access. Treat yourself!)

So what’s wrong with p < 0.05? Why devote an entire issue of an journal to explaining why it should be done away with?

The problem is that it’s a seductively simple idea that was never equipped to be used the way that it has been. We took something that might have been an okay guideline for thinking about whether to explore a statistical relationship further and turned it into something so hard-and-fast that careers are made and broken on it, that people are incentivized to do bad science because of it (whether that’s bad actors manipulating results or more subtly the cumulative, unintentional harm of something like the “file drawer problem”). We took p < 0.05 and applied it thoughtlessly, rigidly, imperiously, and with total disregard for context. To the point that the statisticians tell us that the solution to p < 0.05 is not to tweak it, to start using “p < 0.10” instead or confidence intervals or to come up with a more complicated system of rules that let us keep doing essentially the same thing as we always have but “better” this time. Instead they ask us to recognize that the entire concept of a fixed threshold for statistical inference is flawed and we must shift to a way of thinking that incentivizes nuance, humility, and care. It is a call to transformation, to a world beyond p < 0.05.

So why am I bringing this up, since this post isn’t actually about statistics? (Surprise!) Bear with me—we’re about to go on a bit of journey.

I bring up p < 0.05 and its critiques because I realized that to me it speaks to the same issues I have with sex and gender binaries*. We have taken what is a fluid, complex interplay based around complementary elements that are meant to be more like tent poles, lifting up a fabric of possibility that drapes around and between them, and we have stripped them of nuance and severed what connects them. We have sacrificed depth and breadth in our understanding and experience of sex and gender for convenience, control, and predictability, leaving ourselves with two bare stakes in the ground. Because just like with p < 0.05, there’s something seductive about the notion of a fundamental, highly predictive, nearly-universal binary division of sex and gender, something deeply appealing about the idea that this dichotomy is rooted in biology and manifested at all individual and sociocultural levels. We can see the evidence of the attraction in how often it shapes the most basic of our everyday activities—going to the washroom, putting on clothes, talking about other people, filling out forms, etc. All of which feeds back into the perception that these are innate, meaningful, nigh-universal differences, which is why it is so powerful to take note of where the contrasts, contradictions, and variations arise, in language, biology, psychology, culture, and elsewhere. And how these variations are not statistical noise and or mere outliers but vital parts of the whole picture.

Because I’m not saying there is no pattern to sex and gender. I’m saying that the pattern has been reduced, over-simplified, and blown completely out of proportion, with individual components being mistaken for the entire phenomenon, like a painting being the sum of the pots of paints used to produce it. Now imagine a world in which every piece of art was classified based on its relative proportion of red and blue hues just because those are two primary colours. Whole galleries divided into “red wings” and “blue wings”. Vast swathes of art from across the colour spectrum lumped together without regard to the rest of their palettes (and never mind all the other defining characteristics one could consider about them). New technology devised for the sole purpose of pinpointing with stunning accuracy the exact amount of red and blue present in a given piece. The stubbornly unclassifiable relegated to storage because it’s only a small proportion anyway and it just upsets what is otherwise such an elegant, simple binary scheme. Because establishing clear, bright lines around sex and gender categories does make life easier in many ways. But convenience comes with a cost (think about Amazon and Uber), usually a profound human cost that is disproportionately distributed along lines of power and influence although we each pay our own price for it.

Rachel Pollack, a science fiction and comic book author and trans woman, captures the heart of the struggle with in her recent essay, “Trans Central Station”, where she shares her experience of coming out as a trans woman in the 1970s (a time before the language of “transition”, “transgender”, and “trans woman” even existed in English):

“What I felt, what I desired was unspeakable because, for me, at least, the words did not exist. Or rather, the telling did not exist. … The mind could not form the thought. I did not wish to tell people and didn’t dare. I simply could not imagine doing it. … I was not trapped in the wrong body, I was trapped in the wrong universe. In order to become who I was, I had to break the world open. I had to embrace a kind of science fiction life. … The physical world may be made out of elementary particles (and dark matter) but the world of our lives is made out of language. With the wrong language, one of strict categories and confinement, the world becomes a fake, a stage set whose actors don’t know they are in a play … Most people do not notice this because their own sense of self, of language, more or less fits the received version of existence. They still suffer, for in a world of strict and very limited categories, they must constantly check themselves against the model of a ‘real man’ or a ‘real woman’. The ones who reveal the fake are the ones who simply cannot make themselves fit. To not fit can bring great pain and often very real danger, yet who else can discover the light behind the screen?”

As I shared when I wrote about my pronouns, the words to describe my gender (and my sex and the relationship between them and my relationship with them) don’t exist in my language. I borrow words like “queer” and “trans” and “nonbinary”, because they come as close as they can, but they are a finger pointing at the moon, not the moon itself. I suppose all of language is a finger pointing at the moon, but there are varying degrees of remembering and forgetting that the finger is not the moon. With p < 0.05, we forgot the moon existed at all, and I think we do the same with “male” and “female”, “man” and “woman”, the lonely tent poles that they are. Because I don’t think my gender is any more mysterious or complex than anyone else’s or any less coherent, save for the misfortune of being born in the wrong language, one that has the idea that these things are governed by rules, clear and delineated and precise. Sex and gender are messy, multifaceted, ill-defined, and exhilarating. They’re never going to boil down to simple rules so we should start thinking about what kinds of principles might help us here. We need to embrace the ambiguity rather than keep trying to erase it, because it belongs to all of us, not just those of us who simply cannot make ourselves fit.

I don’t have 43 articles to share of deep exploration into this issue and possible alternatives. But even if I did, our friends at The American Statistician still had to warn their readers, “What you will NOT find in this issue is one solution that majestically replaces the outsized role that statistical significance has come to play. The statistical community has not yet converged on a simple paradigm for the use of statistical inference in scientific research—and in fact it may never do so. A one-size-fits-all approach to statistical inference is an inappropriate expectation, even after the dust settles from our current remodeling of statistical practice (Tong 2019).” A one-(or-two)-size-fits-all approach to sex and gender is unlikely to be forthcoming either. I also don’t have principles to offer for a better understanding of sex and gender right now, though, “Accept uncertainty. Be thoughtful, open, and modest”, are pretty good ones to start with.

We’re going to keep wrestling with these issues, standing in the world of now while we look to a world beyond. (And that world is coming—you can watch it happen in the flux and growth of the language and the ideas that are becoming available to us.) One thing we need to do while we experience this uncertainty is to not try to negate it or reduce it or look for a newer, better p < 0.05 that will let us carry on as usual. It’s not about tacking on an extra category or two. It’s about letting go of the seductive simplicity altogether, and finding a way forward that allows for nuance and wholeness.

That’s what I find so inspiring about the p < 0.05 article. It’s a beautiful exploration of how (and why) to move from disastrous over-simplification to an adaptive embrace of complexity (and within an institution that is itself complex, with a life and momentum of its own that resists change out of an impulse for self-preservation which is understandable even as we recognize that to resist change is also an act of self-destruction through obsolescence**). It may contain no absolute answers (that would be too simple), but it has an abundance of hope, compassion, and courage, as well as a frank reckoning of the systemic, institutional challenges to such a profound shift. I want to see the same combination of depth, hope, and strategy brought to our conversations around sex and gender as well.

The transformation is out there. What starts as science fiction can become the art that life imitates.

Happy Pride, y’all.

*I’m referring to “sex and gender” throughout because while they can be thought of as different things, they are BOTH complex and BOTH socially-constructed. It’s not a case of “sex is simple, gender is complex”. If you want to know more on that, check out the readings I reference at the end of my pronoun post.

**I think I’m going to start summarizing this paradox as “change and/or die”.

BONUS:

Podcast episode recommendation! Check out the Indigiqueer episode of the All My Relations podcast for a look into gender and sexuality from Indigenous viewpoints.

Transformation is awkward.

I mean, we all know that change is hard, but it’s also awkward.

It’s been a big year for me. The last ten months in particular have included a lot of things I never saw coming and just got to run with as they came my way. I’ve made a lot of new friends. I’ve gotten to have some incredibly transformative learning experiences (the Evaluation for Social Change and Transformational Learning program through SFU and the annual Art of Hosting training on Bowen Island being two I can strongly recommend to others!). I’ve been graced with some amazing opportunities to step in and push myself to new levels. I’ve had to deal with some setbacks, disappointments, and missteps, though I contend that all my best mistakes are still ahead of me. (I can only aspire to be a future contributor to a follow-up volume to the wonderfully honest and generously insightful, Evaluation Failures: 22 Tales of Mistakes and Lessons Learned.)

When I think back to myself a year ago, I’m astounded. It’s not that I was doing so terribly before, but it’s such an unexpected, qualitative difference. In less than a year, I’ve transformed my practice. I’ve clarified (at least for now) my guiding principles. I’ve discovered personality traits I didn’t know I had. I’ve changed things I thought were immutable personality traits! (This is why it makes so much more sense to think of personality as a system, as an aside.) My social and professional networks (around here those are basically the same thing) quadrupled at least (I’d give my right arm for a time-lapse network analysis of this, truly). I grew out my dang hair even, which is a big deal! I had that buzzcut longer than I’d lived most places. And, weirdly, I’ve felt more like myself than I can recall for a very long time.

But it’s been uncomfortable. It’s been hard. The biggest, most joyful and breath-taking breakthroughs also brought on periods of grief and despair—grief over the loss of a perceived self, grief over feelings of lost time spent travelling down paths that led away from the core self, despair over how long it takes to find a way back again. I spent a lot of time pondering this visualization of artist block and reminding myself that I am improving all the time, it just doesn’t feel like it when I’m also increasing my capacity to see what else I could be doing.

And even now that I’m acclimating to the emotional waves of change and not getting knocked about quite so much, it’s still awkward. Ungainly. I don’t get to be coolly possessed and confident. My intuition will run ahead of my conscious understanding so I’ll end up feeling strongly about something without being able to articulate why, and then having to backtrack later and explain once I’ve figured it out. Or I’ll find myself entangled in the extinction burst of a past habit and at cross-purposes with myself. A moment of insight and clarity will take the impossible knot I’ve been wrestling with and dissolve it into irrelevance, and I’ll find myself wondering how I managed to waste so much time on something that was resolved so easily. I spend a lot of time feeling incompetent, then realizing that I’m way more competent than I think, and then feeling incompetent for thinking I’m incompetent. Argh! I feel inconsistent and inconstancy still reads as a character flaw or moral failing to me. (Another habit yet to be unpicked and re-stitched! Coherency is preferable to constancy, if nothing else.)

Change is just like that, whether it’s coming in drips, tides, or tsunamis. You don’t get to sail into it, smooth and suave, confident that you know exactly what you’re doing and how to do it. Because you don’t! That’s the point. If we knew what we were doing, it wouldn’t be change. But not knowing exactly what you’re doing but kind of maybe knowing what you’re supposed to be doing and possibly doing some of it wrong but going ahead doing it anyway and trying to pull it off with some level of grace and integrity and humility is kind of where it’s at. So thank goodness for having a really high tolerance for embarrassment (not a skill I’d planned to lean on so much in adulthood, oh well) and a good circle of folks to commiserate and celebrate with, awkwardness and all.

(This is an evaluation blog, or at least the blog of an evaluator on their professional evaluation website, so I always feel compelled to draw some kind of evaluation-specific connections or insights or lessons out of what I write about, but I feel like “transformation is awkward” speaks for itself, y’know? Let’s all go a little easier on ourselves.)

I tweeted a joke about making an adult colouring book with reflective questions on each page for evaluators and it found a VERY receptive audience, so….. I made a prototype! You can download the PDF here. I hope you all colour your way to some satisfying insights! Comment or tweet and tell me what you think, or show off your masterpieces for us all to enjoy. 😀

Shout-out to André Luiz (@andreluizgollo) for putting up some great repeating pattern icons on The Noun Project that I was able to turn into the artwork for this colouring book.