The Evaluation Mindset: Not Your Data

In 2018, 87 years after it was completed, Zora Neale Hurston’s Barracoon was released to the public.

Reading the book today the story of Cudjo Lewis feels completely relevant to our contemporary times. It is heartbreaking, illuminating, and honest.

In the way she writes, Hurston allows Cudjo to tell his own story. They are his words, told in his way. She is an active participant in the collection of those words, a supporting character in the telling, and a translator through her authorship.

And because of Hurston’s insistence that the book remain that way, and not adapted to meet the desires of publishers wanting the writing altered from dialect to language, the words were almost lost to time.

Poor Zora. An anthropologist, no less! A daughter of Eatonville, Florida where truth, what was real, what actually happened to somebody, mattered. 

Alice Walker’s blog post introducing Barracoon

Who owns the data?

A few years ago I had the honor of attending a presentation by Stafford Hood and Nicole Bowman. And something from the Q&A has been stuck in my head ever since.

For me, it’s a no brainer…If they are saying they don’t want that data published, it’s over.

Stafford Hood – 2016 EERS Eleanor Chelimsky Forum [YouTube recording! Thanks EERS!]

Data is too simple a word for what it represents. As if collections of characteristics, thoughts, feelings, and stories are not reflections of humanity and filled with complexity.

Of course simplifying something so personal allows it to become marketable. If you detach data from people, turn the observations into a series of ones and zeros few can understand, all of a sudden it’s a product that can be sold?

Disconnecting “data” from people doesn’t make it objective. It just hides the origin. Just like giving a cell a name like HeLa, and allowing it to be bought and sold, doesn’t mean it wasn’t lifted without permission from a person named Henrietta Lacks.

Putting Research before People

For long before you and I, science has been used as a justification for both ethical and unethical practices.

The human subjects protections we have today don’t exist because scientists and policymakers were forward thinking. They exist because of past atrocities and unethical studies that put the pursuits of science over the rights and well-being of people.

The men were never given adequate treatment for their disease. Even when penicillin became the drug of choice for syphilis in 1947, researchers did not offer it to the subjects. 

The CDC’s Page on the Tuskegee Syphilis Study

Today’s Data Challenge

There’s a good case for using smartphone data in the COVID-19 response, but Americans deserve an explanation.

Casey Newton for The Verge

Do you want to know how to make a member of an Institutional Review Board freak out? Just show them the amount of personal data available to your run-of-the-mill online marketer.

Want to know who clicked what, when, and where, well our digital world keeps a record. And the records can be compiled together to paint a bigger picture. Ultimately you can get a story of a person who likely had little idea they were ever being followed.

Most of the time the data being collected seems harmless enough. At least until it’s not.

Right now the same marketing data is being engaged in efforts to combat the spread of a pandemic. Does that make the social media industry’s collection, storage, and commercialization right? Does contact tracing justify China’s development of a surveillance state?

These are open questions, and not always easy ones to answer.

Being a data steward

Evidence has value, but it also comes with a price tag (societal cost). Our values, and the values of our organizations, guide our usage.

That is, if it’s on anyone’s radar.

As an evaluator it is often not your role to choose, but it is your role to guide choice. Who does data belong to? Where did it originate? How is it being used?

These are all questions we need to help answer. Because there is no guarantee that anyone else will.