What is Research on Evaluation (RoE)?

The RoE TIG convened a working group to help promote the
field of RoE, and one of our first tasks was to come to a consensus on the
definition of RoE.

There are many definitions of research on evaluation (RoE), which
can make it difficult to understand. Through searching the literature and
discussing with colleagues, we found four definitions (presented chronologically):

1. [Any] … systematic inquiry into the methods,
   practices, and profession of program evaluation, with potential implications of
   its findings for evaluation theory (Brandon & Fukunaga, 2013).
2. Systematic empirical inquiry resulting in
   original findings or reexaminations of existing data about the practice,
   methods, or profession of program evaluation (Brandon, 2015).
3. Any purposeful, systematic, empirical inquiry
   intended to test knowledge, contribute to existing knowledge, or generate new
   knowledge related to some aspect of evaluation processes or products, or
   evaluation theories, methods, or practices (Coryn et al., 2015).
4. A research investigation that generates findings
   with the intended purpose of creating a stronger evidence base and
   infrastructure for the applied practice of evaluation (Fierro, working
   definition).

The first three definitions have a lot in common: they are
focused on systematic inquiry using empirical methods aimed at
examining a variety of aspects of evaluation, including the methods,
practices and processes, products, theories, and profession.

The Brandon and Fukunaga (2013) definition also discusses
implications for evaluation theory and the Coryn et al. (2015) definition
discusses implications of testing knowledge, contributing to existing
knowledge, or generating new knowledge. Essentially, these
definitions point to RoE as research for research sake.

But the Fierro working definition suggests that RoE is for
more than just creating a stronger evidence base, one that seems built by and
for researchers. Rather, RoE is also for building an “infrastructure for
the applied practice of evaluation” (emphasis added).

It is for that reason we adopted the Coryn et al. (2015)
definition in conjunction with the Fierro working definition. The definition
might look something like this:

Research on Evaluation (RoE) is any
purposeful, systematic, empirical inquiry intended to create a stronger
evidence base and infrastructure for the applied practice of evaluation.

We believe this broader definition both (a) encompasses
everything the Brandon and Fukunaga (2013), Brandon (2015), and Coryn et al.
(2015) definitions propose and (b) emphasize the importance of RoE for
researchers and evaluators alike. RoE is not conducted for the sake of conducting
it, nor is an evidence base of research important unless it is useful
and used by the intended audience—in this case, practicing evaluators.

We hope this definition encourages researchers of evaluation to pursue topics that not only create a stronger evidence base but also informs evaluation practice.

Special thanks to everyone in the working group who has been involved in this so far: Kathleen Doll, Michael Harnar, Nina Potter, Eric Barela, Gregory Greenman, Esther Nolton, Ramjee, Miriam Jacobson, George Harrison, Seema Majato, Leslie Fierro, and more!