What is Developmental Evaluation? Cartoon Glossary

This is a series of posts providing quick of overviews of important topics in research and evaluation. Each post in this series will include at least 3 cartoons from my archives and at least 3 links to recommended resources. I only give quotes here and recommend that you follow the links below each quote for more detailed information.

Developmental evaluation supports innovation and development, especially in complex and dynamic situations.

Accountability evaluation, formative evaluation, and summative evaluation all depend on there being a program or model to evaluate. Developmental evaluation is relevant when the program or model is still being created and those involved are figuring out what they want to do and how to do it. Thus, developmental evaluation helps people developing new initiatives to get ongoing, real-time feedback about what is emerging and its implications for making a difference.

Evaluation Flashcards developed by Michael Quinn Patton for the Otto Bremer Trust

On a cold Minnesota morning in February, I met with them after two and a half years and said, “You folks have been a great group to work with, you’ve been open to feedback, you’ve made changes, you’ve really adapted, but now we’re moving into the summative period, where we have to decide if the model that has been developed works, and so you can’t make any more changes in the program, because if you keep changing, we can’t answer the question of did it work. It’s got to be stable, standardized, fixed, and that’s now the challenge. So change is done, next two and a half years, everybody
gets the same intervention and then we’ll follow up, and see what’s happened to participants, what they’re doing in their communities, what kind of differences they’re making, how they’re communities view [them].”

The director of their program looked at me and he said, “But we don’t want to stop changing the program.” I said, “No, I understand you’ve been really good about changing the program, but we’re now doing what’s called summative evaluation, and that
means you can’t keep changing the program, the formative piece is over. The Board has contracted me to do a summative evaluation to answer the question does it work? There are a lot of people watching what you’re doing. People want to know if they
should emulate this model. That means summative evaluation.” He said, “No, no, no, no, you don’t understand. We understand that we can’t keep the program the same, we need to keep changing the program because the world around us is changing.” Then he
looked at me, fairly hostilely, and he said, “Formative evaluation, summative evaluation, is that all you people have?”

Well, in truth, those have been the dominant paradigms, with an accountability version of summative evaluation, which is a lot of what IEG does. Quite taken back, I said, “Well, I suppose, if you really wanted to, you know, we’d have to renegotiate the contract, but you know if you really wanted to, we could try doing developmental evaluation.” And they said, “What’s that?” I said, “That’s where you keep developing and adapting.” And they said, “That’s what we want to do. How do we do that?” I said, “Well,
we’ll have to figure that out. I’ll get back to you on that.”

Summary of a Workshop High Impact Evaluations Exploring the Potential of Real-Time and Prospective Evaluations A Workshop Conducted by the Independent Evaluation Group. Washington, DC – January 27, 2010

Developmental Evaluation (DE) is an evaluation approach that can assist social innovators develop social change initiatives in complex or uncertain environments. DE originators liken their approach to the role of research & development in the private sector product development process because it facilitates real-time, or close to real-time, feedback to program staff thus facilitating a continuous development loop.

Michael Quinn Patton is careful to describe this approach as one choice that is responsive to context. This approach is not intended as the solution to every situation.

Developmental Evaluation – Better Evaluation