allblogs

Jul 28 2021

Mind Maps For Complex Projects

A mind map is a powerful, simple organizing tool that features a lot of information on a single page. We use mind maps and other visual tools to communicate relationships between ideas, people, project components and topics with our clients and partners. We present a short introduction to how and why you might want to use mind maps in your work.

A mind map is a visual representation of a project that links actors and actions together. This article focuses on using mind maps, not creating them.

Below is an example from software provider Lucidspark (which is a tool for creating mindmaps) that illustrates what a mind map is and how it can connect ideas and actions together.

Visualizing our Relationships

A search of Google (or any other search engine) will find thousands of visual examples of what a mind map is. There are few ‘essentials’ for mind mapping, which is among its most attractive features.

A complex project is one where there are many things happening at the same time, at different time scales, and involving many interdependencies and actors. That’s just a way of saying: there’s a lot going on.

It doesn’t matter how you organize your mind map only that it is useful. That means it provides information that you can make decisions with and act on. If it does that, it’s a good mind map.

Once we have that in place, the mind map allows us to examine the relationships between the nodes and topics to assess their fit. The expression ‘getting on the same page‘ reflects an ability for all of those looking at the document to have a similar representation of what’s on that page. The mind map allows us to verify relationships between entities, determine gaps or inaccuracies, and explore different ways of connecting things together.

There may not be a single ‘master’ map that is better than all, but there may be a map that is more useful than others.

How to Use Mind Maps

When we use this with clients and partners on projects we aim for the following:

To examine the assumptions involved in the project
To explore alternative relationships of fit and see if the map is the best representation of what we are looking at
To guide the project and keep the topics on track
To provide a building block for connecting ideas together (in ways that weren’t done previously)
To map out a system of relationships
To use colour and visuals to surface feelings, thoughts, and other emotional aspects of the project that might influence the strategy or outcomes

By creating a visual representation we see the whole project and the relationship context. Relationships are the key in any complex system. When we visualize these relationships it allows us to see how the actions in one relationship can affect others. Visual maps can aid us in anticipating possible effects.

(Note: We don’t use the term ‘unintended effects’ – just effects because in complex systems it’s too hard to predict cause and effect with any degree of accuracy).

We continually refer back to our maps to help ensure we are accounting for all relationships of importance throughout the project and not just those that have the most amount of activity within them. It’s incredibly difficult to track this in our heads or in something like a spreadsheet or text document.

Mind maps can be great tools to visualize a lot of information and get you and your collaborators on the same page when it comes to understanding what is going on in a complex system and program. Try it out — they are easy to create and simple to use.

If you want to use this approach to visualizing your programs and using this to help you innovate, contact us – we’d love to hear from you.

The post Mind Maps For Complex Projects appeared first on Cense.

Jul 27 2021

Evaluation Roundup – July 2021

Welcome to our monthly roundup of new and noteworthy evaluation news and resources – here is the latest.

Have something you’d like to see here? Tweet us @EvalAcademy!

New and Noteworthy — Reads

Research vs. Evaluation

Ah, the age-old topic! This blog post by Viable Insights reflects on an activity with a group they were working with where they reflected on what evaluation is compared to research. In this post they outline five distinguishing features: 1) Underlying motivation, 2) Outcome use, 3) Generalizability, 4) Conclusive goals, and 5) Collaboration. Do you agree?

Evaluation Guru – A New Video Series

The Asian Development Bank’s (ADB) Independent Evaluation Department (IED) recently started a new video series called Evaluation Guru. The series illustrates, how to design, implement and manage evaluations. In the first episode, Nathan Subramaniam (Director of Independent Evaluation Sector projects) walks people through Evaluating Private Sector Operations in an 11-minute whiteboard video.

Systems Diagrams: A Practical Guide

Bob Williams recently released a book that outlines six approaches to system diagrams, including: 1) Rich Picturing, 2) Influence Diagrams, 3) Causal Diagrams, 4) Cynefin, 5) Viable System Model (VSM) and 6) Cultural-Historical Activity Theory (CHAT). The book provides an overview of each system diagram, their purpose, examples of what they look like, and steps on how to draw them.

Telling Your Story of Change

Tamarack Institute recently posted an article that discusses Solutions Storytelling. In this article, the author argues that the work non-profits do “carries enormous potential to be shared with a broader audience in the form of insights, and stories that offer concrete solution and reflection to many of the conversations our society is having right now” – enter Solutions Storytelling. In short, Solutions Storytelling is about bringing the same attention and rigor to stories about responses to problems as is often done with the problem themselves. As evaluators, Solutions Storytelling might offer a new approach for how to present our findings.

New and Noteworthy — Events

Tensions: A Dialogue

Organized by: Robert Wood Johnson Foundation (RWJF) & Equitable Evaluation Initiative (EEI)

Date: August 4, 2021

Moderator: Marcia Cone, Director of Practice Engagement & Evolution at Equitable Evaluation Initiative

CDC Applied Research and Evaluation Fellowship

Organized by: Centers for Disease Control and Prevention (CDC)

Application Deadline: August 18, 2021

Next-level logic models for your ATE proposal and beyond

Organized by: EvaluATE

Date: August 18, 2021

Presenter: Lyssa Wilson Becho

Working with values: values literacy for evaluation

Organized by: Australian Evaluation Society

Dates: August 20 & August 27, 2021

Facilitator: Keryn Hassall

Developmental Evaluation

Organized by: Clear Horizon Academy

Start Date: August 27, 2021

Jul 26 2021

Factors that promote use: A conceptual framework

This blog post is a modified segment of my dissertation, done under the supervision of Dr. Tiffany Berry at Claremont Graduate University. You can read the full dissertation on the Open Science Framework here. The rest of the blog posts in this series on my dissertation are linked below:

Factors that promote use: A conceptual framework
Defining evidence use
Overview of my dissertation study: sample, recruitment, & measures
Question 1: To what extent are interpersonal and research factors related to use?
Question 2: To what extent do interpersonal factors relate to use beyond research factors?
Question 3: How do researchers and evaluators differ in use, interpersonal factors, and research factors?

There has been a lot of research examining factors that promote evidence use in both research and evaluation. Multiple frameworks for categorizing the different factors that promote evidence use have been proposed in the past (Alkin & King, 2017; Nutley, Walter, & Davies, 2007; Palinkas et al., 2011). In particular, King and Alkin (2018) stress that use occurs when certain types of users interact with certain types of researchers, who conduct research in a certain way. These categorization schemes can be summarized into intrapersonal (e.g., researcher and practitioner factors), interpersonal, organization, and research factors, which I’ve hypothesized their relationship in the figure below.

Note: The paths from interpersonal and research factors to evidence use were the only ones I tested in my dissertation study.

In this conceptual framework, the researcher and practitioner factors lead to evidence, but they also lead to evidence through research factors (e.g., rigor, relevance, timeliness, credibility) and organization factors (e.g., organizational capacity and size), respectively. Furthermore, researchers and practitioners must also work together to promote evidence use through interpersonal factors (e.g., relationship quality, communication, commitment to use, stakeholder involvement, cooperative interdependence).

Below, I will briefly discuss some of the literature on these five factors and how they promote evidence use.

Organizational factors: Various researchers and evaluators have suggested organizational context affects evaluation use, including the age, development, and size or scope of the organization or program; institutional arrangements and autonomy within the organization; and community influences on the program (Alkin & King, 2017).

Research/evaluation factors: Much research has examined research rigor on promoting use, but ample evidence suggests that (a) researchers and practitioners differ on their definition of rigor and (b) rigor is perhaps necessary but not sufficient to promote use. Rather, the research or evaluation must also be relevant, trustworthy, useful, and accessible to promote use (Alkin & King, 2017; Tseng & Gamoran, 2017a).

Practitioner factors: These factors include the practitioners’ predispositions about research or evaluation, including their prior experience with evaluation (Taut & Brauns, 2003) and anxiety towards evaluation (Donaldson, Gooler, & Scriven, 2002), as well as their interpersonal skills and the personal factor. The personal factor is “the presence of an identifiable individual or group of people who personally care about the evaluation and the findings it generates” (Patton, 2008, p. 66). When such stakeholders are present, evaluations are more likely to be used. However, little research has empirically examined the personal factor’s effect on evaluations (Fleischer, 2014).

Researcher/evaluator factors: Similar factors for the practitioner are also necessary to see in the researcher or evaluator, including their prior experience working with practitioners and their interpersonal skills. Like the personal factor, the interpersonal factor—the presence of an evaluator who personally cares about promoting use—has received some attention in the literature on promoting use.

Interpersonal factors: There are a variety of interpersonal factors shown to promote use.
- Communication quality: First, partners need to communicate effectively, which involves clear, frequent, and wide discussions across a multitude of media (Fleischer & Christie, 2009; Henrick et al., 2017; Maloney, 2017; Nutley et al., 2007). Furthermore, researchers need to communicate recommendations or implications of the evidence produced to support practitioners’ decision-making (Cousins & Leithwood, 1986; Fleischer & Christie, 2009; Maloney, 2017; Masaki, Custer, Eskenazi, Stern, & Latourell, 2017; Nelson et al., 2009).
- Practitioner involvement: Evaluators have increasingly recognized the importance of stakeholder involvement to promote use (Alkin & King, 2017; Froncek & Rohmann, 2019; K. Johnson et al., 2009). However, there are a wide variety of theories and approaches to stakeholder involvement that differ in their approach, and research has yet to fully examine whether and how practitioner involvement promotes evidence use.
- Relationship quality: Ample research on collaborations have stressed the importance of relationships, identifying it as one of the core dimensions of an effective partnership and factor to promote use. Research on research-practice partnerships have thus far supported the importance of relationships in promoting use. However, much of this research treats relationships simply as trust, mutualism, and long-term commitment, which are important but insufficient for true relationships. Relationships also involve positive emotions such as liking one another, feeling close to each other, and feeling respect, dependability, warmth, and overall friendship (Berscheid, Snyder, & Omoto, 2004; Carmeli, Brueller, & Dutton, 2009; Dave et al., 2018; Marek, Brock, & Savla, 2015; Stephens, Heaphy, & Dutton, 2012). This allows partnerships to go beyond transactional relationships to communal relationships. Ample research from developmental, organizational, and social psychology support the importance of relationships in partnership work.

Although my dissertation only tested a few of these pathways in my study, my hope is this conceptual framework can help others examine other research questions in relation to evidence use. For example, to what extent do research factors partially mediate the relationship between researcher factors and evidence use? As another example, to what extent do both practitioner and researcher factors lead to interpersonal factors to promote use? These are important questions that will help the field better understand how to promote evidence use in research and evaluation studies.

Jul 21 2021

Knowing Thyself: Colleagues, Coaches and Consultants

Self-knowledge is a great asset, however the reason we often hire an external consultant is because we know a part of ourself so well that it becomes difficult to see other parts of who we are.

We obviously have a bias toward the role of external consultants because that’s what we do, but we also recognize that part of role that we are often asked to play can be filled by different people or sometimes the same person playing many of them.

We refer to three C’s: Colleagues, coaches, and consultants. These are not mutually exclusive from one another and can be separate or combined. In each case, however, the role is focused on specific tasks that we want to highlight in helping you identify what best serves the particular situation or problem set you face.

Colleagues

Perhaps the most neglected role of the three are colleagues (or peers). A colleague is someone who works within the same field, department, team or organization as you and can provide feedback and perspective on the work you do and your performance. Because they know the work, they can speak to technical issues very well. However, because they are close to the work (and maybe you) they may include the same biases and limits to perspective as you do, too.

Colleagues offer the following:

Deep knowledge of the situation, context, and circumstances of the work. They know what you know and experience what you experience. There is little need to spend extra time empathizing with your situation.
This familiarity allows for more quick-to-start and nuanced conversations about issues.
Familiarity also allows for great compassion: it’s easy to talk with your peers about issues that might seem too far removed from others.
This familiarity also brings the same ‘blind spots’ and potential prejudices around actions, behaviours, policies, and possible outcomes. This can limit what is ‘seen’ and what is proposed.

Coaches

When you think of the highest performing athletes do you think about their coach? They all have one.

Athletes know the value of coaches because it’s only through that blend of expertise, motivational energy, and shared commitment through feedback that they can improve. The same is often true for organizations and individuals.

A good coach provides:

Someone on your team who is committed to the same project as you. Their success is yours.
Expertise in performance and how it relates to your specific industry or context.
A more intimate relationship in that they are focused on you (or your organization) and thus is more sensitive to its ‘moods’ and rhythms than someone who is more distant.
A coach has a separate role from who they support so, unlike with colleagues, there is less confusion and more clarity about what specific benefits a coach can and will provide without the potential for it to be conflated.

Consultants

A consultant is like a coach in that they bring an outside perspective – even more so.

A consultant is someone who is freed from the specific demands associated with the role, outcomes, and processes associated with your work and offers expertise and a different perspective on what you do. Many times we are so engrossed or familiar with what we are doing that we are unable to see new possibilities or consider alternative pathways for working.

Consultants provide many distinct benefits:

A consultant may not have specific domain expertise as your organization.
Alternatively, a consultant may have deep expertise in the domain you work in, but less in specific processes. They might also possess both. In every case, their value is in separation from the day-to-day activities of your work. This provides a more dispassionate view of your work. (Note: we don’t believe it’s necessarily more objective, just different — we all have biases).
Consultants can provide tactical and strategic advice much like a coach but without the closeness to their client and the same investment in the work. This isn’t to say that consultants care any less for the success of their clients, rather they are not part of the team and thus are not affected by the same fears of failure or concerns as a coach is.
Because consultants work at a distance they also are more likely to be versed in methods, tools, knowledge and practice skills that allow them to work well with groups who are unfamiliar to them (yet are similar in need or content expertise). This allows them to be bridge-builders within and across organizations and networks more easily.

The post Knowing Thyself: Colleagues, Coaches and Consultants appeared first on Cense.