Dissertation RQ3: How do researchers and evaluators differ in use, interpersonal factors, and research/evaluation factors?

This blog post is a modified segment of my dissertation, done under the supervision of Dr. Tiffany Berry at Claremont Graduate University. You can read the full dissertation on the Open Science Framework here. The rest of the blog posts in this series on my dissertation are linked below:

1. Factors that promote use: A conceptual framework
2. Defining evidence use
3. Overview of my dissertation study: sample, recruitment, & measures
4. Question 1: To what extent are interpersonal and research factors related to use?  
5. Question 2: To what extent do interpersonal factors relate to use beyond research factors?
6. Question 3: How do researchers and evaluators differ in use, interpersonal factors, and research factors? 

I was also interested in seeing how use, interpersonal factors, and research factors differed between researchers and evaluators. This is somewhat of an extension from one of my research studies examining how researchers and evaluators define evaluation and differentiate evaluation from research (if at all).

Participants who self-reported themselves as evaluators rated relationships, communication, cooperative interdependence, and commitment to use higher than participants who self-reported as researchers. There were no differences between the two groups on research factors or evidence use.

I also compared correlations between interpersonal and research factors with evidence use between self-reported evaluators and researchers. Correlations for interpersonal factors and instrumental and conceptual use were generally stronger for researchers than for evaluators. Relevance was a stronger correlate of instrumental and conceptual use for evaluators than for researchers, whereas rigor was a stronger correlate of instrumental and process use for researchers than for evaluators. However, it should be noted that z-scores comparing correlation strength were not statistically significant.

Overall, although evaluators rated interpersonal factors higher than evaluators and no differences in reported use, there were stronger correlations between interpersonal factors and use for researchers. These findings suggest that evaluators and researchers are achieving the same outcomes (i.e., the same levels of instrumental, conceptual, and process use) but get to those outcomes differently. For example, researchers may see greater impact of interpersonal factors and rigor compared to evaluators whereas evaluators may see greater impact of relevance and stakeholder involvement.