Power and Participation: Relationships among Evaluator Identities, Evaluation Models, and Stakeholder Involvement

Abstract

Stakeholder involvement is widely acknowledged to be an important aspect of program evaluation (Mertens, 2007; Greene, 2005a; Brandon, 1998). However, limited work has been done to empirically study evaluators’ practices of stakeholder involvement and ways in which stakeholder involvement is affected or guided by various factors. As evaluators interact with and place value on the input of stakeholders, social, cultural, and historical backgrounds will always be infused into the context (Mertens & Wilson, 2012; MacNeil, 2005). The field of evaluation has done little to critically examine how such contexts impact evaluators’ perceptions of stakeholders and their involvement. The present study attempts to fill these gaps, focusing specifically on the relationships among evaluator identities and characteristics, evaluation models, and stakeholder involvement. Using the frameworks of critical evaluation theory (Freeman & Vasconcelos, 2010) and a theory of capital (Bourdieu, 1986), the present study utilized a sequential explanatory mixed methods approach. A sample of 272 practicing program evaluators from the United States and Canada provided quantitative survey data, while a sample of nine evaluators provided focus group and interview data. Regression analyses and thematic content analyses were conducted. Findings from the quantitative strand included relationships between: (1) measures of individualism-collectivism and stakeholder involvement outcomes, (2) contextual evaluation variables and stakeholder involvement outcomes, (3) use of use, values or social justice branch evaluation models and stakeholder involvement outcomes, and (4) whether the evaluator identified as a person of color and the diversity of involved stakeholders. Findings from the qualitative strand demonstrated the role of dominant frameworks of evaluation serving to perpetuate systems of power. Participating evaluators revealed ways in which they feel and experience systems of power acting on them, including participation in, recognition of, and responses to oppression. The qualitative strand showed that evaluation models may be used to help recognize power dynamics, but that they are also used to reinforce existing power dynamics. Implications and recommended directions for future research are discussed.