How to Assess Evaluation Capacity

Take stock of your firm's evaluation capacity.

Evaluation capacity is the ability of individuals, agencies or organizations to perform effective evaluations. Components of evaluation capacity include the human resource capacity to execute an evaluation, appropriate evaluation methods and associated skills, a general understanding of the importance and process of evaluation, the selection and analysis of indicators, and efficient and reliable data collection skills and methods. Having the capacity to evaluate leads to better assessments, which lead to project effectiveness, efficiency, accountability and process transparency.

Instructions

1. Appoint an assessment facilitator. This person should be the one who is most knowledgeable about the necessary and needed evaluation activities and the personnel roles.

2. Select your target group. Determine who will need to undertake evaluations in your organization. These will be the people whose individual and collective evaluation capacity you will need to determine.

3. Establish evaluation capacity indicators. Determine which skills and dimensions of evaluation, as well as the amounts, are necessary for evaluation in your organization. Articulate realistic and measurable areas that you need information on to improve evaluation execution. For example, if surveys are important to your firm, then creating, executing and statistically analyzing them will be necessary. To assess this dimension of evaluation capacity, you need to learn how many of your staff have these skills and how adept they are with them.

4. Choose an assessment method or tool. A pre-made evaluation capacity checklist, such as the Evaluation Capacity Diagnostic Tool from BTW Information Change, is a quick and easy way to undertake the assessment. Otherwise, you will need to create your own assessment with various quantitative or qualitative approaches, such as empowerment evaluation, Getting to Outcomes (GTOs) or surveys based on your indicators.

5. Hire an external consultant if your organization has little to no experience with evaluation or has difficulty with self-assessments in terms of a bias toward inflated expectations. An outside expert can determine where your organization needs evaluation and the relevant skills needed to successfully execute them.

Tips & Warnings

• External evaluation facilitators can build resentment if they are not properly aligned with the culture of your organization.

References

• Open Thesis: "Community Capacity and Governance -- New Approaches to Development and Evaluation"; Cindy Banyai, PhD; 2010
• BTW Informing Change: Evaluation Capacity Diagnostic Tool
• American Journal of Evaluation: "Empowerment Evaluation Yesterday, Today, and Tomorrow"; David Fetterman and Abraham Wandersman; 2007

Resources

• Read this Article in Spanish
• American Evaluation Association