How to Perform Process Evaluations

Focus groups provide data for process evaluation.

According to Melanie Bliss and James Emshoff of Georgia State University, process evaluation uses empirical data to assess the delivery of programs. It verifies what the program is and whether it is being implemented as designed. Conducting a process evaluation will clarify the internal dynamics of how a program, organization or relationship operates (See Reference 2, p. 159).

Things You'll Need

• Target project or program
• Evaluation facilitator
• Evaluation participants
• Logic framework
• Evaluation tools
• Feedback mechanism

Instructions

1. Develop a logic model. A logic model connects activities to goals, clarifies intentions and becomes the structure on which the evaluation framework is built. Start by articulating the overall goal, impact or change in the world that the project should make. Establish the outcomes, changes in specific target groups intended. The basics of the project, such as the numerical outputs, activities and inputs should also be outlined (See Reference 4, p. 36).

2. Form evaluation questions. Evaluation questions help focus and provide structure to an evaluation, guide the evaluation planning process, facilitate decision making about evaluation methods to use and inform discussions about how evaluation results can be used to make improvements (See Reference 3). Three guiding questions in process evaluation are:

   1. What is the program intended to be?

   2. What is delivered, in reality?

   3. Where are the gaps between program design and delivery? (see Reference 1)

3. Decide on evaluation methodology. Qualitative research is highly appropriate for studying process because it requires detailed descriptions of how people engage with each other, the experience of process typically varies from person to person, it is fluid and dynamic and cannot be fairly summarized on a single rating scale at one point in time. In addition, participants' perceptions are a key process consideration (See Reference 2, p. 159).

4. Collect data. Qualitative data collection methods include interviews, focus groups, observations and documents. A questionnaire with open-ended questions is another approach to data collection in process evaluations. The evaluation facilitator works together with evaluation collaborators and participants to decide which of their determined evaluations questions are the most pressing and relevant for inclusion on the evaluation.

5. Analyze and report data. A narrative case study is one way to analyze and present process evaluation findings. In the case of larger studies, qualitative coding can be used to analyze data. Michael Quinn Patton calls this the logical process, where you create categories and insert the collected data where they most appropriately fit (See Reference 2, p. 468). Findings can be written in a report, presented orally or graphically, or incorporated into a training/planing session based on the needs of your organization.

Tips & Warnings

• Reconfirming the findings and analysis with evaluation participants before concluding the final report promotes its substantive validity.

• Be patient and prepared to be flexible in your evaluation design, unexpected findings are common in qualitative research.

• Process evaluations cannot be done without collaboration or participation of project managers and implementers.

• Qualitative data is subjective, and this must be understood as acceptable by all parties.

References

• "Workbook for Designing a Process Evaluation"; Melanie J. Bliss, M.A. James G. Emshoff, Ph.D.; 2002
• "Qualitative Research and Evaluation Methods 3rd Edition"; Michael Quinn Patton; 2002
• Center for Disease Control and Prevention: Evaluation Briefs; 2009
• JICA: Part II - JICA's Evaluation Methods

Resources

• American Evaluation Association