ONLINE FIELD GUIDE

PLANNING AN EVALUATION

How does one prepare for an evaluation, whether commissioning one or conducting one? Are we ready for an evaluation, and, if not, what can we do to assess whether we are on the right track? What do I need to do to prepare? Why is theory of change important in evaluation, and how do I identify and use theories of change in evaluation? What evaluation approach or design can I use, and how to I choose? This section provides resources to help answer these questions.

Is the Program or Initiative Ready for Evaluation?

This section features key documents on the importance of considering whether an organization is fit for evaluation, as well as on the practical aspects of actually assessing evaluability and on how to build an evaluative culture. Commissioners of evaluations and peacebuilding practitioners tasked with conducting evaluations may find these materials useful, as they help understand how the environment in which evaluation takes place affects the end result of evaluation.

Evaluability Assessments

Organizational Readiness

What are Key Preparation Steps? What do I Need to do to Prepare an Evaluation?

In order to prepare a sound evaluation, a series of crucial steps must be taken into account. Although ensuring high quality evaluation is linked to the early stages of analysis and design, following the steps and recommendations suggested in the materials referenced in this section will help stakeholders involved in planning and implementing high quality peacebuilding evaluations.

This section is intended mainly for (but is not limited to) peacebuilding practitioners with little or no evaluation experience, and for more seasoned evaluators who can benefit from a more structured approach to planning an evaluation.

Prepare for an Evaluation

What Options do I have if my Program/Initiative is not Evaluatable or we are not Ready?

The following are alternatives to formal evaluation that can be used if it is determined that a formal evaluation—whether formative or summative—is not appropriate or feasible.

Alternatives to Formal Evaluation

What Evaluation Approach Should I Use?

Evaluation approaches refer to the principles or framework guiding the design and implementation of an evaluation. This section provides resources on a select number of evaluation approaches that can be and are commonly used to evaluating peacebuilding initiatives.

Situational appropriateness is increasingly being seen as the best criterion for choosing approaches and methods. Commissioners of evaluations, together with evaluators, should decide what approach is appropriate for the kinds of evaluation questions being asked, the users’ needs, the nature of the intervention and the context in which the evaluation will be conducted, as well as the availability of resources (both financial and human).  In Designing for Results (Chapter 8), Church and Rogers provide guidance on how to decide what approach to use and summarize the pros and cons of each.

Resources on some key Evaluation Approached are listed below.

Most Significant Change

  • Davies, Rick, and Jess Dart. The ‘Most Significant Change’ (MSC) Technique: A Guide to Its Use. 2005.
    BEGINNER

    This practical guide walks users through a clear step-by-step process of how to implement MSC, and also provides insights on its history, and how it compares to other approaches. In addition to reference to further reading, the guide is complemented by samples of story collection formats, sample MSC stories, and other annexes that can be useful for practitioners looking into using this approach for the first time.

Developmental Evaluation

  • Gamble, Jamie A. A. A Development Evaluation Primer. Canada: The J.W. McConnell Family Foundation, 2008.
    BEGINNER
    INTERMEDIATE

    This ‘primer’ introduces the concept of developmental evaluation and provides tools to foster its use. The first part of the book discusses the basics of DE, as well as a series of myths around it, and highlights some of the conditions needed to assess if organizations are in an appropriate space to apply DE. The second part of the book focuses on how to apply developmental evaluation, and discusses the key features of a developmental evaluator, as well as tools, issues and challenges.

Empowerment Evaluation

Goal-Free Evaluation

Utilization Focused Evaluation (UFE)

Process Tracing

Outcome Mapping and Outcome Harvesting

Participatory Approaches

Despite the fact that other approaches can be conducted in a participatory manner, participatory approaches are based on the premise that the structured participation of stakeholders throughout the different stages of the evaluation and in the decision-making process is essential to the conduct of evaluations.  There is little guidance for use of these approaches in evaluation of peacebuilding; these resources provide guidance on participatory approaches in other fields and could be adapted and tested for peacebuilding.

Theory-Based Approaches to Evaluation

Case Studies

Real Time Evaluation

Additional Resources

This list of approaches is by no means exhaustive. The BetterEvaluation site provides a more comprehensive list of Evaluation Approaches, explaining their meanings and main features, and providing different degrees of details about each of them, as well as additional resources for further reference. The site currently has information on the following approaches:

Experimental/Quasi-Experimental Approaches

These approaches require a high level of technical expertise.  The resources below provide a good overview of the approach as well as broad guidance on how they are done, when and for what purpose they might be used, and challenges and ethical considerations—and can be useful for commissioners of evaluations and program teams engaging with evaluators using these designs.

What Resources can I Draw on to Identify or Clarify Theories of Change in Evaluation?

This section includes some critical documents that help understand the concept of theories of change and the implications of their use –or their absence thereof– in evaluation.  These resources can be used by program teams in the design phase, in review and reflection on programs and projects, as well as by evaluators in identifying theories of change and program logic during the evaluation process.

Understand the Concept of Theories of Change and the Implications of Their Use

How do I Determine that a Program is Effective and Addressing Drivers of Conflict?

This section provides resources that discuss criteria to evaluate interventions.

Criteria to Evaluate Interventions