Photo Credit: Luca Galuzzi – www.galuzzi.it
The Negotiation/Mediation IMPACT (Initiative to Measure Peace and Conflict Outcomes) workshop series, facilitated by the United States Institute of Peace (USIP), is now moving into its second year. The aim of the IMPACT Initiative is to improve the evaluative practices in the peacebuilding field, the first series of workshops is focusing specifically on dispute resolution projects.
Through a collaborative process with partner organizations working in the dispute resolution space, IMPACT developed a general theoretical framework, accompanying indicators, and data collection tools for dispute resolution projects. The core idea behind the development of a general framework and tools was that dispute resolution projects, when you strip them down to the nuts and bolts, share very similar project logics.
A pre-post quantitative participant survey, a pre-post qualitative focus group guide, and a quantitative monitoring tool were developed to capture as many of the identified indicators as possible. All the IMPACT tools are administered to direct beneficiaries. Consequently, we are not currently able to assess outcomes on the community or higher-order levels – though there are discussions for developing a community survey as an addition to the current tool kit in the near future.
Late last year USIP conducted our first field test of the IMPACT toolkit with USIP’s Alliance of Libyan Facilitators (ALF).
What did we learn from our first field test with the Alliance of Libyan Facilitators?
Simplicity Should Beget Simplicity
To maximize adoption of the developed tools both within and outside of USIP, the development process focused on simplicity, generalizability and, above all, the need for tools to be short in length. At the same time, through these tools IMPACT is attempting to capture complex concepts such as identity, trust, and interaction.
As a general rule, simplicity should beget simplicity. So the more we can move away from complex theoretical concepts and towards behavioral indicators, the better the end product will be. Trust, for example, is widely studied but there still does not appear to be a consensus around the definition and, as a result, how it should be measured (see Capra, Lanier and Meer’s article for a useful review of the trust literature). However, it is important to note that behavioral questions present their own set of challenges; especially related to recall bias.
Diversity of Disputes
Quantitative measurement of dispute resolution processes is possible. However, we need to be aware of the diverse nature of dispute processes both within and across contexts. It is possible for one dispute resolution that addresses core issues to be significantly more important to a community’s long-term stability than multiple smaller ones. We have to formally capture these differences. Consider a hypothetical participant that was resolving several small disputes in the local neighbourhood prior to becoming a member of a dispute resolution training project. Post-training, the individual feels equipped to take on a highly complex and long-lasting dispute between two tribes competing for scarce resources. The participant works throughout the entire project cycle to resolve the dispute and finally manages to defuse the situation and mediate a written agreement. This highly positive outcome could be noted as a single dispute resolved, compared to the multiple small disputes resolved before the project started.
Through our monitoring efforts, the post-training activities will be obtained in a detailed fashion, but we will have significantly less information about the activities carried out before the project started. While project staff will always be aware of the extreme examples such as the one above, we might need additional indicators to formally address the non-incrementality of disputes for more subtle scenarios.
External Factors Will (Massively) Impact the Success of the Project
In the case of the ALF, most of the participants are unable to use their new skills due to the current security situation in Libya. Many are currently living abroad in Cairo, Amman, and Tunis. Does this mean that the project is unsuccessful as a whole? No. Although the majority have been unable to use their skills at home, we have seen promising signs of participants aiding in dispute resolution processes among Libyan refugees living abroad. However, everything equal, we are expecting the participants to face more barriers to successful dispute resolution abroad than in their own communities, likely leading to a decrease in key indicators from baseline to end-line. Again, if we are simply counting numbers pre and post, without taking the context into account, we will severely underestimate the effectiveness of the project.
As data accumulates, we will be able to compare and contrast projects in different contexts and start creating useful lessons learned about where and under what circumstances the IMPACT tools developed are most effective.
Beyond the ALF, numerous projects at the Institute have adopted the tools retrospectively to aid their monitoring efforts. Over the coming months, we expect additional projects to adopt the tools. Moreover, we have partner organizations implementing the toolkit in a wide range of contexts on their dispute resolution projects. The results of these tests will give us a unique opportunity to tailor and improve the tools into something truly valuable to the peacebuilding field. And while we are not able to assess ‘impact’ in the strict sense of the word, (see this article for an excellent study of the impact of dispute resolution training in Liberia, by Blattman, Hartman and Blair) correlational studies are (still) highly important and will give us strong indications of what drives success in this space.
Ruben Grangaard is a senior research analyst supporting the Learning and Evaluation Team at USIP. He joined USIP after a year and a half as an independent consultant and on-site research analyst with Mercy Corps.
Ruben has experience developing research designs, survey designs and conducting data analysis for a broad range of evaluations. Moreover, he has field research experience from India and Bangladesh. Among his research interests are economic development as a tool for peace and drivers of youth violence. His current focus is on how we can better measure ‘effectiveness’ of peacebuilding programs.
Ruben holds an M.A. in International and Development Economics from University of San Francisco and a B.A. in Economics from University of Queensland, Australia.
You must be logged in in order to leave a comment