Last week we discussed an emerging vision for accountability and how it would change who we are accountable to and how. This week we examine who organizations should be accountable to and how.
Accountability in Evaluation, How?
“An organization must want to be accountable. Although this may seem self-evident, it cannot be stressed enough. Ultimately, the degree to which an organization is committed to accountability will determine the quality of the accountability mechanisms it puts in place and of the reforms it undertakes to increase its accountability.”1
The One World Trust suggests that like accountability, traditional conceptions of stakeholders should be expanded. In this view, stakeholders are “individuals or groups that can affect or are affected by an organization’s policies and/or actions.” This definition is more inclusive than defining stakeholders simply as individuals or groups that can affect an organization’s policies and/or actions—and recognizes that those who are able to do so are often different than those affected by those actions or policies. A very ‘peacebuilding’ way of looking at stakeholders, don’t you think?
So there are internal and external stakeholders. It is then necessary to identify key stakeholders who significantly influence or are significantly influenced by the organization’s actions. Internal stakeholders are pretty obvious: staff, board members, trustees. External stakeholders include funders, supporters, beneficiaries, partners and peer organizations. Certainly the accountability relationship to each of these actors will take different shapes and forms, as will the information that is required for each of the actors to hold the organization to account.
To become more accountable to beneficiaries and other external stakeholders, an organization must address the power imbalance between the organization and the external stakeholders. This can be as simple as listening to beneficiaries. For example, a common practice in peacebuilding is to conduct a highly participatory conflict assessment so that local perceptions of the conflict are taken into account when designing programming – and it also increases local ownership of the project(s).
But what does accountability to external stakeholders in evaluation processes mean?
There are a range of participatory evaluative methodologies and tools that can be integrated into the evaluation design.
Hot Resource! M&E Tips: Conducting a Participatory Evaluation by USAID
But simply ensuring beneficiary participation in evaluation does not necessarily result in accountability to beneficiaries.
If we as agents of change are entrusted to act on behalf of or in the interest of local beneficiaries, then we need to be honest about our results. For evaluations to be honest we need to be asking challenging questions that are relevant and will improve future performance – not just questions we want to know the answers to because of internal factors such as the continuation/justification of funding or a particular type of intervention. Listening to beneficiaries gives you a better idea of the type of information they want and need to make informed decisions on your programming, and these can become part of the evaluation scope and objectives!
Hot Resource! Writing Terms of Reference for an Evaluation: A How-to Guide by the World Bank
The scope and objectives of the evaluation need to be thoroughly thought-out and, ideally, this should include key local partners and/or beneficiaries who can substantially contribute to the identification of priorities.
In evaluation reports we usually (or at least are supposed to) identify any biases that may affect the analysis. Similarly, it should also be clearly stated who is the intended audience and/or user of the evaluation. In this spirit, perhaps we should also reflect in the report on why the evaluation questions were chosen, and their intended learning objectives?
Even if an evaluation is being done primarily for the purpose of accountability (in all likelihood to the donor), there is still opportunity for learning and making these objectives explicit will help keep us honest about our results, learning and how we intend to use that learning. You might even consider writing an organizational response to the evaluation. How will the information contained within be used to improve future programming? Does the organization agree with the evaluator’s conclusions, why or why not?
Being honest about results also involves budgeting an appropriate amount of funds for the evaluation to help ensure that the chosen methodological approach and data collection tools can be utilized to their full extent. If the data we collect is flawed in some way, such as a high margin of error, then the data is not as useful as it could be in holding ourselves accountable to beneficiaries and learning how to improve our performance. Even designs that intend to maximize beneficiary accountability and learning can fail because of budgeting errors.
Accountability in evaluation should also involve the dissemination of results to external stakeholders. This can get tricky since the evaluation was designed and written for a particular audience (probably the donor), and what the donor wants to know may not necessarily be the same as other peer organizations or beneficiaries. So some re-framing might be in order—and this is where producing Key Findings documents for quick and easy consumption can be really useful.
Hot Resource! Designing for Results: Integrating Monitoring and Evaluation into Conflict Transformation Activities, Chapter 10 by Cheyanne Church and Mark M. Rogers
When it comes to disseminating evaluation results to beneficiaries, you might need to take into consideration factors such as literacy, education and language. Data visualization tricks can come in real handy here. And get creative! Evaluations shouldn’t be boring – these are, hopefully, exciting changes that are occurring in people’s lives!
As a field, peacebuilding is still figuring out what it means to hold ourselves accountable, to whom and how. Evaluation is critical component of the accountability equation, but just like peace, accountability is a journey – not a destination. As our evaluation practices advance, hopefully so too will the ways in which we hold ourselves accountable: to donors, to ourselves and to the people we are meant to serve.
Jonathan White is the Content Manager for the Learning Portal for DM&E for Peacebuilding at Search for Common Ground. Views expressed herein do not represent SFCG, the Learning Portal or its partners or affiliates.
- 1. Monica Glagescu, Lucy de Las Casas and Robert Lloyd, “Pathways to Accountability: The GAP Framework,” One World Trust, accessed 26 July 2012, http://oneworldtrust.org/publications/doc_view/210-pathways-to-accountability-the-gap-framework?tmpl=component&format=raw.
11/10/2012 at 3:37 PM
But what does accountability to external stakeholders in evaluation processes mean?
In many ways, I think that the answer to this question is grounded in the definition of stakeholders that you put forth: “individuals or groups that can affect or are affected by an organization’s policies and/or actions.” If we operate under the assumption that stakeholders are an integral component to the nature of the organization and that they in turn can affect or are affected by whether or no the organization is accountable, than it is therefore essential that they be brought into that process early on. I think that Patton1 speaks to this when he talks about the essential need to define and create evaluations with those that will be using it. What does accountability to external stakeholders in evaluation processes mean? We need to ask them. If they aren't involved in the process of defining the meaning than I would say it is irrelevant what sort of conclusions we come to on our own.
11/08/2012 at 11:18 AM
I appreciate the series of accountability posts you've written - it seems that the next logical step might be a discussion of ethics and accountability in evaluation. As you mention in this article, honesty about results is critical, and with external stakeholders involved, I imagine it can be a temptation in certain situations to be less-than-honest with results (or to create metrics that give the data that one wants to see) when the evaluation has a direct impact on funding of a project.