What is Outcome Mapping?
Measuring the results of programming for social change can be difficult, particularly at the behavioral level. Outcome mapping captures the casual connections between behavior and programming in order to support organizational learning. Here, we will use our experience with outcome mapping of The Team: Tanzania to discuss the benefits and challenges of this methodology, as well as best practices for communicating results and reporting to a wider audience.
SFCG’S Outcome Mapping of The Team in Tanzania
The evaluation of The Team in Tanzania, is an excellent example of why outcome mapping can be useful: SFCG wanted to know whether their media programming could have concrete effects on communities, both at the individual level and through community initiatives. In Tanzania, Dunstan Kishekya, John Mauremootoo, and Richard Smith (RDS Consulting Ltd) evaluated The Team to see whether programming had actualy inspired people to change their behavior.
Since 2008, SFCG has used The Team in several countries across Africa and Asia as a vehicle to tackle polarizing societal issues by stimulating learning in a persuasive, but non-confrontational manner. For The Team: Tanzania, the goal of The Team was to contribute to strengthening the implementation and enforcement of gender-sensitive legislation in Tanzania.
In SFCG-Tanzania’s outcome mapping of The Team, evaluators identified 64 different behavior outcomes linked to The Team in the three regions sampled. As expected with most media programs, The Team was not the sole factor in shaping behavior during the time period that it aired. However, through the mapping, evaluators showed what changed about gender dynamics, who changed their behavior, and how programming contributed.
Results of The Team: Tanzania
The Team was most effective in contributing to individual changes when combined with mobile cinema and focus group discussions that drew on local context. Some of these changes had an unexpected ripple effect in communities – domestic violence and schools attendance serve as two good examples of this.
In Kilwa and Tarime, The Team contributed to increased involvement of women in economic decision-making and reducing violence against women. After watching The Team, a group of men in Tarime advised a friend to stop beating his wife, and he followed their advice. There are several instances of men advising friends to stop practices of gender-based violence after watching episodes of The Team and participating in focus group discussions.
In Mvomero, The Team inspired a school principal to start a girl’s football team, which led to an almost 30% increase in female school attendance at Hembeti Secondary School. The girls’ team attracted higher enrollment in the district.
These types of change, because they are unexpected and difficult to identify, might have been overlooked in a different type of study. However, outcome mapping was able to show not only individual change, but also the effect these changes can have on larger communities when people’s actions shape their relationships and institutions. The mapping allowed SFCG to show how the community took ownership of principles surrounding gender equality, and the potential for sustainability.
Read the full evaluation of The Team in Tanzania here.
Lessons Learned from The Team Outcome Mapping
For a large-scale media project like The Team, outcome mapping proved useful in identifying those ways in which TV programming can affect communities and individuals. However, we also learned some lessons from the process of communicating those results.
1. Experiment with Outcome Mapping to fit your project
Not everyone reading your program evaluation will be familiar with the outcome mapping methodology and approach. It is important to explain and justify the use of this methodology to your audience in clear and simple terms. SFCG’s evaluation includes an explanation in the beginning of the document to this effect, in order to help readers understand the methodology.
At the same time, if you need to modify the basic approach or include other documents in order for your audience to learn about the results of your project, then do so. The Team’s evaluation included proto-outcomes on changes in attitudes and knowledge because it was important to the project, even though traditional outcome mapping does not include these.
2. Define terms
Outcome mapping gives new meaning to terms that are commonly used in other methodologies, so it is important to define these terms for the reader. For example, ‘outcomes’ traditionally have a broader definition than in outcome mapping, where they refer specifically and exclusively to changes in behavior. Be sure that in evaluations, you are being consistent with terminology throughout the document.
3. Use a clear and simple structure
While any report should have a clear structure, some datasets can be more challenging to simplify for the reader. The Team evaluation spoke to 64 outcomes and proto-outcomes. This was managed through clear organizational structure. Our assessment covered four areas for learning:
-Effectiveness of interventions
-Validity of the project’s Theory of Change
-Value for money
-Obstacles to achieving results
Conclusions and recommendations spoke to these four areas and used outcomes to illustrate the main arguments. In our case, the use of extensive annexes helped us to have a report that explained project results without over-reporting. All relevant partners, donors, and interested groups should be able to interpret programming results and achievements.
Concluding Thoughts
The outcome mapping methodology was designed to work for development projects and partners, to give them a tool for learning about that level of influence that is the most difficult to measure: behavioral change. SFCG’s experience with The Team shows that media can have long-term and sustainable effects on communities through television series, concerning behaviors towards women’s capabilities and potential. This methodology allowed us to document types of change that might have otherwise been overlooked, such as the unintended outcome of a 30% increase in girls’ enrollment in schools in Mvomero. When using this methodology, make sure that it supports learning with clearly defined terms, scope, learning objectives and recommendations, so that future programming can use both the intended and unintended outcomes of projects to evolve in supporting positive changes in communities.
Adrienne Lemon is Search for Common Ground’s Design, Monitoring and Evaluation Specialist for East and Southern Africa. She is also currently a PhD candidate in Sociology at Boston University, with her dissertation being on post-conflict reconstruction and political participation in Burundi. She has consulted with numerous organizations in the Great Lakes, on program development, and monitoring and evaluation. She has worked on projects addressing children’s issues, combating sexual violence, natural resource-driven conflicts and DDR.
You must be logged in in order to leave a comment