Myanmar Impact Toolkit: Monitoring and Evaluating Counter Hate Speech Initiatives

Social cohesion in a country as religiously, ethnically, and linguistically diverse as Myanmar continues to be challenged by historical legacies. One of the overt ways in which this challenge manifests is hate speech; a discriminatory expression of hatred towards people because they belong or are perceived to belong to a certain religion, ethnicity, minority group, or gender. In such a context, numerous organizations have been working to document and counter hate-speech. However, while efforts in this direction continue, it is simultaneously imperative to monitor and evaluate these efforts so as to improve their effectiveness. Recognizing this, 23 CSOs and local and international NGOs have created the Myanmar Impact Toolkit: Monitoring and Evaluating Counter Hate Speech Initiatives with support from Search for Common Ground and the Kingdom of Netherlands. While the tips, tools, and examples are specific to Myanmar, the recommendations outlined could be tailored for use outside of Myanmar as well.
The Toolkit makes its recommendations through the following 3 steps:
Design projects more effectively
Monitoring and Evaluation (M&E) should not be limited to the final impact analysis of the program; rather, M&E should be integrated into the process of design and implementation  to help maximize impact. To that end, the Toolkit recommends that an evaluative approach be integrated into the design process with step-based suggestions and specific tools:

• Understand the problem: Use the Problem Tree Tool; hate speech is the trunk of the tree, the causes of hate speech are the roots, and the effects of hate speech are the branches.
• Understand the context: Use the PESTLE (Political, Economic, Social, Technological, Legal and Environmental) Tool to determine key drivers, trends, forces of hate-speech, implications, stakeholders, and strategic response options.
• Understand the stakeholders: Use the Stakeholder Analysis Tool to identify stakeholders and their roles in counter hate-speech projects. This diagrammatic tool should include actors who are categorized as per their level of influence and interest in countering hate speech.
• Understanding capacities and what can be achieved: The SWOT analysis is useful in determining the strengths, weaknesses, opportunities, and threats of any actor. The PESTLE Tool can be used for reference. The next step is to determine the key strengths that will help achieve the target and external opportunities that will help overcome internal weaknesses.
• Developing practical ideas: The Dimensions of Change Tool allows an actor to determine the nature and examples of personal, relational, structural, and cultural changes that the actor/s intend/s. The KAB (knowledge, attitude and behaviour) Tool allows the actor to determine the changes they intend in the knowledge, attitude, and relation domains.  
• Deciding which tool is the best fit: The Decision-Making Tool allows the actor to determine which idea or approach fits with the on-ground requirements, and organizational mission and capacities.  
• Building the idea into a project: The Logical Framework helps crystallize the connections between objectives, inputs, activities, assumptions, outcomes, outputs, and goals of the project.

Create indicators and set realistic project targets
A good program design requires indicators that measure the extent of change in comparison to targets.

• Developing and designing indicators, baselines, and targets: The indicators and targets should be SMART (specific, measurable, achievable, realistic and time-bound.) The Indicator Quality Tool helps determine the reliability, feasibility, and utility of the indicators.
• Disaggregation: The data should be disaggregated according to social, economic, and demographic variables.  
• Indicators and logical framework: Indicators must be integrated into the logical framework. The media program indicators must account for the 3 R’s- Reach, Resonance, and Response of the outputs, outcomes and impact.

Implement monitoring and evaluation
The Toolkit places significant emphasis on creating a monitoring and evaluation plan before project implementation. However, this does not mean that the plan is not open to modifications in the process of project implementation. It is useful to determine the process of gathering information in advance, adhere with the process, and ensure that personal biases do not affect data collection.
The Toolkit makes the following suggestions with specific tools/recommendations for each:

• Developing a monitoring and evaluation plan: M&E plans must include impact, outcome, and output indicators; targets, disaggregation of targets, information methods and sources; timing and key dates, responsible person(s), and escalation points (that results in intervention by responsible persons).
• Approach: Participatory monitoring and evaluation integrates the stakeholders by making them a part of the design, data-collection, and analysis stages of the M&E process.
• Methods and sources of gathering information: These could include direct observation, interviews (KIIs, FGDs), community meetings, case studies, staff reports, critical event analysis, change matrix, most significant change, document reviews, tests, questionnaires, and surveys. A choice of the method and source must be made through a combination of the aforementioned KAB and SWOT tools in a pertinent manner.
• Documenting results: A time-based results table allows the stakeholders to be aware of the predetermined targets of the project while allowing the implementing actor to keep track of project progress.
• Analysing information: The Toolkit emphasizes the importance to intuitive thinking whilst analysing the project results, patterns, and trends.
• Disseminating results: On the basis of how the results of the project will be used, relevant people must be informed and suggestions for modifications must be sought.
• Learning from the project: While the entire process derives from constant learning, it is imperative to finally account for the lessons learned, implications, and modifications. In addition, an evaluation report may be a useful tool in critiquing a project.

The Toolkit provides a step-by-step guide in setting up a system that allows for better monitoring and evaluation of counter-hate-speech initiatives. In doing so, it also identifies useful tools that could be used in designing counter hate-speech initiatives.
To learn more about this Toolkit, please click here
To learn more about the work that Search Myanmar does, please click here.