Stories of change from refugees and people living with disabilities that changed programming
Conference session: “Participatory Evaluation: Engaging Participants in the Evaluation Process”
TIG: Collaborative, Participatory and Empowerment Evaluation
AEA Conference 2023: The Power of Story
Friday 13th October 2.30 to 3.30 pm – Grand Ballroom 4
Authors: Soledad Muniz, InsightShare. Amanda Burrows and Melissa Bell, Opportunity International UK.
Between 2019 and 2023, InsightShare and Opportunity International UK (OI) came together to incorporate Participatory storytelling into the internal MEL strategy of OI. The programmatic team wanted to better listen and understand lived experiences and outcomes of financial inclusion programmes, particularly supporting people living with disabilities and their carers, as well as refugees and host communities in Uganda. After training the international and local staff, as well as community members with an active role in the programmes, local evaluation teams used Participatory Video combined with Most Significant Change (PV MSC) to support participatory storytelling in baseline and mid-term review for the refugee programme, and in mid-term review and endline for the programme targeting people living with disabilities and their carers.
Here you can find their stories:
RISE (Refugee programme):
Baseline: https://youtube.com/playlist?list=PLr4Zjc7sz5uXJPRMEHEQUPEwqgus9u0UZ
Mid-term review: https://youtube.com/playlist?list=PLr4Zjc7sz5uWxBu9kLe2zI9Ju_BEC0SjO
Financial Inclusion for People Living with Disabilities:
Mid-term review: https://youtube.com/playlist?list=PLr4Zjc7sz5uW5VJ8nsESkY0oopDOjuI4u
Endline: https://youtube.com/playlist?list=PLr4Zjc7sz5uWegMI15jGQpnIYqZyWV-ZQ
“The entire process is an empowering one for both the communities and the project team involved in it. The affected population had an opportunity to freely share their experiences and successes through drama, songs and storytelling. The refugee groups involved believe their voices were captured rightly and are excited to learn that appropriate programmes are being designed to address the challenges shared in the videos”
– Noah Ssempijja, Country Lead
The process of monitoring and evaluating generally serves two main purposes: to improve the delivery and impact of programmes, and to promote accountability by learning from past successes and mistakes. Participatory monitoring and evaluation recognises that to obtain information and learning that truly reflects programme impact, the people who have had first-hand experiences of the programme must play a central role in the process, define their own measures of success, and assess whether a programme responds appropriately to the real life aspirations of the community. Consequently, using participatory methodologies can promote a positive cycle of sharing, learning, reflecting, and transforming. “Participatory monitoring and evaluation involves the assessment of change through processes that involve many people or groups, each of whom is affecting or affected by the impacts being assessed. Negotiation leads to agreement on how progress should be measured and the findings acted upon. It is a challenging process for all concerned, as different stakeholders must examine their assumptions about what constitutes progress – and together deal with the contradictions and conflicts that emerge.” (Guijit, 1999).
Participatory Video (PV) is a set of techniques to involve a group or community in shaping and creating their own film. The idea behind this is that making a video is easy and accessible, and is a great way of bringing people together to explore issues, voice concerns or simply to be creative and tell stories. The process can be very empowering, enabling a group or community to take action to solve their own problems and also to communicate their needs and ideas to decision-makers and/ or other groups and communities.
It is not always easy to gauge and communicate what a programme has meant to the lives of those it was meant to reach. Those best placed to explore and convey these messages are the project participants themselves, who can speak first-hand about it. They can select relevant individuals to interview in their communities and decide how to self-represent themselves. As footage is collected through time and various spaces, all actors can reflect back in the community through screenings, where stakeholders are brought together to reflect and discuss. Following stringent informed consent procedures, these stories can then be used to communicate lessons or new ideas to new groups, other organisations or decision makers.
Key findings:
- Both programmes were able to identify key enablers of change and their contribution to individual and community level change with points of comparison. In the disability programme, comparison between two different locations. In the refugee programme, the comparison was longitudinal over time between baseline and midline of a diverse set of refugee groups and host community.
- The local evaluation teams, composed of local staff, partner staff and community members with a key role in the programme, were able to analyze the data collectively through participatory analysis exercises and design recommendations. That promoted real-time uptake of learning and transparency.
- Community screenings were crucial spaces to verify information in the stories, promote accountability to communities and listen to their suggestions on how to improve the programmes. The screenings are also a powerful way for key stakeholders to understand the challenges being faced.
- In the refugee programme, there are multiple languages involved due to the nature of the settlement, but videos, with minimal translation, were accessible to everyone attending the screening. This enabled all of those present to feedback on the videos and to learn from other groups and understand what they are experiencing.
- From an OI UK perspective, the exercise was a key mechanism for connecting with the communities, gaining insights into their experiences and ensuring that they felt heard. For Opportunity Bank Uganda, it helped emphasize the impact they are having on the communities and identify opportunities to respond to their needs.
- In the refugee programme, having a baseline was an effective way of gaining an understanding of the challenges faced in the settlements, as refugees were a new population for OI to be working with. The midline could then demonstrate the real impact the programme was having.
- Not only did the fieldwork and the videos amplify the voice of the participating community members, but the local evaluation team gained new skills to utilize within the community and helped them grow their confidence.
- Truly inclusive approach that enables the participation of everyone regardless of skill or ability due to its visual as well as the aural nature of the storytelling. It’s extremely easy to adapt for people with learning difficulties, hearing or visual impairments, unlike traditional interview processes.
- So much is learned about the community and challenges faced in the process, beyond those that are directly related to the project running but that still may have an impact on the lives of those involved and their ability to make changes. It really helps to contextualize the work that is being undertaken rather than looking at projects and impacts in isolation.
Guijt, I. (1999) Participatory monitoring and evaluation for natural resource management and research. Socio-economic Methodologies for Natural Resources Research. Chatham, UK: Natural Resources Institute.