Distance Monitoring, Evaluation and Learning: Capturing the Learning Experiences of the most Marginalized Girls in Nepal during COVID-19

Local facilitators, who are young female members from the same communities sharing similar socio-cultural experiences as the girls, are at the forefront of Aarambha's teaching and learning (T/L) process. These trained female facilitators lead all community level T/L sessions with the girls. In the first year, Aarambha is working with 1,709 adolescent (10-19 years) girls through 83 Community Learning Centres (CLCs) across four rural municipalities. The girls come from one of the most disadvantaged ethnic communities of Nepal, and are marginalized in a number of ways (33% Muslims, 19% Dalit). More than 50% of the girls have never been to school and the rest have dropped out at either primary or secondary schools. All of these girls are married or are promised to be married. Furthermore, 2.5% of the girls were identified as having some form of functional limitation.

The girls used to gather every day in a group of 15-20 in pre-assigned CLCs to attend classes. However, in light of the COVID-19 pandemic, the Nepal Government declared a nationwide lockdown on March 24, 2020 and CLC activities were suspended. To avoid disruption to the learning and to continue to support the girls, the project piloted and then scaled up an alternative approach to education, prioritising distance teaching learning (DTL) through mobile technology. Mobile phone based DTL was the most feasible low-tech solution as internet penetration in the community was low or non-existent.

Initially, DTL Guidelines were developed, aimed at program implementing team, including the facilitators, to provide guidance on distance teaching and learning process, in the current context. This was supplemented with COVID-19 Safeguarding Guidance, using the GEC's Keeping in Contact with Girls and Safeguarding Guidance, an analysis of potential risks, a capacity development plan for frontline staffs and the revision of lesson plans. Preparation also included the development of reporting tools and a Quality Standard Checklist (QSC) for remote monitoring.

In line with the DTL Guidelines, a list of girls was developed, focusing on who could be reached for the DTL. Internal monitoring data and the preliminary findings from telephone tracking were used to enlist the girls with access to mobile phones either by themselves or their family members. For the girls who did not have direct access to a phone, the program plans to conduct T/L sessions in small group settings in its medium-term response phase, in line with government guidelines on safety. In the DTL sessions, the girls’ and their families' consent and willingness to actively participate was mandatory, as well as the availability of mobile phones. Before actually scaling up to all girls, the DTL approach was piloted with small group of girls to explore the possibility of distance method as an alternative to CLC in COVID-19 context. 

After the pilot, internal monitoring data of learning, attendance (in phone-based sessions) and safeguarding/protection were analysed to determine the feasibility of the approach. Phone-based interviews were also conducted from the girls, their husbands/in-laws and facilitators to complement the findings. This pilot assessment was key to adapting the technical guidance, revising sessions into micro-session plans and updating MEAL tools/templates, which analysed the weekly quantitative data on teaching and learning, but also explored the opinions and experiences of facilitators, girls and their families through interviews. Aarambha's teaching and learning follows a three-pronged strategy: experiential, peer learning and gradual release. Assessment is done in terms of both content of the curriculum as well as the girls' participation in the learning process.

In the ongoing DTL, girls are taught three sessions per week by the facilitators. On average, the call duration for a girl is 30 minutes per week. The facilitators assess the learning after the completion of each lesson. The basis for such assessment is girls' attainment across assignments, their level of engagement during the sessions and the facilitators' feedback. The facilitators also include the comparative learning and engagements of each girl during "normal" classes at the CLC and the phone calls.

A joint team of monitoring and safeguarding staff oversees the process by using the joint QSC, in close coordination with the program team, to ensure that basic quality standards and safety measures mandated by the DTL Guidelines are being met during the calls. In addition to the learning data, the team also collects qualitative data from the girls, facilitators and families to continuously capture their experiences and opinions, during remote monitoring. Findings from remote monitoring is shared and discussed during weekly team meetings and meetings with partners via online platforms. Urgent issues (such as those concerning safeguarding/protection) are raised immediately to designated person/s for required support and referral.

The project also provides girls and families with information around COVID-19 preventive measures, Community Feedback and Response Mechanisms, free telephone psychological counselling services and other relevant information during the calls. The joint MEAL and Safeguarding QSC captures an assessment of whether such information is being disseminated and how, and the regular remote monitoring assess it by directly talking with the facilitators.

The learning assessment grid and guidelines used by the facilitators are designed in a way that captures both the learning in terms of content as well as facilitators' reflections of the progress the girls are making in each lesson. Such mechanisms have helped the project identify girls' specific learning needs and support (e.g. revision sessions) and potential barriers. For instance, the girls who were not attending the CLCs due to baby-sitting responsibilities are now actively participating in the mobile-based learning. All data is disaggregated to assess the different learning experiences of girls and provide tailored support where possible.

To continue with the distance monitoring, evaluation and learning, the project is utilizing a monitoring framework that is designed to monitor delivery and quality of adapted approaches during COVID-19. The Framework also outlines the aspects of quality monitoring in each of the outputs through a gender and social inclusion (GESI) lens, safeguarding/protection and content delivery; and remote approaches of conducting such a quality monitoring. Beside regular DTL, project also engages with i) the girls, their families and communities with referral support on COVID-19 related services and other essential services, ii) local government in developing education contingency plans, and its implementation. The project tracks its progress towards these indicators each month through an Indicator Tracking Table. It reviews gaps, challenges and lessons learned both at internal review meetings with team and partners, and externally with the Fund Managers, External Evaluator and other GEC partners.