About
The use of low-tech SMS messaging for education has grown in recent years, particularly as a result of the Covid-19 pandemic. SMS can be cost-effective, while personalised learning can be beneficial for girls. However, the empirical evidence base for such interventions is extremely limited, and further research is needed to understand how these observations play out in practice and how they can be enhanced to further improve learning outcomes and programme design. This study was conducted to help answer those questions and used a rigorous, sequential mixed-methods research design. This work was undertaken in partnership with M-Shule, a Kenya-based SMS-based personalised learning platform.
Key Findings
- The study found that learners had significantly better learning outcomes as a result of the intervention. The ASER literacy scale was used to measure learning outcomes in the Tusome dataset.
- Assessments were carried out by telephone, due to Ongoing Covid-19 concerns at the time. At baseline, the assessments from learners in the treatment and control groups were not significantly different.
- At the endline, the treatment group had made statistically significant gains in the ASER assessment compared to the control group, who had not used the intervention, for a total effect of 0.181 Learning-Adjusted Years of Schooling (LAYS) per learner.
Bridging the Gap
The Challenge
While mobile learning has the potential to enhance equity and access to education, there is currently an evidence gap in terms of combining SMS and personalised learning. There is also an evidence gap in terms of gender equity more generally.
How this Work Aims to Address It
The project sought to tackle these evidence gaps and practical questions through analysing the use of M-Shule, particularly the tool’s impacts on:
- learning outcomes and cost effectiveness
- engagement and potential to enhance learning
- design implications.
By analysing and collecting this data, new recommendations and insights on actionable, scalable solutions for low-tech personalised learning support may be developed.
Objective
To improve understanding of the elements of personalised learning that can be implemented in a more sustainable and cost-effective manner through lower-cost modalities. This work aimed to understand the scalability of DPL in low-income contexts, and with a specific focus on improving girls’ education.
Research Questions
- How effective is this form of EdTech in improving learning outcomes?
- Do the experiences of girls and boys differ when taught through this form of EdTech, and why is this the case?
- How can messages be designed in order to further enhance girls’ learning outcomes and support caregivers?
- How cost-effective is this form of EdTech?
Study Design and Methodology
The project used a mixed-methods research design, drawing upon primary and secondary data to understand the research context in different ways.
- Secondary analysis of existing data (effect sizes) about learning outcomes for girls – focusing on different types of EdTech interventions. This showed that personalised learning is effective relative to other types and a good candidate for research.
- Analysis of M-Shule data collected during recent initiatives: the Keep Kenya Learning programme and adapting and delivering Tusome content via SMS. We looked at data about how learners interacted with the content and learning outcomes (ASER scale).
- A telephone survey about learners’ and caregivers’ perspectives on using M-Shule throughout. We also worked with M-Shule to design and implement new content.
- Endline ASER tests were then compared to baseline results.
Timeline of Activities
October – December 2021
Statistical analysis of content and existing data
January 2022
Write up findings and submit to ‘Learning at Scale’
March 2022 – July 2022
Survey and focus groups; write up of journal paper 1
The Importance of the Results
This study shows the potential to scaling DPL through low-cost messaging technologies. This has equity implications for girls and marginalised learners in particular. With a better understanding of when and in what contexts low-tech personalised learning does and doesn’t have potential, decision-makers can adapt and tailor potential interventions accordingly.
This research has been used to help identify salient and practical recommendations for government officials. As part of this process, several promising foundational learning priorities were supported by our evidence, with findings indicating that literacy, numeracy, and socio-emotional learning can be fostered through an SMS-based platform.
Implications for Policy and Practice
The policy implications of this study lie in supporting the role of messaging to encourage participation in education, alongside broader efforts to improve consistent engagement where economic and seasonal factors are key barriers. Simple and general uses of messaging, for example, in reminding parents of term dates, or practical information, can lead to improved attendance as much as more nuanced and tailored nudge messages, and the role and responsibility of formal education systems to provide that information in an accessible manner can be met through basic SMS messages.
The research has potential to play an influential role in policy formation. The project has generated evidence which supports several of the key issues and recommendations made in the recent ‘Inclusive, Effective And Resilient Digital Learning In Kenya: Presidential taskforce submission’ report. Areas which are policy priorities and/or identified in recommendations through the presidential report, and are supported by the research project and its findings, are:
- Foundational learning: Including literacy, numeracy and SEL. The project has generated evidence that these subjects can be delivered through an SMS-based platform. Learning gains were demonstrated in relation to literacy.
- Inclusion: Demonstrating that SMS can be effective is particularly important in the context of Kenya, where ICT infrastructure is not uniform and connectivity can be low, particularly in the Arid and Semi-Arid Lands. In areas where infrastructure is lacking, the use of mobile phones and SMS could be particularly valuable as a more accessible and sustainable way to provide support to learners, especially if they are out-of-school.
- The role of EdTech startups in ICT integration: The recommendations highlight that there is a vibrant EdTech startup ecosystem within Kenya, but there are missed opportunities at present in terms of the relationship with the government and policy. This project represents a key example of this and how bridging this gap could be beneficial.
Another important impact of the study was the improvements and adjustments M-Shule, our project’s implementing partner, has made based on the insights and feedback from the project. These changes include:
- Content Enhancement: M-Shule updated and enriched its educational content to align more closely with the curriculum and the specific needs of students in the target regions.
- User Experience Improvements: M-Shule streamlined the user interface and incorporated user-friendly features to make the platform more accessible and engaging for students and teachers.
- Customisation Features: M-Shule introduced new customisation features that allow teachers and parents to tailor the learning experience to the specific needs of their students, as suggested by the project’s findings.
Study Team
- Dr Katy Jordan, Principal Investigator
- Kalifa Damani, Co-Investigator
Key Partners
Video Gallery
