About
This study, conducted in partnership with Oppia and Nairo Bits, explores how children learn on an offline technology-supported personalised learning (TSPL) platform in Kenya, generating evidence on the role of facilitation, design and peer interaction to inform future scale-up and alignment with Teaching at the Right Level (TaRL).
Technology-supported personalised learning (TSPL) has the potential to improve student learning at scale and holds promise in low- and middle-income countries (LMICs). However, the pathways to impact are complex and mediated by a number of factors. In this paper, we use a factorial Randomised Controlled Trial (RCT) design to experimentally test the causal influence of three mediating factors – facilitation ratio, peer learning, and access to devices – in improving learning outcomes on a TSPL numeracy platform.
The programme was implemented with Grade 5 students in an after-school format in an informal settlement in Nairobi, Kenya. Facilitators implemented the programme after school for approximately one hour, twice a week, on Android phones for a total of nine weeks.
Key Findings
This study shows that peer learning and device sharing can be among the most effective aspects of digital personalised learning (DPL) and can lead not only to better learning outcomes but also to reduced costs, suggesting it may be a cost-effective and scalable model for DPL.
Oppia’s Teacher-supported digital personalised learning DPL tool has a statistically significant impact on primary level numeracy outcomes:
Numeracy improvement up to 1.0 SD
Impact of peer interaction 0.50 SD
Sample size of students 364
Bridging the Gap
The Challenge
While technology-supported personalised learning (TSPL) platforms show promise for improving foundational numeracy and literacy, there is limited evidence on how learning takes place within these platforms, particularly in offline, low-resource settings. Questions remain about the role of facilitation, design principles and social interactions in supporting effective learning. There is also a need to understand how TSPL could align with or support established approaches such as the Teaching at the Right Level (TaRL) methodology, especially in the context of scaling personalised learning affordably.
Why It Matters
To date, the majority of research on DPL has focused on high-income contexts, and the majority of existing DPL studies focus on “supplementary” DPL (ie, used outside of regular classroom instruction), which means there is a gap in understanding for how DPL operates within the realities of low- and lower-middle-income contexts.
How This Work is Aiming to Address It
This study seeks to understand how numeracy and literacy learning occurs on an offline TSPL platform delivered on low-cost mobile phones in an afterschool setting in Kenya. Using a design-based research methodology and a mixed-methods approach, the study will examine the roles of facilitation, design, and peer interaction in the learning process. It aims to generate rigorous evidence to inform the design of effective TSPL interventions and contribute to the development of a future large-scale causal inference study. The research also explores how learning on a TSPL platform could support the scale-up of the well-established TaRL methodology.
Objective
To investigate how three aspects of a teacher-supported DPL intervention affect early-grade numeracy outcomes in Kenya: the teacher-student ratio, the number of devices per child, and promotion of peer learning.
The Research Questions
- How does learning occur on a technology-supported personalised learning platform?
- Can technology-supported personalised learning platforms be used to implement/scale a Teaching at the Right Level approach?
Study Design and Methodology
This study is a randomised control trial testing pre and post learning outcomes with a numeracy application for mobile phones for 364 Grade 5 students in Kenya.
The factorial RCT design varied these three factors:
- the facilitator-to-student ratio (1:15 or 1:30);
- the number of devices per child (one device per child or one device for two children);
whether peer-learning was promoted or not.
Timeline of Activities
December, 2021
Inception phase ~ 2 months (November-December 2021)
April, 2022
Finalisation of instruments
May – September, 2022
Data Review Board (DRB) phase
The Importance of the Results
The study findings are important in demonstrating the broader impact and potential for scaling DPL when integrated within existing schooling structures, in this case through teacher support. The study shows that peer learning and device sharing can be among the most effective aspects of DPL, contributing not only to improved learning outcomes but also to lower costs. This points to a potentially cost-effective and scalable model for DPL. Further research is needed to confirm these findings, as the authors note that the “sample size is small and students are limited to one school… [and] the intervention is short.”
Implications for Policy and Practice
The policy implications of this study are most evident in the potential for public–private partnerships, particularly the collaborative roles played by the Oppia Foundation, smaller local NGOs based in Kenya, and the schools where the intervention was embedded. The study points to a promising opportunity for school systems to leverage the strengths of large multinational tech corporations in DPL. It also highlights how the implementation process surfaced potential challenges and how these were addressed in practice, which can be considered for future design and scaling of DPL initiatives.
Study Team
- Dr. Akanksha Bapna, Principal Investigator
- Dr. Christina Myers, Co-Principal Investigator
- Namrata Sharma, Research Officer
Previous contributors to this study:
- Jay Thakrar
Key Partners
