Technology-based Schools Monitoring and Learning Outcomes:Evidence from Public Schools in Pakistan
Inayat Ullah, Booyuel Kim, and Hee-Seung Yang
2020
Working Paper No. 20-02
We examine the effect of a technology-based monitoring of public schools on teachers’ attendance and learning outcomes. Our identification is based on a large-scale monitoring program implemented in over 28,000 primary and secondary public schools in the Khyber Pakhtunkhwa (KP) province, Pakistan. We find ideal conditions for a natural experiment and apply difference-in-difference, event study and instrumental variable approaches to causally attribute changes in the learning outcomes of enrolled children to the monitoring program. We utilize seven rounds of a nationally representative annually conducted independent and systematically random survey called ASER-Pakistan. Our findings suggest that technology-based monitoring has increased teachers’ attendance by nearly 8 percentage points in the first year after the program. Despite a slight decrease in the second year after the intervention, the long-run effect of the program strongly persists with significant impact on learning outcomes of enrolled children. We find that enrolled children’s standardized Reading, Math and English test score in the monitored schools has improved significantly by 0.08, 0.09 and 0.10 standard deviations points respectively at the lower (0 5) grades. Using exogenous program effect (obtained through diff-in-diff) as an instrument, we use observed variation in teachers’ attendance to predict changes in the standardized test score of children. The 2SLS results are strikingly (nearly two-times) larger than fixed-effect OLS estimates. More specifically, on average, standardized Reading, Math and English test score in the monitored region has improved significantly by .33, .22 and .62 SD points respectively at the lower (0 5) grades. We also utilize the post-merger (of two regions) data to test the difference in observed outcomes when the monitoring program was extended to the comparison region. We also examine the program’s effect on enrollment and school participation. Our results are robust on a number of alternate specifications, sub-samples, and falsification tests.
We examine the effect of a technology-based monitoring of public schools on teachers’ attendance and learning outcomes. Our identification is based on a large-scale monitoring program implemented in over 28,000 primary and secondary public schools in the Khyber Pakhtunkhwa (KP) province, Pakistan. We find ideal conditions for a natural experiment and apply difference-in-difference, event study and instrumental variable approaches to causally attribute changes in the learning outcomes of enrolled children to the monitoring program. We utilize seven rounds of a nationally representative annually conducted independent and systematically random survey called ASER-Pakistan. Our findings suggest that technology-based monitoring has increased teachers’ attendance by nearly 8 percentage points in the first year after the program. Despite a slight decrease in the second year after the intervention, the long-run effect of the program strongly persists with significant impact on learning outcomes of enrolled children. We find that enrolled children’s standardized Reading, Math and English test score in the monitored schools has improved significantly by 0.08, 0.09 and 0.10 standard deviations points respectively at the lower (0 5) grades. Using exogenous program effect (obtained through diff-in-diff) as an instrument, we use observed variation in teachers’ attendance to predict changes in the standardized test score of children. The 2SLS results are strikingly (nearly two-times) larger than fixed-effect OLS estimates. More specifically, on average, standardized Reading, Math and English test score in the monitored region has improved significantly by .33, .22 and .62 SD points respectively at the lower (0 5) grades. We also utilize the post-merger (of two regions) data to test the difference in observed outcomes when the monitoring program was extended to the comparison region. We also examine the program’s effect on enrollment and school participation. Our results are robust on a number of alternate specifications, sub-samples, and falsification tests.