There is much to admire about South Africa’s relative progress and stability in the almost nineteen years since its first multiracial election, but its school system is truly an embarrassment. Despite the country’s upper-middle-income classification, many areas that were formerly designated as so-called “homelands” (i.e., the Transkei and Ciskei), resemble the poorest areas of many of Africa’s poorest countries.
I paste below a link to a wonderful but depressing story about the dismal state of Eastern Cape schools — sometimes referred to as the “mud schools” because several are made of mud. It highlights how the Legal Resources Centre (LRC) has been working to protect children’s rights to education, suing the government to provide appropriate resources. It should not be necessary to litigate for decent schools for the poorest kids, but at least that avenue is available and seems to have had some impact.
I head off tomorrow for a few weeks of meetings and research in Kenya. I’ll join my collaborators, Lily Tsai and Dan Posner, and our project manager, Jessica Grody to continue our work trying to understand the impact of the Uwezo initiative. We’ll get started in Nairobi, and from there we’ll do some traveling around the country.
In our research so far, we have examined whether the direct provision of information to parents – specifically, their own children’s literacy and numeracy test results as well as information about what parents can do to be more active citizens – actually leads them to higher levels of local involvement in education. Like many other scholars, we are trying to understand if and when the provision of information leads to greater government accountability in terms of the delivery of key services.
For good reason, a great deal of impact evaluation research these days is being carried out using experimental methods. And if the definition of an experiment requires that the analyst randomly assign treatment to subjects, then technically speaking, ours was not an experiment. But fortunately, Uwezo had decided for various reasons to carry out this portion of their initiative to a random sample of villages around the country. Thus, one could not make the claim that there was something systematically distinctive about the villages that received the intervention. In order to estimate the effects of these informational “bundles” we required a control group. So prior to beginning our study, we identified a set of villages that were highly similar to the ones that had been randomly selected by Uwezo. Then we conducted surveys, focus groups, and other research in both sets of villages. We will complete a draft of a scholarly paper on this study fairly soon, and we will be presenting our results at the late August meetings of the American Political Science Association.
As for our estimates of the effect of those “information bundles”…. Well, we didn’t find any substantial impact. But in fairness, we didn’t hold particularly high expectations for an effect. It was important to see if this aspect of the initiative could drive desired outcomes on its own. It can’t. And now, we will investigate whether some of the much larger information dissemination campaigns make a difference.
On this trip we won’t be carrying out any systematic research – just learning about what Uwezo has been doing and what they plan to do going forward. Hopefully, we will finalize some interesting proposals for research in the coming years.
Uwezo Uganda just released a short film on the dreams (of becoming president) and challenges (of going to school when burdened with responsibilities) of an 11-year-old boy named Kyosiga.
Since the goal of this NGO is to inspire parents to place a higher premium on education, the intuition seems just right: Drama is a potentially powerful tool for disseminating key ideas. Along these lines, social scientists like myself need to make peace with the idea that coldly presented facts about the value of education — for example, with evidence about income gains from schooling — are probably not particularly persuasive for most people. Those facts may be necessary for getting a full picture of reality, but in turn, most people seem to be convinced by stories. A dramatic film that makes the case for education through the drama of a single individual who can stand in for the “average child” sounds promising. Whether the intended audience will actually get to see the film, and how they will respond to it remains to be seen.
When you visit a small, crowded African classroom, often with holes in the roof, and no electricity, it doesn’t take much imagination to think that such conditions make it difficult to teach and to learn.
So I must admit that reading the Poverty Matters Blog post about new solar-powered classrooms that provide air-conditioning, lighting, and internet access really caught my attention. As they point out, rural electrification is about 25%, which really limits schools and communities in all sorts of ways. The prototype classrooms manufactured by Samsung are made of shipping containers and provide about 9 hours of power.
As I sit here thinking, these could simultaneously improve student attention, the quality of the materials they work with, reduce rates of teacher absenteeism (by making it more pleasant to BE in the classroom), and help to connect students to the world and a world of information resources. There is always good reason to be cautious about any technological fix as being a panacea or even a major solution, but this strikes me as a seriously exciting possibility.
Solar Classroom in Johannesburg (from Poverty Matters)Kirinyaga district school (courtesy Kelly Zhang)
Yesterday, Uwezo released the results of its 2011 assessment of the educational attainment of Tanzanian children — it’s a comprehensive study, which covered more than 70,000 households in a nationally representative sample.
2011 Uwezo Assessment
As in Kenya, the report shows a high degree of variation across the country… but the results are generally quite disappointing given the investments in and attention to educational development in recent years.
In Tanzania, large numbers of children are not acquiring very basic literacy and numeracy skills until very late in their education, and even then, many still lack those skills. As late as standard 7, the majority of school children cannot read a very basic story in English — one geared toward the level of a standard 2 learner.
The report is available here and from the Uwezo website.
Deon Filmer, one of the authors of the World Bank’s Making Schools Work book, blogs on the possible links between information, accountability and better education that are at the core of the Uwezo initiative that Posner, Tsai, and I are currently evaluating. In his post, he talks about what he is looking for in terms of a school for his own children in Washington, D.C., and relates this to broader theories of the link between information and accountability. And he summarizes some of the mixed evidence from extant research in Africa and South Asia.
Filmer argues that, the two key features for success seem to be, “(1) keeping information simple, and (2) making sure that the intended recipients understand it.” And that this is true at both the household and school levels.
The fieldwork from the first phase of our research should be complete in a few weeks, and we look forward to disseminating information about the extent to which these conclusions also apply in the context of the Uwezo initiative both in Kenya and in Tanzania.
Today, our four field coordinators are in place to observe the launch/release of the 2011 Uwezo learner assessment results in two Kenyan provinces. Uwezo’s report, available at the link above, provides very detailed information on learner attainment across Kenyan counties, and on a range of other relevant indicators, including student and teacher attendance. While there is a lot of variation within Kenya, the fact that in the context of free primary education, in many districts, half of children are out of school… is sobering.
And as I wrote about earlier, on most scores, Kenya is performing better than East African neighbors, Uganda and Tanzania.
In the weeks to come, we will assess the diffusion of and responses to the release of this information.
We continue to have a team of researchers working on an evaluation of the impact of the Uwezo initiative in Kenya. Last week, Uwezo released this report on numeracy and literacy across East Africa, based on the results of their 2010 assessments of student learning in Kenya, Uganda, and Tanzania. Several news outlets covered the release, including The East African, which highlighted the finding that performance is not tied to the quality of infrastructure but to the quality of teaching, including the amount of time teachers give to children. In my visit to a handful of rural schools in Kenya last month, in several of the classrooms I visited, the room was packed with students… but no teacher!
Uwezo Learning Assessment Report
But the report also points out that Kenya is performing best in the region.
While Uwezo’s study is intrinsically important for understanding development trends in the region, our research is quite distinct: We are focused on how citizens react to such information, and over the next few weeks, we will be studying attitudinal and behavioral responses to the release of the report and related campaigns.
According to the PovertyMattersBlog , the same crowdsourcing technology I posted about earlier — which allowed people in Kenya to track ethnic violence in 2008 — will now be used to monitor education and health service delivery in that country, and perhaps elsewhere in Africa. Or at least that’s the plan. According to the posting, one of Ushahidi’s partners is Twaweza — the same organization that has contracted my colleagues Dan Posner, Lily Tsai, and myself to study the Uwezo education initiative in Kenya (we are busy preparing for field research as I write…)
After months and months of proposal writing, revision, and planning, I was excited to learn over the past few days that Princeton approved the human subjects protocol and all the other bureaucratic details for an evaluation of the Uwezo initiative in East Africa. I described this project briefly in my initial post — but the main idea is that Uwezo will try to improve education outcomes in Kenya, Tanzania, and Uganda by providing various forms of information to citizens. First, they go to randomly selected households and administer a series of tests to children; second, they provide immediate results to the parents; third, they provide printed information for parents and for the school and community more generally about what they can do to improve education directly (e.g., read to kids) and through policy change and oversight; and fourth, they reinforce that information and put it into broader perspective through information campaigns that describe the results of the assessment. There are some other nuances that I’ll write about later, but that’s the gist.
In our work on this project, Dan Posner, Lily Tsai, and I will be trying to assess the degree to which information empowers citizens to effect meaningful changes with concrete benefits for the education of children. Much of the research will consider whether the information provided is actually “news” to parents and other citizens; whether such information changes people’s views on the services they are receiving; and whether it effects the likelihood of their taking any action. We’ll get started with extensive field research in Kenya this June.
But in the meantime, it’s worth discussing the highly relevant, recent World Bank study, Making Schools Work: New Evidence on Accountability Reforms.
This is an excellent review of a series of recent studies that have sought to improve education outcomes particularly in developing countries. What’s particularly important about the study is that starts with the premise that there is a huge amount of variation that is NOT due to differences in resources allocated. We certainly know in the United States that money doesn’t solve all of the problems of providing acceptable universal education.
They focus on a series of studies that involved randomized interventions in various countries, including efforts to effect teacher performance, and school management. But most relevant for our study is chapter two, which investigates a bunch of studies — conducted in India, Pakistan, and Liberia. So far, the results have been pretty mixed, and not terribly encouraging in the sense that information campaigns have not had overwhelming effects on citizen activity or educational outcomes. For example, Banerjee et al (2009) conducted a study in the state of Uttar Pradesh in India, and found that the provision of information about local governance and self assessment tools had only a slight effect on citizen awareness, and virtually none of participation, or learning outcomes, when compared with households from control villages. On the other hand, only a few studies have been conducted, they differ in several ways, and as the authors of the World Bank study conclude, this type of research is in its infancy, and we still know very little about whether and how such interventions can have a measurable effect. To our knowledge, no similar study has been conducted in East Africa. The idea that better information should help citizens to realize the promise of democratic governance is too important to discard based on just a few studies, and while we will let the results of our study speak for themselves, it will be extremely important to understand if and how new information influences citizen participation in development.
