Successful Twaweza evaluators’ workshop

I am on my way home from the Twaweza evaluators’ conference in Dar es Salaam… literally on my way, posting this on the second leg of my air journey to JFK, having departed Dubai around 4am… which suggests that now even international air travel offers no respite from the distractions of the web. But I digress.

The meeting was really extraordinary in so many ways. Twaweza’s Learning, Monitoring, and Evaluation manager, Varja Liposvek, brought together several evaluation teams — including the “LPT” team (Lieberman, Dan Posner and Lily Tsai), the AIID team from University of Amsterdam (Chris Elbers and Jan Willem Gunning), and James Habyarimana, who represented his Georgetown-based team. And a new JPAL/IPA project was presented by Twaweza’s Youdi Schipper. The approximately 40 attendees included various managers from within the organization; and a host of researchers and development specialists, including from the World Bank, Innovations for Poverty Action, Oxfam, the International Budget Project, the Transparency and Accountability Initiative, and DFID.

Although the room contained many distinct perspectives, the conversation was unified around a willingness to rigorously question every proposition, including how to conceptualize and to measure the intervention and associated outcomes, and how to judge the quality of evidence. I have attended many similar events, in which the attendees also came from different perspectives, and this one was remarkable for the unified willingness to engage constructively.

The first day of the conference involved hearing from the various evaluation teams, including ours, and in all cases, the studies are not yet complete. LPT and AIID have actually been evaluating Twaweza’s work, in the case of LPT by focusing on their education initiative, Uwezo. The other projects have initiated RCT’s to test related propositions that will inform future work. It was an extremely useful discussion and rewarding for us to see the care and attention with which our client was listening to and keeping track of these studies and thinking about ways to incorporate the findings into their mission.

Along these lines, more extraordinary from my perspective, was the second day — in which Twaweza’s director, Rakesh Rajani, announced to us all — look, we remain devoted to Twaweza, we know that these evaluations are not complete and only evaluate parts of what we do… but we also know from these studies, and from our experience and intuition to date, that much of what we are doing is not having the impact we would like. So let’s not simply keep doing the same thing; let’s make some substantial adjustments to better position ourselves for success.

In this regard, Twaweza is in a uniquely favorable position as compared with most NGO’s. I don’t know all of the specifics, but Twaweza’s donors appear to have a pretty long and patient time horizon. Unlike the average NGO, which is constantly fighting for its own survival, and under severe pressure to demonstrate quick results, Twaweza can afford to admit they didn’t have all the right answers before they started.

Again, particularly gratifying from the perspective of development/evaluation researchers was the stated commitment (and I’ll report back to see if this happens) to incorporate evaluators to a much greater extent in both the theorizing and design of their work going forward. (That willingness was not part of the first stage…) We now all have a basis for thinking about what doesn’t work and some intuition about why, and hopefully, this will allow both Twaweza and the evaluators to make some better and more focused bets about what might have the desired impact on the key outcomes of citizen agency and service delivery.

I learned a great deal from the workshop. That said, I wouldn’t yet bet my home that Twaweza will achieve its goals or that we or any of the other evaluation teams will be able to detect any treatment effects from research that we might design going forward. The core mission of trying to increase citizen agency and improve development outcomes through information provision is a challenging one. But I think that Twaweza has already set a valuable example for researchers and development practitioners in terms of how to engage in a manner that sparks careful, creative, and critical thinking, incorporating necessary theoretical and practical concerns. I’m glad that 50 hours of flying and brief abandonment of my family was not in vain.

Sadly avoiding Nairobi this week…

I was scheduled to fly out to Nairobi this Saturday night to do 3 days of work, including some social science research methods training to the Uwezo staff, before heading to Dar es Salaam for a multi-day conference with the various teams that have been working on the Twaweza initiative. In light of the awful attack on the Westgate Mall, a place I have frequented several times, and the fact that it was not absolutely essential to be in Nairobi, we decided to bypass Kenya altogether. I’ll go “straight” (i.e., with two other layovers) to Tanzania instead. Notwithstanding the pleas from the Standard Newspaper against international travel advisories… my inbox piled up this morning with lots of them and frankly, I just couldn’t justify to myself or to my family a good reason for entering the fray in the midst of all that’s going on, especially since it would likely be impossible to be very productive. But I’m disappointed to not be going to Nairobi, and more so that Kenyans are taking it on the chin once again.

Meanwhile, I know it’s 2013, but I’m still dumbstruck by the ubiquitous nature of communications technology and social media during a terrorist attack. Seems as if many citizens at the scene, despite threats to life and body, have been using their smartphones to broadcast images from the mall. Shabaab and the Kenyan military have been exchanging public tweets about who has the upper hand in the standoff. CNN writes, “ Are you in Nairobi? Send us your images and experiences, but please stay safe.” Unfortunately, the spectacle and global stage of violence almost certainly makes it a more appealing strategy for the very groups who perpetrated these heinous acts.

P.S. I know I have been a delinquent blogger for quite some time. I really don’t know how these folks with full time jobs can manage to blog so frequently… but perhaps the Fall will bring greater inspiration and speedy fingers in generating blog-worthy posts. I am likely to tweet more frequently at @evlieb

Accountability as a contested concept

In the past week, I’ve read several sets of interesting commentaries on the notion of accountability, particularly with reference to foreign aid. Together, they highlight the extent to which “accountability” has become a term much like “democracy” — evocative, normatively attractive, but employed with so many different intentions that it’s almost impossible to know what it means.

Albert Van Zyl at the International Budget Project highlights the completion of a series of important case studies concerning how civil society organizations in poor countries “hold their governments to account for the use of public resources.” (One of those is a study of HakiElimu, an educational initiative in Tanzania, formerly led by Rakesh Rajani, who now directs Twaweza.) But in a broader discussion of accountability, Van Zyl  also points out that citizens in donor countries remain unclear about the size, role, and impact of foreign aid. Most readers will be familiar with the severe over-estimates that citizens in rich countries make concerning the size of the foreign aid budget as a share of total government spending. Along these lines, he writes,

Just as citizens in poor countries don’t want to be passive recipients of development, citizens in rich countries say that don’t want to be passive providers of development either. They can and want to understand much more about where there funds are going, how decisions were made, what the impact was and why.  Ultimately better communication to citizens in rich countries could lead to an exciting and dynamic new form of development with much tighter linkages between citizens in rich and poor countries.

Meanwhile, IRIN, the UN News and Analysis service, also recently published an article entitled, “Accountability – what’s in a word?” Their discussion of the diffuse understanding of the notion of accountability reminded me of David Collier and Steve Levitsky’s classic World Politics article, “Democracy with Adjectives.”

The concept of “accountability”, like much humanitarian vocabulary, can be complex and elusive. Some organizations, like Médecins Sans Frontières (MSF) do not even like using the word.
“We’re not very comfortable with the term ‘accountability’, because. we are still not very clear on what we mean by accountability,” said Caroline Abu-Sada, coordinator of MSF’s research unit in Geneva.
And if understanding its meaning in English can be a struggle, translating it into other languages can be near impossible.
For example, “In French, it’s absolutely untranslatable,” Abu-Sada told IRIN. The best she and her colleagues have come up with is “redevabilité” – “it’s an unpronounceable, very bizarre word. It’s not really user-friendly.”

IRIN goes on to document the wide variety of ways in which aid and development organizations have defined the term, a few of which I paste here:

“the responsible use of power.” (Humanitarian Accountability Partnership – HAP)

“first and foremost about communication with affected people.” (Jacobo Quintanilla, director of humanitarian information projects, Internews)
“about beneficiaries participating in the process of improving their situation.” (International Federation of the Red Cross and Red Crescent Societies)

“leadership/governance; transparency; feedback and complaints; participation; design, monitoring and evaluation.” (Inter-Agency Standing Committee Sub-Group on Accountability to Affected Populations)
“a shared commitment to learning as the path to excellence and to integrity in fulfilling commitments to stakeholders.” (World Vision International’s Accountability Framework)

… All of these are only loosely connected to the definitions provided by the World Bank in their landmark 2004 World Development Report, which contained an extended and thoughtful discussion of accountability.

So where does that leave us? At the very least, when the term is used, a precise definition is needed, because the term, to quote Giovanni Sartori, has been stretched to the point that the intrinsic connotation is not clear. Given the ubiquity of the term, I suspect further analysis would reveal that much scholarly and policy analysis has suffered from unrecognized confusion.

Monitoring and evaluation job opportunity with Twaweza, East Africa

I met with Rakesh Rajani, Head of Twaweza, yesterday in Nairobi. He is looking for a newLearning, Monitoring and Evaluation manager who would report to him. They want to hire a highly skilled individual with a minimum of an MA, and a PhD is preferred. Sounds like a very neat job (and our research team would be working closely with whomever he hires, so I have high hopes that he will identify someone great).

A full job description is attached here.

Twaweza responds to Ugandan police seizure of calendars

A few weeks ago, I wrote about the Ugandan police’s seizure of Twaweza-produced calendars promoting active citizenship. This week,  Twaweza’s Uganda country director spoke out about the calendar’s intended message and why he believes the police should release the calendars.

What reason did the police give to impound a consignment of Twaweza calendars?

They say the calendars contained messages which could incite the public. But we think this is a message that tells citizens to wake up to the realisation that they have to change their livelihoods not to wait for leaders, governments or NGOs to solve their problems. We believe empowering these people can help them make changes in their lives. We think this is a misunderstanding that the police should not have caused.

So do you don’t (sic.) believe what the police say about your calendars?

Absolutely not; we think the calendars are not inciting as they want you to believe. It is colossal misunderstanding on their part.  There are no political innuendos in these calendars.

The full interview can be found here

The country director argues in the interview that the calendars were not intended to be “partisan” or “political,” but rather to encourage citizens to be change-makers in their own lives. Fair enough. But given that there was absolutely no suggestion of inciting violence in these calendars, their confiscation is a clear denial of open and critical political discourse. Irrespective of the goals or intentions of the sponsoring organization, the Ugandan state’s actions contravened any pretense of upholding democratic norms.

Ugandan government impounds Twaweza citizenship calendars

Last Friday, the Ugandan government confiscated 700,000 calendars produced by Twaweza. As far as I know, they remain in police custody.

The calendar was an effort to realize Twaweza’s mission to promote government accountability. Phrases such as, “who will change your world in 2012?” and photographs of various government officials were intended to provide information and to increase citizen awareness.

According to an article in The Monitor, police have charged that the calendars incite violence. More stories and debate about this incident can be found here.

 

Twaweza offices flooded

I was sorry to hear that the Twaweza offices in Dar es Salaam were flooded last week — causing no injuries, but severe property damage. Twaweza is an organization I have been working with since the beginning of the year (on a project along with Dan Posner and Lily Tsai), and their main goal is to empower citizens in Africa to improve their lives, particularly in areas such as education, water provision, and health. From the update on their website, it sounds like good citizens did a great job of evacuating people and cleaning things up. But I wish them luck in getting things back in order to continue their important work early in 2012.