In June of 2011, when our Uwezo evaluation team was getting ready to go out into the field with our household survey, I marveled — or more accurately, panicked — at the sight of mountains of paper that we kept hauling into our conference room from the local copy center. We needed to verify, collate, and stamp each questionnaire, distribute each to its appropriate box, and transport them out to the rural villages, where we would conduct our studies. Later, those thousands of pages would be marked up, separated from their identifying cover sheets, and transported back to Nairobi for coding. A team of coders would take on the tedious task of inputting all of the responses into a databse. And fearing the possibility of coder error, we transported the thousands of pages of anonymous surveys back to the U.S., so we could check back on any suspicious data entries.
It was 2011, and it was clear that there had to be a better way. But I had just heard from a colleague about his horror stories with a tablet-based survey: the programming was faulty, and despite great cost and effort, the research project was completely unsuccessful. Despite my intrigue, I was easily persuaded that it was too soon to bring tablets into rural Kenyan villages and think that we could effectively field a large survey in a relatively short amount of lead time. So we used up quite a few trees and hours, but successfully completed our work the “old fashioned way.”
Still, researchers are clearly out there using tablet-based surveys for all types of social research, and Markus Goldstein of the World Bank offers two excellent posts (part I and part II) on the use of Computer-Assisted Personal Interviewing (CAPI). In it, he thoughtfully reflects on some of the tradeoffs I discuss above, while clearly coming down on the side of technological progress!
Now I must admit that while I was in Kenya this June, conducting more informational and informal conversations around the country, I found my trusty new ipad to be invaluable. The new version allows you to swap in a local SIM card, so it was possible to use Kenya’s quick, reliable, and very inexpensive mobile 3G network virtually everywhere. While speaking with volunteers in informal offices in rural villages, I could take notes, upload them to a cloud, take photos, and plot the best route home using google maps. And in terms of one issue that Goldstein does not discuss, I did not sense that my device was in the least bit distracting for those with whom I was speaking. (Distracting for me, perhaps, as there is always that temptation to check email…) I had feared on our last survey that the introduction of a high-tech device into people’s homes would make the survey enumeration even more artificial and distorted than the endeavor necessarily must be. I did not find that to be the case. I am quite certain that a glimmering tablet would not have impeded the quality of a household survey — and as Goldstein points out, it almost surely would have assisted in maintaining accurate skip logics (for instance, if you ask a question about whether someone attended school as a child, you won’t pig-headedly ask the follow up question, what is the highest grade you ever completed.)
In any case, I will be curious to learn more about the experiences of others as they leverage this technology to more efficiently and accurately study social processes in the developing world.
I have used iPads the past two years in collecting data in Malawi. Last year was much easier than this year, and I’m not quite sure to what I should attribute that (last year I was there, this year I wasn’t?).
I used a proprietary software: iSurvey. It is cheap: $89/month for unlimited questions and devices. You create the survey in a web browser (easy to use, but still could be improved). You install the free iSurvey app onto each iPad from the App Store. Then, you connect each iPad to the particular survey you’ve created using a special code. Survey results are uploaded to the cloud when the iPad is connected to the internet and you can download the results in a CSV or SPSS format from a web browser.
The challenge is that in Malawi (especially in the rural areas), internet can be difficult to come by. We used a dongle to connect a laptop to pay-as-you-go internet and then broadcasted a signal to wi-fi-only iPads. This worked great in 2011, but was reportedly terrible in 2012.
I encouraged a former student of mine to write up a field note about our having used the iPads in 2011. I believe he is publishing the note in a forthcoming newsletter of the Comparative Democratization section of APSA:
http://people.tamu.edu/~cwimpy/goingpaperless.pdf
Thanks Kim — Very interesting, I’ll have to check this out.