Recently, I had the pleasure of interviewing my colleague Faizan Diwan as part of the TechChange course “Technologies for Data Collection and Survey Design.” Prior to working at Dobility on SurveyCTO, he spent several years as a Research Manager for Innovations for Poverty Action (IPA) in Kenya, where he led multiple large-scale impact evaluations spanning finance, labor, agriculture, and public health, he managed the Busara Center for Behavioral Economics in Nairobi, and he helped lead IPA Kenya’s efforts to shift from paper-based to tablet-based surveys.
(Side note: I’ve been told that Faizan is something of a mini-celebrity in the world of SurveyCTO users. His insightful – and speedy – technical support has benefited countless projects around the world.)
The lightly edited interview is below.
Faizan, thanks for taking the time and agreeing to this interview. Tell us about your time at IPA and Busara Center for Behavioral Economics.
I did all of my M&E work in Kenya. Over a period of four years, I worked on a range of projects that spanned several development sectors – agriculture, public health, finance, labor – but the common theme tying them together, since these were IPA projects, is that they were all randomized control trials or field experiments aimed at testing specific hypotheses about poverty alleviation interventions.
“I think, in this day and age, using digital technology is not even a choice. One hundred percent of the time ICT tools should be used compared to paper.”
I also managed the Busara Center for Behavioral Economics where, instead of testing interventions in a field setting, we had a computer center where we would bring respondents in (20 at a time) and have them play behavioral games against each other. You can learn a lot about people’s cognitive biases and psychology that way.
Wow, this sounds pretty interesting. What kind of data did you collect?
One key field-based project I can focus on is the “Unconditional Cash Transfer” study where we were evaluating whether the model being used by the NGO GiveDirectly – where they simply give people cash, no strings attached – had a positive welfare impact. That is, if you grant people money, no strings attached (as opposed to giving them cash but requiring they do certain things to get that money, like send their children to school, or giving them in-kind donations, like livestock), does that improve their health, education, and financial outcomes? And does it reduce their stress and lead to better decision-making, to help them get out of the “poverty trap”? Or is the money wasted on things like alcohol and cigarettes and hence are such cash transfers a bad idea?
We targeted rural residents in a province in Kenya – our sample consisted of about 1500 households, 500 of which were randomly chosen to receive the cash transfers and another 1000 of which were “control” households and did not receive any cash transfers. We conducted a baseline study to gather demographic data about their household, health, and education as well as financial data about their consumption, income, and asset wealth.
We were collecting almost exclusively quantitative data – we actually had a very lengthy and complex survey where we collected household demographic data as well as information on their finances. We also had a psychology module to assess their mental health.
“The most fundamental issue of any data work you are doing is the quality of the data. Garbage in = garbage out.”
To measure stress levels we also collected saliva samples from the primary male and female of each household, which we then analyzed in a lab for cortisol levels, which are a good proxy measure for how stressed someone is.
When did you start using digital tools for data collection?
My project was one of the first two projects in our Kenya office to experiment with digital data collection (this was many years ago when the norm was still paper surveys). We started with a windows-based software (my surveyors used to walk around with laptops!) but as Android devices became more ubiquitous and Android-based platforms started coming out, we switched to SurveyCTO.
Was adoption of technology for your projects difficult? What do you think are the challenges of going digital?
Since we are talking about adopting tech, I assume we are talking about cases where projects were previously using paper and shifted to digital technology. Operating from that assumption, I can think of the following challenges:
- While I think using digital technology for M&E is infinitely better than using paper, when first adopting such technology, it can be hard to get staff on your project to have the same trust and faith in technology as you.
- Even though you will want to train your staff in the new tech, different people have different skills and it might be that an older staff member who thrived under the old system, despite a lot of effort on your part, is unable to adapt or do well under the new system. This means either they move down in standing in the team or perhaps even have to be let go. That is never easy, especially if the staff are good people who have been loyal to the organization.
- Adopting technology can also change the workflow and timelines of projects, and it takes preparation to adapt to that. For example, with paper-based surveys the work is often end-loaded. One can start a survey quickly – as soon as you have a paper version of the survey – but once the data comes in there is a lot of work involved in scrutinizing, cleaning, and data entry before you get your data. With digital data collection, you need to do a lot of work up front when designing your survey in electronic format. Often this can add 2-3 weeks more to the launch timeline compared to paper. However, once you have the survey in place, the end-work is basically zero. There is no need for scrutiny or data entry and minimal cleaning required. Overall, it saves time, but it requires more advanced preparation.
I think, in this day and age, using digital technology is not even a choice. One hundred percent of the time ICT tools should be used compared to paper. The most fundamental issue of any data work you are doing is the quality of the data. Garbage in = garbage out.
Data collected on paper is really terrible. And even data collected using ICT tools can be terrible. But I think it is impossible to collect good data without ICT tools, and that to me is really the only determinant that matters.
Thank you, Faizan, for your time. It is indeed interesting to learn from your experience.
Of course! Any time.
