Outline India leverages SurveyCTO’s CATI features to assess 15,000 respondents

Outline India took their data collection practices to the next level in an endline assessment, leveraging SurveyCTO’s features for phone surveys

At Outline India, our mission is to empower stakeholders in making data-driven decisions. We are a research and monitoring & evaluation (M&E) consultancy firm, undertaking extensive data collection specializing and conducting Impact Assessments and Evaluations at the grassroots level. We deploy our projects after rigorous methodology, with quality checks at each stage, to ensure that our data is robust, relevant, reliable, and accountable towards achieving social impact. We focus on catalyzing the work of academics, think tanks, corporate social responsibility arms, not-for-profits, funders, philanthropists, and the Government of India.

In the last ten years, we have worked across all 29 states and union territories of India on 200+ evaluations (out of which over 100 were impact assessments) in over 60,000 villages, representing 5 million stakeholders from diverse backgrounds, settings, and sectors. Our focus revolves around the aim to include their needs, opinions, and views to draft and inform the work of our partners and policymakers to drive social impact. 

Our work ranges from delving deep into research designs, to the nuances of cognitive testing, piloting survey tools, data analysis, and report writing. We work in close association with our field teams who come from the informal sector. Our researchers, too, come from diverse subject areas such as Economics, Statistics, Sociology, Law, Psychology, and English to propose a holistic understanding while choosing various research methods. A majority of our work is cross-sectoral and our research spans across Rural Development, Agriculture, WASH, Livelihoods, Energy, Healthcare, Sexual & Reproductive Health, Child Rights, Education, Digital Literacy, Capacity Building, and Governance, among others.

The major challenge: An endline assessment of 15,000 surveys during the COVID-19 pandemics 

In December 2022, Outline India conducted an endline assessment via phone surveys for a premier university based in Illinois, United States to understand the value of electricity and consumers’ willingness to pay. This was studied during the COVID-19 pandemic, when the financial stressors on electricity distribution companies were at high levels. This survey was completed in the two districts of Madhya Pradesh covering a representative sample of 3,000 individual respondents.

The goals were to understand the increasing revenue recovery of electricity distribution companies, and measure the extent of consumers’ inclination to pay their bills to access the service. The funding agency also requested real-time updates of the data collection activity which included the total calls made and surveys completed. While the deliverable quantitative survey was for 3,000 respondents, we received a list of 15,000 – five times higher than the required sample size. This was done due to the higher attrition rate that is quite common in surveys conducted over phone.

The challenges faced in conducting the telephone survey on 15,000 data-sets while also ensuring that they were previously part of the same study and were registered as valid electricity bill holders, needed to be tackled seamlessly. Before using SurveyCTO’s phone-call field plug-in, case management, and pulldata () function, Outline India used a standard data collection procedure which saw us sharing phone numbers and a hard copy of the tool with the field team.

Once the data collection exercise began, we saw that keeping tabs on the changes in the present tool and the exhaustive list of phone numbers were quite inconvenient. The real-time tracking of attempted calls while also collecting data and maintaining a continuous focus on gender distribution also proved to be inefficient.

Using case management with the phone-call field plug-in and the pulldata() function, Outline India successfully managed the high volume of subjects

Outline India had four strategies for executing the data collection using SurveyCTO’s features.

1. The case management feature was utilized to handle and automate workflows for the distribution of the list of respondent phone numbers to each field enumerator, which was created considering the gender and geographical area distribution as per the funder’s request. 
2. Outline India used the phone-call field plug-in to make the calls directly from SurveyCTO Collect, the mobile application.
3. The pulldata() function of SurveyCTO was used extensively. In this, once the unique ID of the electricity bill holder was entered, the number was displayed in the application followed by the pulled data which showed further details like the name and age of the respondent.
4. Outline India created a dashboard for real-time updates regarding data collection. It was essential not only for the funders but also for the field coordinating team to know the up-to-the minute details of field exercises and avoid miscommunication and complications.

Assessment results

Case management implementation became accessible and seamless after using SurveyCTO.  

Firstly, each household head was categorized into geographical areas and gender, then a case management workflow was created and assigned to each field enumerator. Household heads were assigned a unique identifier, a label for each visible to only the enumerator, and then the form was filled out.

Within this case management workflow, once the enumerator opened a particular case for the interview, the phone number of the household head was available for the enumerator to directly call from SurveyCTO Collect, since it was embedded in the form using the phone call field plug-in. This feature also required the pulldata() function which extracted the phone number from the respondent list. The pulldata() function was not only used here, but also multiple times across the course of the study to extract names, phone numbers, sex, electricity bill holder ID, and demographic details.

Lastly, Outline India created a real-time monitoring system to display the data collected through SurveyCTO. It was very useful in tracking the progress of the project and helped in making the adjustment as per the requirement from the funder’s request. The dataset needed to have the proper distribution of geography and gender, and be maintained throughout the process of data collection

“Our experience using SurveyCTO for this project and others has helped us build trust with the platform. The features assist us in better planning and executing the exercises and maintaining an overall quality assurance and data management. We have multiple opportunities to use the tool in the coming year and we look forward to maintaining a fruitful partnership.”

– Shreya Sehgal, Outline India Sr. Partnerships and Communications Associate.

Lessons from the assessment 

There are many valuable lessons after using SurveyCTO. Here are some of them:

• A CAPI-first platform like SurveyCTO helps in minimizing mistakes and in the overall streamlining of the data collection via various distribution channels as per the requirement of the client. 
• It reduces bias in data collection.
• SurveyCTO can track and records the status of each call in a phone survey (call dropped, not available, busy, etc) for simplified project management and reporting.