It was our pleasure to attend EvalFest 2018 organized by the ECOI (Evaluation Community of India) on development evaluation. It truly served as a great learning platform to share the best practices and evaluation approaches that can be replicated and conceptualized both in an Indian and global context.
Representatives from various international and regional organizations like OXFAM, 3ie, Campbell Foundation, CACP and others participated in this event to share their views on topics such as high quality evaluation, gender transformative evaluations, supporting the last mile of sustainable development data, and many others.
Rajib Nandi (Core member, ECOI and Research Fellow, Institute of Social Studies Trust) shared his thoughts on Evalfest and mentioned that EvalFest was conceptualized as a small step in the direction of promoting an evaluation culture in the country so that all stakeholders are convinced about the process and practice of evaluation and accept it as means to evidence-informed decision making and information dissemination.
Dobility India was a part of one of the panel sessions on Data, Evaluation, Communication and Transformation. The session was chaired by Dr. P K Anand (RIS) who stressed on the importance of quality of data in evaluation and social research. Mr. Maulik Chauhan, represented Dobility India and highlighted the challenges faced by researchers in ensuring the quality of data being collected from the field and shared his insights on how these challenges can be overcomed in a cost effective manner by incorporating digital tools in your evaluation project(s) at various levels.
“Maulik spoke about importance of capturing community voice for high-quality data in the development sector, and defined the characteristics of quality data as completeness, reliability, authenticity, affordability, usefulness and security.” – Rajib Nandi (Core Member, ECOI)
Ms. Leena Sushant (Breakthrough) described two case studies interventions – one in which the process of communication did not lead to any behavioural change and another which succeeded. Ms. Anna Shurmann (Independent Public Health Consultant) considered why sound evidence would not always lead to action and suggested that different decisions had specific information needs and these should be met by the evaluator.
Based on the panel discussion, few challenges that every researcher tries to address while implementing his/her evaluation project(s) are:
- Why data quality matters and how do you ensure it?
- Data helps one visualize as to what is happening in reality. Bad quality data, inadequate data or missing data gives a blurred vision of future growth prospects. It can lead to wrong decisions affecting the overall impact of the evaluation. In order to ensure the quality, every researcher needs to track the data collected at various levels and monitor it throughout the data collection cycle. One can incorporate digital tools to keep a track of data quality and for real time data monitoring.
- How can we ensure the authenticity of the community voice reaching us on which our policy is based?
- In order to check the authenticity of community voice, one needs to accommodate digital tools that help in auditing the data in real time even at field level while the data is being collected, setting constraints and relevancy conditions in the survey instrument, monitoring the incoming data, and flagging of the outliers through high frequency quality checks.
- How do we assure the data security of the personally identifiable information (PII) of the community members…possibly strangers… who have trusted us and shared this incredibly personal information with us?
- One needs to consider points like who has access to data, properly training the staff, choosing the right tools that ensure data security, methods to recover data in case of loss or physical damage of device used to collect data, securing data across various hardwares and softwares like server, computer, tablet/ mobile level data security, encrypting data through public private key, cold room computer support and so on.
“I enjoyed Maulik’s presentation at EvalFest 2018, explaining how Survey CTO solves the challenges of collecting and aggregating quality data. This meets an important need – collecting poor quality data is a drag on organizations’ time and efforts – and undermines trust in data and the use of data for decision making.” – Anna Shurmann (Independent Public Health Consultant)
Stressing that community voice is at the heart of any program evaluation, Maulik touched on the points above and shared his knowledge based on experiences working with various users of SurveyCTO. (SurveyCTO is an ODK-based data collection platform that has been used to collect millions of submissions in over 150 countries by organizations such as the World Bank, J-PAL South Asia, Innovations for Poverty Action, OXFAM, Clinton Health Access Initiative, and many others committed to rigorous data collection and analysis in rural economic development, agriculture, health, education, and other sectors.) He reviewed the cost comparison between digital, paper and digital-paper options, as well as highlighted how SurveyCTO facilitates properly designed survey instruments, GPS mapping, audio audits and data encryption to ensure enumerators collect the highest quality possible and keep it safe.
“The presentation was good and comprehensive. It was useful to hear about the processes which people can adhere to, to ensure safety of the data collected. Survey CTO platform as you are aware has been used by our third party evaluators and have been appreciative of its use.” – Dr. Leena Sushant (Director Research, Breakthrough)
Couldn’t attend the conference? Email Maulik at maulik@surveycto.com if you have questions about how this applies to your own projects.
