Many SurveyCTO users design and deploy complex, survey forms daily for projects all over the world. With that in mind, Simrin Makhija, a Research Analyst at the International Food Policy Research Institute (IFPRI) – and SurveyCTO super-user – wrote the following blog to share pro tips, best practices, and even a part of a sample complex survey instrument based on a recent agricultural survey in Ghana and her years of field research experience.
Background
When working on evaluations of agricultural development projects, it is extremely important to get high quality data at the plot level of the farm. Since farmers in developing countries often have fragmented landholdings, data must be collected on multiple plots on which farmers might potentially grow different crops and employ different practices. As part of an evaluation of a training-of-trainers program on integrated soil fertility management, our team conducted two surveys in Ghana’s Volta region. In this context, this post shares some best practices for data collection using SurveyCTO based on our experiences in Ghana and other countries where we have been using SurveyCTO.
Start with paper
It is always useful to start with a paper questionnaire. Once you have coded (or programmed) a relatively final draft of your paper questionnaire in SurveyCTO, you can make modifications directly in SurveyCTO. Time permitting, it is advisable to make changes in the paper questionnaire simultaneously. But we all know that field work is demanding, and given competing demands in the field, this may not always be possible.
Quick coding tips
Below are my top five form programming tips and tricks:
- For easier and quicker testing, it is better to leave the ‘required’ column of the survey sheet in the spreadsheet template blank until the very end so that you can quickly skip through the questionnaire and jump between questions easily. Keep in mind that while you can easily skip fields that are not required, your relevance programming may not work as planned unless you answer certain necessary questions.
- If you plan to use Stata, keep your field names lowercase and 32 characters or shorter.
- Create pre-coded options or a finite range of responses wherever possible.
- Create checks and balances – like prompts that help guard against illogical responses – in the questionnaire wherever possible.
- Create your ID variable in SurveyCTO instead of trying to generate one ex-post. It helps with quicker identification of duplicates during data collection.
Test the questionnaire
Always test your questionnaire before training to make sure it works without errors. Don’t rely only on SurveyCTO’s ‘Preview’ function: also try it out on the tablet you will be using in the field. During tests of our baseline survey on Samsung tablets we were to use in the field, SurveyCTO crashed at the same question. However, the survey worked smoothly on Lenovo tablets that we had procured for a different project. On reaching out to SurveyCTO’s support team and sending a crash report, we found that the specific Samsung tablets we were using (which were a few years old at this point) could not process a large (70+ iterations) repeat group in the questionnaire. This problem was solved by simply breaking up the large repeat group into three smaller ones.
Test the data, too
In addition to making sure that the questionnaire runs smoothly, export the data and look through it to make sure that your data is in the format you want it in (see example below).
Repeat groups
When using repeat groups, fix the number of times a group will repeat and hide the instances that are not relevant (refer to repeat group ‘crop_rep1’ in the Ghana survey). While this adds additional lines of code, it ensures that the data is exported in a more convenient format
This is because you’ll be able to tell what each repeat instance relates to from the order of the repeat group, and each repeat group that addresses the same list of items will line up.
-Team SurveyCTO
You can also use a ‘select_multiple’ field to populate a repeat group, but this can lead to a misalignment of responses in your data if your repeat group is setup to repeat once per selection in your select_multiple field.
To avoid this, we recommend users design the repeat group using method 4 outlined in this documentation topic, which will ensure that repeat instances never misalign with select_multiple answers even if a surveyor swipes back and changes the answers after initiating the repeat group).
Nested repeat groups
Nested repeat groups are especially helpful in agricultural studies:
- Think through how you want your data to look (sometimes sketching it out can help) and then work backwards.
- Repeat groups can never overlap – make sure that the groups are completely encased within one another.
- When referencing data from within a repeat group, be sure to use the indexed-repeat() function. SurveyCTO has great guides on how to use the indexed-repeat function.
- When writing long complicated code like the code in the Ghana survey (for example: cell J103), it is easier to first write the code using a text editor (I like Sublime Text), and then copy it to Excel.
- Writing complex nested repeat groups can get frustrating, but stick with it! There is nothing like the feeling of sweet relief when the code works as planned.
Enumerators are your best code checkers
In our experience, we have always had enumerators who are very diligent during training and are quick to report problems with the survey. They are a great resource for catching problems with code – especially if it pertains to responses in the local language, conversion of units, and other questions that are heavily context specific.
Pre-testing with real-world respondents
When enumerators practice among each other, we often ask them to select responses that allow for the questionnaire to be completed in the stipulated time frame of the training session. But pre-testing with real-world respondents – actual farmers selected for a pre-test – is a great way to test both the instrument and the code, especially on questions that sometimes get missed during training.
Getting unstuck
In the case of coding in SurveyCTO, the best way to learn is through trial and error. SurveyCTO’s product documentation is a great resource too. However, getting stuck on code can be rather frustrating and it is sometimes better to just ask a colleague or submit a support request via the Support Center. Give yourself a deadline to fix a bug, and if the bug isn’t fixed by the deadline, ask for help. Often, as Murphy’s law would have it, as soon as you hit send on a help request, you’ll figure out how to solve it!
Data dashboard
The biggest advantage of using software like SurveyCTO is that data reaches you in nearly real-time. It is important to make the most of this. SurveyCTO does have a feature where you can manually create checks and balances to monitor your data (see the Data Explorer, automated quality checks, and review and corrections workflow). I personally prefer writing a do file in Stata to check for inconsistencies in the data. The inconsistencies you look for will depend on your data and how well you coded the questionnaire to restrict responses (using constraint expressions). You should run your quality checks (in the Data Explorer, or in Stata, etc.) on your data, and send feedback to your field teams for correction as frequently as possible and while the survey responses are fresh in their minds.
You can now also upload and merge the output of your own .do files with the form data and explore the combined data in the Data Explorer! Refer to the “Advance mode” section in this documentation topic for more information.
Making changes after the questionnaire is launched
Be extremely vigilant especially during the first two weeks of deployment to make sure everything is going smoothly. If you do catch an error through your quality checks or through enumerator feedback, be sure to upload a modified version of the questionnaire as soon as possible. Ensure that all enumerators know to re-download the questionnaire before continuing with data collection.
SurveyCTO has a mechanism for tracking versions and auto-merging data across versions so you don’t have to worry about losing data.
Open lines of communication
When managing data collection from halfway around the world, it is important that you have open lines of communication with your survey team. I prefer having one point of contact in the field. If you have more than one, like we did in Ghana, create a WhatsApp group. I found it to be a great way to communicate with our four survey team leaders in Ghana and send information in a consistent manner.
Code and clean
While this may seem like a lot of work, it is always beneficial when the same person who codes the questionnaire also trains the enumerators and cleans the data. This way, the data is in a format you are comfortable working in, and if the data are a mess you only have yourself to blame!
Acknowledgements
I would like to thank Mike Murphy at IFPRI for always sharing his SurveyCTO code and helping me troubleshoot mine. The survey and research described here were undertaken by the International Food Policy Research Institute (IFPRI) with funding from the International Initiative for Impact Evaluation (3ie) and the CGIAR Research Program on Policies, Institutions, and Markets (PIM). PIM is, in turn, supported by these donors. The survey and research received additional support from Africare and the Alliance for a Green Revolution in Africa (AGRA). The views and opinions expressed here are those of the author and do not necessarily reflect the official policy or position of any of these organizations.
