Considerations for students

Graduate students have more to think about when it comes to electronic data collection. As students, timelines and budgets are tight, especially for those collecting data abroad. There are three short months in the summer to land in-country and collect data before returning to campus. And there’s likely little room for error with the budget. Luckily, with some planning and preparation, electronic data collection can maximize results within these constraints.

This post is a student-specific companion to my earlier post on planning for electronic data collection. Do see that post for more advice on planning.

Making the most of your travel time

Many survey tasks can be done from any location, before hitting the field. The figure above outlines the usual tasks for a field survey, organized into three categories: “organizing” tasks to do while still on campus, “getting started” tasks that can be done either on campus or in the field (or in some combination), and “implementing” tasks that really must be done in-country.

Organizing tasks

1. Identify sample. Do this before any other task. There are two dimensions: theoretical and practical. Theoretically, decide what population to survey, how many must be surveyed, and how to select those to be surveyed. Practically, then, there are a few considerations:
   .
   • Timeline and budget. Longer questionnaires, larger samples, and geographically-spread respondents take more time and money. Work with the sample size, geography, timeline/budget and survey length (hours/minutes) to find a combination that fits all constraints. For example:
     .
     • The sample is 500 households in 50 villages of the country.
     • All villages are in the same region.
     • Fieldwork must finish within July & August.
     • Surveys are an hour long.
     • Households are close together within the village, so one enumerator can realistically survey 4 households in one day. (Warning: it is easy to be over-optimistic here. Average performance may fall substantially short of best-case performance.)
     • 500 households/4 surveys per day = 125 enumerator-salary-days
     • The survey can be completed within July & August if you hire just 4 enumerators, a very manageable team size (125 enumerator-days/40 working days = 3.1 enumerators). (Warning: it is easy to be over-optimistic about the number of working days; holidays come out of nowhere and last longer than you might expect, enumerators get sick, etc.)
   • Know how the sample will be identified in the field. Whose door will the enumerator knock on? How will they find that door? Know the information available to identify the sample in the real world. Perhaps sufficient administrative data (from the government or maybe your local partner) is readily available. Otherwise, budget extra time and money for enumerators to obtain it.
2. Draft questionnaire. For long surveys, don’t start from scratch. Check for surveys done on similar topics in the same country. It will save a lot of time later on. For example, if the survey has a Household Assets module, obtaining a context-specific list of assets that a household may have from another survey will help a lot.
3. Translate questionnaire. Save enough time for this task so the translator doesn’t need to rush, and be sure the translator has country and sector-specific knowledge. Use a second translator to “back-translate” everything so that you can check and discuss the wording differences that inevitably emerge.
4. Draft survey manual. This master document is a guide to the enumerator training, fieldwork, and interpretation of the final dataset. For each question, be sure to include all specific instructions and explanations that don’t themselves appear in the questionnaire itself.
5. Submit IRB application. Your university’s IRB will typically require information about the study methodology and a draft questionnaire. Submit early as the approval process takes time. Check whether there are any local IRBs in-country and submit to them as well.
6. Draft electronic template. This can be done simultaneous to, or even before, the questionnaire translation. Just be sure not to delay any translation needed to submit to the IRB. Also, if planning to preload any data into the template, it can be prepared at this time.
7. Desk test electronic template. Test the template, from the comfort of your office, to be sure it operates properly. Invoke different skip patterns to check the logic. See how questions will appear to the enumerators. Edit as needed and test again.
8. Purchase devices and related equipment. Scope out the best price for the devices (see “Making the Most of Your Budget” below for a discussion on device strategy). If buying yourself, it is often cheaper to do so before going to the field. Consider the field location and purchase additional power devices (car chargers, extra batteries, back up generator, even solar panels) if needed.

Getting-started tasks

All of these tasks can be completed before or after arriving in the field, depending on time constraints, your knowledge of the in-country context, and whether you already have contacts on the ground. A range of timings is presented in the figure, from the earliest they can possibly begin (if working pre-field) to the latest date for completion (if working in-the-field). No task should take the full time allotted.

1. Hire survey firm and/or key staff. Maybe the sample size is small enough such that you can collect all the data yourself, or maybe you just need a translator to help out. As surveys increase in size, logistical and supervision complications arise, and it may be helpful to work with a survey firm or other in-country partner. Though the actual hiring of a firm or staff will likely happen in-country, initial steps can happen before arriving: scope out possible firms or candidates (talk to other who have worked in-country), set dates for meetings, draft TOR’s, etc.
2. Create field plan. This is the daily schedule for the enumeration team, detailing where they will be each day and whom they will survey. This is easier if you have in-country experience; if not, ask around among other field staff or researchers working in-country to get a sense of the ease and time it takes to travel.
3. Set up data flow. Use encryption to keep data secure. Practice sending surveys to the server, downloading to a computer, and exporting the data to a useable format. Depending on the sensitivity of the data, it may be necessary to set up an offline data flow, which you can test anywhere. If your data is extremely sensitive, you may need to set up a “cold room computer” to keep unencrypted data isolated from public networks; this would require more planning and set-up in-country, so be sure to engage with your IRB(s) early enough so that you understand their data-security requirements.
4. Set up monitoring systems.
   .
   • Progress. This can be a simple Google Spreadsheet, for tracking completed surveys against the full sample.
   • Quality control. Decide how data quality will be monitored. Electronic data collection offers lower cost and faster solutions to standard backchecks or audits (where a different enumerator re-visits a respondent). Instead, short audio recordings or text audits describing the path and speed taken through a survey are possibilities. Set a protocol to follow up on results and solve data issues.

Implementing tasks

1. Hire enumerators and other field staff. Review resumes in advance, if hiring yourself. Set interviews for your first days in-country.
2. Pilot questionnaire. Test the content of the questionnaire in practice interviews. Choose respondents similar to, but not part of, your actual sample. Consider the flow of the questionnaire, respondent understanding of each question, and the comprehensiveness of the answer choices. Make edits to the electronic template as needed. Submit test surveys to the server and look at the data output before the survey goes live. Finally, most IRB’s will require submission of the final questionnaire with any changes marked in “Track Changes”, for final approval before fielding.
3. Logistics for field and training. Here a survey firm or other local counterparts may come to great use. Rent a space for training, determine where the enumerators will stay in the field (hotel), and how they will get around (bus, van/car, taxi, foot, boat).
4. Enumerator training. Plan for 1-2 weeks for classroom training, a field pilot, and getting all hiring logistics set. A longer questionnaire will require more classroom training to explain all of the questions. If not running the training (or co-running with a translator), be present for the duration of the training to clarify questions as they come up and to be sure the questions will be asked the way you intended.
5. Field work. Plan to stay in-country for the duration of the field work. Constant supervision in the field is useful, as is checking the data regularly and providing the enumerators with relevant feedback.

Postscript: Making the most of your budget

Finally, a few pieces of advice for those with especially tight budgets:

1. Devices. If it’s just you and a translator, buying one device is simple and affordable. If you have enumerators, the situation becomes more complicated. Extending the fieldwork over a longer period of time helps – this means fewer enumerators and fewer devices. If you don’t want to purchase devices yourself, seek out a survey firm or in-country partner willing to lease devices to you for the duration of your project.
2. Fieldwork. Proper planning and monitoring will save time and money. Be sure that estimations of surveys per day are realistic, and that enumerators have an incentive to meet that target. Verify the accuracy of the sample list to avoid wasting time in the field searching for respondents. Avoid last minute changes to the questionnaire. Take the time to train enumerators well, and start monitoring data quality right away, so there’s no need to revisit households to address data-quality concerns.
3. When to sub-contract for data collection. Thinking through the points above on timeline and budget will help with this decision. In general, sub-contracting is a good option if the constraint is time over budget. Doing everything yourself will save money, but take a lot of time. Survey firms have staff in-country, experience organizing logistics, and a pool of enumerators to hire. If you require a large enumeration team, sub-contracting may be a particularly attractive option; such surveys will be harder to supervise and manage, with the potential for logistical challenges and data-quality issues.
4. What not to sub-contract. You always want to carefully monitor, yourself, for data quality. This always means reviewing incoming data for signs of trouble, but, if you have the budget for it, you may also want to directly employ field monitors who can randomly accompany enumerators, perform back-checks by revisiting those who have already been surveyed, and generally keep watch over the quality of field operations. You may also want to configure your survey forms to include random audio audits, in which case you would directly employ office staff to review the recordings, compare with the data, and flag any signs of trouble. It’s really easy to collect bad data – particularly when sub-contracting – and you just don’t want to do that.

With a little planning and a lot of hard work, you can do world-class research as a student – even with a tight budget. Good luck!