Planning for electronic data collection

You need to collect some data. You’re familiar with pen-and-paper data collection, but you’ve decided to make the leap and enjoy the benefits of electronic data collection. Now you have a lot of questions: Isn’t it expensive? Could I lose my data in the cloud? Or threaten people’s privacy? Who can program the software? What hardware do I need? How much time will it take? What challenges should I expect? This post will discuss the key issues to consider when planning for electronic data collection.

Hardware

Broadly, your choices here fall into one of three categories: mobile phones, tablets, and netbook computers. 

1. Consider the design and complexity of your questionnaire:
   .
   • Screen size: Will enumerators need to see many questions on the screen at one time? If so, you may want a larger netbook or tablet screen.
   • Memory and CPU: Will there be many rosters and loops? For a more complex survey you’ll want a newer, more powerful device. For a simple survey, something older and cheaper will do.
2. Features: Think about any special functionality you’ll need from the hardware to collect your data. For example, a good camera to take photos? Or a good microphone to record interviews? The ability to take very accurate GPS measurements? Make sure that these features function adequately in your chosen hardware.
3. Battery: The battery life and recharge time of devices vary. Pick one appropriate for your field setting.
4. Fonts: Check to be sure the device supports the fonts and languages you need to conduct the survey. Even for the same model, this may vary depending on where the device is purchased.

Before committing to a specific device model, your best bet is to try, on that model, a survey form that’s comparable to the one you will field. Does it show the fonts properly? Is it speedy enough? Since there is so much variation in both devices and survey forms, actual testing is your safest strategy. 

Note that hardware and software are interrelated. Different hardware comes with different operating systems, supporting different software platforms. For example, if you choose SurveyCTO as your software, you won’t consider netbooks, as SurveyCTO runs on both the Android and iOS operating systems.

Software platform

The ideal software for your project offers both the functionality you need now and the functionality you might need as your use of electronic methods grows – but is not so complex that somebody on your team cannot feasibly learn, maintain, and extend the system over time. If you can be online all the time, very simple platforms like Survey Monkey might work. If you’re a programmer type who is comfortable writing code, then an advanced system like Blaise can work as well. The Open Data Kit (ODK) system runs a range in the middle: both very simple and very complex surveys are possible. SurveyCTO then improves upon ODK, adding hosting, support, and an improved interface that makes it easier to use for beginners and also adding tons of advanced features for sophisticated quality control and long, complex, world-class survey research.

Other key points to consider when choosing software:

1. 1. Server: Some software comes with server hosting. For others, you’ll need to set up and run your own server. Certain software works entirely offline and doesn’t require a server.
   2. Data security: Consider the sensitivity of your data and the security level required. The key to data security is how and where the data flows and who has access to it during this flow.
      .
      • For the typical “online” flow, data moves from the device to the server to a computer and a final storage location. Note that an “online” system typically requires connection to a server only periodically – as little as once per week.
      • An “offline” flow is needed for contexts where it is extremely difficult to ever connect devices directly to a server (even via 2g or other 2g/3g devices operating as local wi-fi hotspots).
      • Encryption: It’s important to encrypt data during this flow, and to decide who should have access to the decryption keys. Encryption is a process that makes the data unreadable to all but those who possess the computer-generated decryption key.

1. Overall cost: Think about the resources on your project team, and those you’ll need to acquire, when considering the total cost of running the system (including any consultants, servers, IT resources, etc.). Even if you have technical resources on-hand at the beginning of a project, consider the cost of sustaining them through to the end of the project. Often, a professionally-hosted option is cheaper and far easier once the full costs of alternatives have been considered.

You ideally want to invest resources in software that will grow with your project.

• Expanding data collection over time? You may want a good server and software that can handle longer surveys.
• Seeking the same people for a panel data set? You may want the ability to preload data into the survey to easily show and allow edits to existing data.
• Conducting similar rounds of data collection again and again? It may be worthwhile to choose software with which your team can fully manage form revisions, to save time and expense over the many rounds. (You don’t want to have to keep hiring consultants and starting over every time!)

Designing

The electronic version of the questions you want to ask are typically called the “template” or “form”. Decide who will design this version. Depending on the learning curve for the software you’ve chosen, that may be you or someone on your project team. The time it takes to design the form also depends on the length and complexity of your survey. A very rough rule of thumb is to plan for 1 day of work for every 2 pages of survey.

Projects sometimes choose to outsource form design to save time at the start. Even if you outsource, assigning a team member to start to learn the basics of the software can prove very beneficial later on.

In-house form design has great advantages:

• Team members have a clearer vision of the projects’ needs and the final product.
• Edits can be made to the template in real time during the testing and fielding stages.
• Data cleaning becomes easier, as the team can look to the form to know exactly how and when a question was asked.

If you choose to outsource the task, make sure you:

• Create clear specifications for your questions.
• Determine a plan for clear communication, especially on revision, timelines, and future rounds of testing.
• Expect the testing step to take a little more time.
• Have a plan for how to handle revisions to your forms over time.

Testing

Test your form first in the office, with a few others familiar with the project. Does the template work as you intended? Check that skip patterns work correctly, and that questions appear on the screen as you want them. Try moving around in the survey and making corrections, as may happen in a real interview. Edit or adjust the form as needed.

Then test in the field, under realistic survey conditions. Make sure the hardware holds up (e.g., do the batteries last? can enumerators see the screen clearly?), and see how well the form flows in an interview. Note that if you chose to outsource your template, you’ll need to plan for more time at this step. It may take a few iterations for the designer to understand what you need, and may take some additional time between pilots to communicate and make edits.

Fielding

Say that you’ve chosen the best hardware and software, then designed and tested the perfect form. For your technology to interact effectively with the real world during data collection, you’ll need great data collectors (enumerators) and field management systems.

• Consider technological abilities when hiring enumerators. If your data collection team isn’t comfortable with mobile-phone technology, plan to spend significant time training on the devices.
• Managing fieldwork can be simple with electronic data collection. Some platforms, such as SurveyCTO, allow the manager to assign specific surveys to different teams or individuals, and to easily track their progress in Google Docs as they submit surveys to the server.

Plan ahead to keep your electronic devices charged, secure, and functioning efficiently.

• Think about who will charge the devices and how. It’s simple, but vital. Will there be adequate electricity supply, at least in the evenings? If not, plan to purchase a back-up generator, extra batteries, or solar power banks. Make sure you have enough electrical outlets, surge protectors and charging cords. Always use surge protectors to guard against fluctuating currents and prolong the life of your devices’ batteries.
• Store devices securely when not in use. When in use for the day, keep a system to track who is responsible for which device.
• Put controls in place to prevent both viruses and undesirable use of the device (e.g., surfing the web between interviews). There are many free apps that can restrict the use of the devices (search for “parental controls”).
• Finally, make sure your devices are set to the proper date/time when you begin surveying, and ensure enumerators don’t change these settings, as they will appear on your survey’s time stamps.

Budget

Choices on hardware affect choices on software (and vice-versa), choices on software affect design, choices on design affect testing and fielding… and probably what you’re most worried about is how this will all determine the final cost!

A sample budget for the technology to support a large household survey is included below. You can also download a more complete example we’ve put into Excel and adapt it to your own project. Note that costs given in the sample budget are for illustration only – they will differ widely with context. Note also that the budget pictured below focuses on just the technology costs: it omits costs associated with your enumerator, supervisor, and quality-control teams as those will vary quite a lot by context. For example, you might outsource the actual data-collection to a firm that quotes you a simple per-interview cost. Still, we have included those extra costs in the Excel template so that you don’t forget them.