Automated fieldwork management in Ethiopia’s HIPSTER program evaluation

Christian Meyer, a doctoral student in economics at the European University Institute, spoke to us recently about how he has been automating field research in his work in Ethiopia. He uses lab-in-the-field and field experiments to understand the decision-making of the young women who work in Ethiopia’s growing manufacturing industry. Are they using their jobs as a stepping stone to find better opportunities? Are they able to save money while working these jobs, and does that money help them or their families? Are these employment opportunities empowering for the women who take them up? These are just a few of the questions Christian’s research looks to address. Together with his coauthors, Christian is also looking to evaluate the effects of a large government program which aims to industrialize Southern Ethiopia. Christian’s work focuses on the Hawassa Industrial Park Sourcing and Training Employees in the Region (HIPSTER) program. Hawassa Industrial Park is one of 10 industrial parks in Ethiopia, which aims to employ and train 60,000 garment industry factory workers.

Christian’s research involves the practical tasks of randomly selecting workers for inclusion in the study and interviewing them at different points over time. Beneficiaries are enrolled in face-to-face interviews, then contacted again by phone every two weeks for eight follow-up interviews. With new cohorts of study participants being recruited for the purpose of the evaluation each week, this sort of process represents a significant administrative challenge for a small team to ensure that follow-up interview dates are kept. Given the large scale of the HIPSTER program, survey logistics for the evaluation that Christian and his team are working on will likely be even more complex. This thought lead Christian to consider what parts of the process might be automated.

The basic parts of the system for collecting data for this evaluation are two digital forms that use SurveyCTO’s case management functionality.

• Case management is an alternative data collection workflow where enumerators first select a case, then open an associated form to capture data.
• A case can be any unit of observation in a study, and in Christian’s study, a “case” is a program participant.
• Case management in SurveyCTO can be augmented further using data publishing whereby a form submission can create a case, update the details of a case, or close a case.
• Christian made use of this functionality, creating an enrollment form that created new cases.
• The list of cases for this study amounted to a “waiting room” of program beneficiaries who could be selected by enumerators for subsequent interviews.
• Once an enumerator selects a case from the case list, they take ownership of that case for all follow-up calls.
• That case becomes assigned to that enumerator, only appearing in their case list on their mobile device.

A system to keep track of who had enrolled, how many calls each participant had received, and who had received incentives was also required. Christian decided to publish the case assignment list from SurveyCTO to Google Sheets (a feature that SurveyCTO supports natively). Google Sheets is a free and easily-shareable spreadsheet tool which we’ve blogged about before as a tool for dashboards, here, and here. In Christian’s case, his field manager monitors the dashboard on a daily basis, following up on irregularities, distributing incentives, and generally managing the flow of cohorts of respondents. Conditional highlighting across the sheet also helps to draw the user’s attention to the relevant cases that require attention. Here’s a mockup that shows the general principle of this dashboard:

A system that depends on centralized management to be on track can be quite difficult and everyone can use a reminder or nudge to stay on schedule. This is where Christian decided to invest in some automation. He employed Zapier, a web automation tool which SurveyCTO supports.

• Zapier helps non-programmers connect different systems to automate processes, triggering events and sending data.
• These automatic tasks or jobs are known as “zaps”, which are easily configured by people with no programming skills.
• Zapier “zaps” connect services and applications (like SurveyCTO, Google Sheets, and Google Calendar) in automated tasks that trigger actions in one service when new data is available in another service.

Christian was kind enough to share a screenshot of a zap configuration that creates a calendar event when a new row arrives in a Google Sheet which meets certain conditions:

While it is possible to directly publish data from SurveyCTO to another system using Zapier, Christian elected to publish data first from SurveyCTO to Google Sheets, then have Zapier “watch” Google Sheets for changes and trigger events in response. Zapier was used to monitor a Google Sheet for new data from SurveyCTO that indicates that a participant is now enrolled and is assigned to a particular enumerator. Based on the enrollment data, eight events are automatically created by Zapier “zaps” in the Google Calendar of that enumerator. Then all that an enumerator has to do to stay on top of follow-up interviews is to check his or her Google Calendar for the day, for a list of calls to make. In addition to calendar appointments, enumerators would also receive SMS reminders about calls they had to make, also triggered by Zapier.

Together, these parts have made for an effective survey system, significantly improved through automated management:

• Enumerators have been kept largely on schedule through automated reminders.
• The project’s field manager has been able to help keep track of overlapping cohorts of respondents through use of an automated spreadsheet dashboard that automatically gets updated.

According to Christian, not everything could be automated, though, like mobile money incentive payments. At this time, neither Ethio Telecoms nor the new Ethiopian mobile money system “CBE Birr” have easily-accessible APIs that could be connected to Zapier or SurveyCTO.

In spite of planning and care, surveys are not without challenges. One of Christian’s tips is that Zapier’s Professional level subscription plan is a good investment as “zaps” that fail for some reason will be automatically run again, so whatever Zapier is automating for you is more certain to happen every time. Internet connectivity is also challenging in Ethiopia, which necessitates that his field team synchronise their devices with the server where possible, via Internet cafes, and sometimes even traveling back to survey headquarters to submit form submissions. If survey forms dynamically respond to the server dataset, it is also important to make sure that syncing happens often enough. So definitely consider how your enumerators will connect to the Internet in the field.

Christian will be busy for some time with his research and the ongoing HIPSTER program evaluation. So far, automation has been a key ingredient in making this evaluation efficient and manageable.

Photo credit: Christian Meyer, European University Institute