Your playbook to improving longitudinal data collection using case management

Longitudinal data collection comes with specific data quality and efficiency complexities.

We’ve put together insights, examples, and best practices on how organizations address these challenges and why case management should be the biggest tool in your toolbox for improving long-term studies.

In this playbook

What is longitudinal data collection?

Projects, surveys, and research studies where you track the same key data points on the same entities or subjects through repeated observations over a long period of time are known as longitudinal data collection.

The entities can be people, objects, households, firms, countries—any group you are collecting the data on.

This type of data collection is fundamentally different from standalone, one-off surveys, also known as cross-sectional data collection, where data is collected at a single point in time, without any further follow-up with those subjects.

There are many benefits to longitudinal data collection, including:

Deeper insights into trends and social changes than what a one-time survey would reveal
Providing the most up-to-date information on subjects that change over time
Greater ability to control for common survey errors, like cohort effects

While longitudinal studies are often associated with more traditional modes of academic research, like medicine, science, and public opinion, longitudinal data collection actually encompasses even more than that.

Longitudinal data collection can look like many different workflows across industries, such as:

Running baseline, midline, and endline surveys, or other long-term studies for research purposes
Tracking products in transit across multiple geographical points in a supply chain
Managing ongoing healthcare for patients through health records
Following up consistently on humanitarian program recipient well-being
Conducting ongoing inventory management for the same product categories
Analyzing data across a population over time, for topics from employment to health to education and more

If you’re thinking that these kinds of data collection projects are likely difficult to carry out—particularly because they can be so expansive and lengthy—you’d be right!

Let’s take a look at some of the main challenges with longitudinal data collection in the next section.

Longitudinal data collection challenges

Longitudinal data collection is complex and multi-faceted. Too often, organizations and companies are stuck spending precious time and resources managing challenges like these:

Disorganized data
Duplicating entities
Lack of internet, requiring travel in order to sync forms back to the cloud
Inability to effectively monitor data collection
Protecting sensitive respondent privacy

Duplicating surveys
Difficulty following up with respondents for future studies
Confusion amongst data collectors regarding assignments
Poor reporting and analysis
Resource-consuming data cleaning

Longitudinal data collection’s increased complexity means that the standard measures used to ensure data accuracy and completeness aren’t always enough.

Multiple iterations of a study or project means more data to clean, more margin for errors like duplication, and increased chances of surveying the wrong entities.

Additionally, these complex projects tend to require multiple forms, even during individual intervals of a study, which means that if traveling to areas with low or no internet connectivity is part of your project, you might not be able to update or access your forms.

These challenges are real, but they aren’t impossible to solve! They simply require digital solutions that empower teams to:

Track study subjects accurately through unique IDs
Interview the right people, and the right number of times
Ensure workflow integrity
Easily assign data collectors to their subjects
Work in areas without reliable Internet
Gather high-quality data that tells powerful stories for maximum impact delivery

Enter case management

Case management features in data collection platforms are incalculably powerful tools. This feature helps make sure that you’re not duplicating efforts and that your data is of the highest quality. Case management also provides the flexibility you need for every kind of unique longitudinal workflow, from offline projects to surveys in areas constrained by specific cultural or regional guidelines.

In the next section, we’ll explore how and why case management is a great solution to help you improve your longitudinal data collection efforts.

Addressing pitfalls in longitudinal data collection with case management

Firstly, let’s go over what case management is. Case management is a solution or feature that organizes and systematizes the process of collecting, managing, and tracking data. It becomes particularly valuable when collecting, managing, and tracking longitudinal data.

Case management solutions enable you to better follow specific entities or subjects, broadly called “cases,” over the course of data collection by organizing your project around the entities you’re studying, instead of around each form. Using one convenient master dataset, called a cases dataset, case management tracks and organizes each form that is filled out for each subject.

Now, let’s take some of the main challenges with longitudinal data collection mentioned in the section above and go over how case management solution addresses them.dddd

Organize your data better

Case management links each form in your workflow back to individual subjects through unique IDs for both subjects and forms.

This creates clean exports that display data from each form in your workflow in a chronological and orderly way.

Eliminate duplicative efforts

The ability to assign a list of cases to each enumerator ensures no one accesses the same cases in the field.

Tracking each form submission in the cases dataset also works to prevent duplication.

Never lose previously collected data

Your cases dataset stores respondent information, which can then be pulled into each form in your project so you never lose important information needed to locate or contact a case.

No internet? No problem

Case management solutions with offline capabilities store form data locally on devices, meaning data can be pulled from one form to another for seamless one-time interviews even when multiple forms are needed.

Enhance data monitoring

Case management enhances data monitoring dashboards and reports with additional case-specific information.

Sophisticated case management solutions let you assign cases to data collectors so that you can monitor data at the most granular level.

Reduce manual data cleaning

No need to manually match each new form submission back to a master dataset for your study entities–this is automatic with case management!

Improve impact reporting and analysis

Tracking case data effectively over time means that you’ll see trends in data emerge sooner and be better-positioned to articulate the story your data is telling.

Heighten data privacy and security

Storing sensitive respondent data in a cases dataset means you can design each form in your project to pull only absolutely necessary data from that dataset into each form.

The ability to assign cases to data collectors also ensures that only those assigned can see responses.

Smoother workflows. Safer data. Less time spent cleaning data. It’s no wonder organizations are turning to case management for their longitudinal data collection needs.

Organizational use cases for case management

Here’s a few real-world examples of how organizations like yours are gathering impactful longitudinal data using case management:

Use case: Endline assessment survey

Outline India turnkeyed a project for 3,000 respondents to a survey for 15,000

Problem: Social impact survey firm Outline India was hired to run an endline assessment to follow a baseline survey that had been done for 3,000 respondents. But when the project was put in place, they received a list of 15,000 respondents—nearly five times what they had planned for. In addition to the unexpected volume, Outline India did not have information from a baseline survey for the 12,000 new respondents.

Solution: With case management: Case management enabled Outline India to automatically assign respondent lists to their enumerators, saving precious time for a team working hard to operate on a much larger scale than anticipated. In combination with phone survey features, Outline India used case management to track and record the status of each call (call dropped, not available, busy, etc.) so that they could manage follow-up for each respondent. With case management, Outline India successfully transformed their workflow to grow from 3,000 to 15,000 respondents–and still completed the project on time!

Use case: Complaint management system in a humanitarian aid zone with unreliable internet

Oxfam built a system for improved humanitarian aid—and scaled to over 12 countries (and counting)

Problem: Tracking complaints and feedback in humanitarian disaster zones is critical—but difficult. Follow-up is often required, involving multiple forms for each complaint. International social justice organization Oxfam’s commitment to empowering the people they serve means they wanted the best possible feedback and complaints system possible. They also wanted to use local people to gather this feedback. These challenges meant they needed a robust digital solution for tracking feedback and complaints that was also user-friendly.

Solution: Using SurveyCTO’s offline case management functionality, Oxfam developed a system where they created cases in the field by entering each new feedback or complaint from a program beneficiary as a case. This enable straightforward and simple follow-up, allowing Oxfam to assign complaints to different field staff and have them complete all needed follow-up forms until each case was resolved. First piloted in Iraq, this workflow was so effective it’s been scaled to multiple countries. It’s also impressed donors so much it’s helped teams increase their funding!

Use case: Measuring economic impact during a crisis

ATR automated their workflows to survey 14,000 during the COVID-19 pandemic

Problem: Survey firm Assess, Form and Reach (ATR) Consulting needed to conduct a survey on the economic impact of COVID-19 via phone, in areas where respondents were difficult to reach. They needed to cover a nationally representative sample of 3,500 individual respondents in each round. No more than 25% of respondents could come from a single province, and at least 30 percent of them needed to be women. Previously, the ATR team used standard data collection forms to gather phone numbers from the field and then verify, clean, and distribute them to their call center agents. This was extremely inconvenient because any error resulted in multiple interviews, and there was no log of attempted calls.With the rise of the COVID-19 pandemic and a looming deadline, they needed a different method!

Solution: Using SurveyCTO’s case management and CATI features together, ATR successfully tracked attempted calls and call statuses at different intervals across time. CATI + case management let call center agents track attempted and dropped calls and streamline rescheduling calls. These features also prevented them from needing to distribute physical lists to call center agents. Instead, they were able to build and assign case lists for them in SurveyCTO. Despite being on a tight schedule with multiple criteria to meet, they completed two survey rounds in only two months.

Strategies for better longitudinal data collection through case management

When it comes to improving your data collection efforts, case management is an invaluable tool. Here are some strategies for how case management can be employed to reduce manual lift and better data quality!

Create and assign cases to data collectors. Because longitudinal data collection occurs over a long period of time, it’s important to keep data collectors and forms organized. By creating and assigning cases to specific data collectors, this ensures that they only access and collect data for the forms they’re supposed to, reducing the likelihood for multiple data collectors collecting data for the same form. This is an added benefit in situations where you don’t have direct supervision over data collectors.
Set up workflows to automate case assignment. Similarly, you can automatically assign follow-up forms and data collectors based on responses in earlier forms to ensure quality control and seamless routing. For large projects with many data collectors and multiple follow-up surveys, this also introduces time savings and reduces confusion for data collectors.
Screen segments easily for baseline surveys. When you need to get benchmark data or get an initial understanding of your survey segment before conducting further surveys, case management gives you the ability to easily set up qualification workflows so that you send only qualified respondents to future survey rounds.
Take advantage of capabilities that let you collect and manage data offline. Some case management solutions give you the ability to not only collect and store data without an internet connection, but assign cases and make responses collected on one device be available on another as well. Having offline capabilities gives your team the freedom they need to conduct data collection regardless of their environment.

There’s a better way to conduct longitudinal data collection

Increase efficiency, ensure data quality, and improve reporting through a data collection platform with native case management features.