Innovations for Poverty Action (IPA) Uses Mystery Shopping to Measure the True Cost of Mobile Banking in Uganda with SurveyCTO
Meet Innovations for Poverty Action (IPA)
Innovations for Poverty Action (IPA) is a global research and policy nonprofit that promotes effective solutions to global poverty problems. By designing and evaluating potential solutions in real-world contexts, IPA ensures that decision-makers use high-quality evidence to improve the lives of the world’s poor.
- Offices: Washington, D.C., and globally across 15 countries
- Sector: Non-profit / International Development / Research
- Use case: Financial inclusion tracking and consumer protection auditing via field intercept surveys and multi-pronged mystery shopping workflows
- Employees: 1,000+ globally
- Features used: SurveyCTO Collect mobile survey app, SurveyCTO web forms distributed via links in WhatsApp, offline data collection capabilities, and case management for agent tracking and location assignment
The Challenge: Measuring mobile banking as a means of financial inclusion in Uganda
Financial inclusion is a vital component to eliminating poverty. Being unable to access common systems like banks can prevent people from obtaining steady work and achieving financial stability.
IPA’s Financial Inclusion Program works with financial service providers, policymakers, and investors to design and test financial products and foster better environments inclusive financial innovations. They also work to protect low-income consumers against the risks that can come with participating in financial institutions.
Within IPA’s Financial Inclusion program lives the Consumer Protection Research Initiative. Here, IPA works with regulators, financial service providers, and researchers to help fill critical evidence gaps on how to protect consumers and promote financial health.
One such area with a potential evidence gap relates to low-income users of mobile money networks in Uganda. Mobile money is a service that allows people to use their mobile devices as a digital wallet that can store, send, and receive money. Across much of sub-Saharan Africa, these mobile money networks have become major drivers of financial inclusion, giving people who may not have access to traditional banks and were previously limited to using only cash in their daily lives the ability to save and access money in a new, digital way. While there are many positives to the expansion of these networks, there are also some risks involved with their use.
One unique aspect of mobile money networks is that transfers of e-money occur through human mobile money agents, who essentially act as “human ATMs.” Agents sometimes charge a fee for their services, which can be prohibitive for some consumers. There can be other issues, too—sometimes, agents aren’t available or are unable to complete transactions, which means consumers can’t send or access their money.
Research on using digital financial services like mobile money suggests that it can improve consumer welfare and reduce poverty. However, the costs for using these services, such as hidden fees and agent availability, could limit usage.
IPA’s Consumer Protection and Financial Team set out to gather evidence around the true financial and logistical costs that everyday consumers face when using mobile money networks. To do this, they launched the Transaction Cost Index (TCI) project in Bangladesh, Tanzania, and Uganda.
IPA needed to track two different types of transaction costs:
- Financial costs: Documenting fees set by official digital financial service (DFS) providers versus unofficial, illicit overcharging tacked on by independent local agents.
- Logistical costs: Measuring how much time was spent at an agent location, and the cost of failed transactions due to an agent lacking liquidity or experiencing network downtime.
Data needed to be gathered across both urban cities (like Kampala and Mbarara) and more remote settings (like rural Uganda). IPA decided to deploy four distinct data collection methodologies simultaneously:
1. Consumer intercept surveys
2. Professional mystery shopping
3. Local consumer mystery shopping
4. Remote local consumer mystery shopping
Gathering standardized data across these disparate workflows while navigating areas with unreliable internet connectivity presented significant challenges. IPA hired local enumerators for their in-person surveys, which meant they needed to have a digital tool that could be learned quickly by people of varying levels of technical skill.
This operation wasn’t going to be simple.
The Solution: A mystery shopping workflow built on SurveyCTO
As longtime SurveyCTO users, the team at IPA knew that they had a data collection tool that could handle this level of complexity. They planned to leverage SurveyCTO’s tools for in-person and web surveys, as well as its offline functionality to deploy and manage the different fieldwork approaches.
For the Consumer Intercept Surveys, IPA deployed a team equipped with tablets running the SurveyCTO Collect mobile survey app. Stationed strategically near mobile money agent locations, enumerators intercepted real customers immediately following their transactions. Using programmed screening questions deployed to the Collect app, enumerators recorded standardized consumer experiences directly into their devices, ensuring data integrity even when working offline in remote rural environments.
For the Professional Mystery Shopping layer, trained enumerators acted as undercover shoppers. To ensure objective comparisons, IPA established a rigid protocol: enumerators all conducted identical transactions—such as a specific cash-in deposit or wallet withdrawal—using a uniform amount of 40,000 Ugandan shillings (~$12 USD).
The most innovative phase of the project involved the Local Consumer Mystery Shopping. To capture more authentic human mobile money agent behavior, IPA recruited actual local consumers as the mystery shoppers. Field managers distributed unique SurveyCTO survey links via WhatsApp, and the local shoppers tapped the link to launch a clean, self-administered survey directly inside their phone’s web browser, where they could seamlessly enter the transaction fees, agent behavior, and time spent with mobile money market agents.
IPA also relied on SurveyCTO’s case management feature to pre-assign mobile money agent locations to specific shoppers, which made it easy for team managers to tell if a shopper gathered data at an incorrect location.
The Results
With SurveyCTO’s features and a well-planned workflow, IPA successfully collected rich field data that exposed the operational realities of mobile financial systems across Bangladesh, Tanzania, and Uganda. The project allowed researchers to isolate fees set by official digital financial service providers from unauthorized, off-the-book agent overcharges.
The flexibility of SurveyCTO’s forms and user-friendly platform also enabled IPA to successfully convert ordinary citizens into an active, localized data-collection network, expanding on their methodologies for measuring financial inclusion.
Ultimately, the data generated by the Transaction Cost Index project can be used to give financial regulators and consumer protection agencies the information they need to enforce fair pricing, protect low-income consumers, and contribute to a more transparent financial system across communities that rely on mobile money networks.
