AppLab’s Initial Social-Impact Measurement Efforts Pay Off
February 08, 2011
Eric Cantor has led Grameen Foundation’s AppLab efforts in Uganda for the past three years, and continues to serve as an advisor on the project.
Grameen Foundation takes outcome measurement seriously. We want to make sure that our programs and services are effective, and that we can demonstrate their benefits before implementing programs or practices on a wider scale or urging others to replicate them.
With this in mind, we recently completed one of the first randomized control trials designed to assess the impact of a mobile phone-driven health service aimed at improving the lives of the poor. The service we sought to measure was Health Tips, part of the Google SMS suite launched throughout Uganda in 2009 with our partners Google and MTN Uganda. Our social impact partner Innovations for Poverty Action (IPA) performed the study.
Preliminary findings from the study are substantial, supporting some of our initial hypotheses and refuting others, and informing our approach to building pro-poor, mobile phone-driven solutions going forward. In short, findings indicated that when people learn of such services, they use them. People also seem to learn from this particular text-message query-based product. But we also found that, because of the limitations of human motivation and barriers like language and literacy, we have a lot more work to do.
The Health Tips study was conducted in Uganda over an 18-month period. Before the launch of Google SMS in June 2009, IPA conducted a baseline survey of 1,800 people in 60 rural communities, assessing demographic profiles, attitudes, and knowledge and behavior regarding sexual and reproductive health, and collecting data from local clinics. When we launched the service, we initiated a marketing campaign that randomly targeted half of those communities (the “treatment” areas) and did not reach the other half (the “control” areas).
[caption id="attachment_1471" align="aligncenter" width="300" caption="Our studies have shown the value of "trusted intermediaries" -- such as the Mobile Midwife counselor in the photo above -- as a way to make mobile phone-based communications to the poor more effective. "]
[/caption]
Through randomization, IPA chose two sets of communities that were uniform in every relevant respect – except that one was exposed to the product through targeted marketing campaigns, while the other was not. Nine months later, they began a follow-up survey of 2,400 people to detect changes. They looked at data from surrounding clinics, conducted qualitative interviews and assessed the information provided to the communities. Because the targeted marketing in treatment villages was effective – we saw more than four times as much usage in the treatment areas as in the control – we were able to assess the effect of the service on attitudes, knowledge and behavior relating to sexual and reproductive health.
