Statistics vs Outcomes — Rethinking the “Why” behind data collection in the social sector.
How does data collection happen today? Most enterprise data platforms in social development are simply bundled survey tools + data warehouses + dashboard products. These systems all depend on the quality of the data collection processes. By their very nature, they are disconnected from the program activities and lead to a data collection process that turns into a “feed the beast” exercise for field staff. This tax on field staff is enormous and leads to poor quality data that is difficult to clean and derive. To solve this challenge, the program data collection effort is often delegated to researchers and monitoring and evaluation staff that have stronger data collection practices but are deployed on a much smaller population and yield an incomplete data set. This leads to extrapolated values and turns individuals into research projects and statistics!
It also creates a culture of distrust and leads to source data being primarily available to the department or group that gathered it rather than those who need it to adaptively manage the program and directly improve people’s lives. And if there is a problem with the data, it is less likely to be discovered. The funder is often at the end of this data chain, getting the oldest, lowest quality, extrapolated data sets — second only to the field staff who often never sees the data and of course the participant who is not even considered a data customer in this paradigm.
Our position on individual & personalized data
At Impact Atlas, we understood that if we could provide an ever-evolving individual-level profile and dashboard, that compares progress against goals and consolidates comprehensive, cross-organizational touchpoints into this single individual profile, and we made the same data available to all data customers, including field staff and foundation staff, we could catalyze unprecedented progress in the quality of social programs. We also understood that data quality would increase significantly if the collection process would result in useful field-level tools for adaptively managing interventions on the ground. And if data problems developed, the entire organization would be incentivized to resolve it, given the data is core to the workflow.
This philosophy of cultivating individualized profile data is widely accepted by the best and largest technology organizations such as Google, Apple, Amazon, and Facebook. They all recognize the value and importance of individual, business, and group-level data to create macro-level analytics. And wouldn’t trust any data for major decisions that’s source couldn’t be tied back to the individual level. This provides them with extraordinary abilities not only to analyze but also influence behavior for macro population groups and micro-segments within a timeframe that is relevant. This kind of data culture is standard for organizations that intend to use data to achieve their mission.
“At Impact Atlas, we understood that if we could provide an ever-evolving individual-level profile and dashboard, that compares progress against goals and consolidates comprehensive, cross-organizational touchpoints into this single individual profile, and we made the same data available to all data customers, including field staff and foundation staff, we could catalyze unprecedented progress in the quality of social development programs.”
Outcomes vs. Statistics
Foundations and Implementing Organizations that are serious about achieving outcomes and continuously improving program quality and efficiency need high quality, right-time, actionable data across the program lifecycle. Most foundation impact management and monitoring and evaluation systems are robust survey systems developed for management purposes only and separated from the program delivery systems. This by default creates an enormous downgrade in data quality.
Many foundations think of impact data for their own grant approval and board reporting process, which has them create grantee reporting portals, with custom survey forms for each partner to complete. This data is useful only to the foundation and does not meet any modern data quality standards. We recognize that the information reported in grantee reports are also important data sets. However, we believe their purpose is achieved when the information augments what should be the primary data set, which is source data obtained directly from the technology enabling program delivery.
Given that foundations are at the top of the food chain, it is typical for implementing organizations to try to satisfy donor reporting requirements by responding to report templates and survey questions, creating unique data sets for the foundation, and counting activities, while the organization quietly struggles internally to understand what’s really happening at the program level.
This disconnect exists in 99% of the cases, however, it is difficult for funders to expose, because it would feel accusatory and threatening to the partners. As a result, the industry has over-invested in third-party randomized control trials and expensively managed development impact bonds to attempt to validate program performance or tie payments to success. We believe the sector should have a more nuanced approach and a key role to play.
What we believe at Impact Atlas
At Impact Atlas, we believe it is possible for technology to carry the load in creating a circular information highway that constantly improves, is refreshed, and available in the same version to the entire customer matrix at the same time.
We also believe that the data gathered should be an asset to the participant and the implementing organization. To achieve this will take courage and leadership. The technology has arrived to make it possible. An outcomes-focused foundation or a dedicated small group of funders would be well-positioned to unlock this extraordinary potential by supporting its partners to build a data revolution.
“At Impact Atlas, we believe it is possible for technology to carry the load in creating a circular information highway that constantly improves, is refreshed, and available in the same version to the entire customer matrix at the same time.
If you are a funder, a partner or an implementer who believes in this new paradigm, please reach out and let us know what your organization is doing with data, how we might be able to work together, or general thoughts on how we can collectively create positive lasting change.”
— Chantal\ CEO & Co-founder of Impact Atlas
About Impact Atlas
Impact Atlas is a real-time intelligence platform for operations and impact around the world. It is personalized to monitor program lifecycle delivery and transform the way development programs are managed and financed. For more information or to schedule a demo, visit impactatlas.com