As a social innovator, you’re likely measuring and reporting on the social impact of your interventions in some capacity — but how meaningful is the information you’re collecting?
According to the authors of The Power of Lean Data, the practice of impact measurement in the social sector is coming up short. While we emphasize reporting our outputs — such as the number of lives reached, or number of jobs created, we largely fail to capture deeper insights about our work — how customers experience our goods and services, if our products are making a difference, and whether we’re reaching the people who need us most.
If you’re a social enterprise or nonprofit with limited resources, you may have hesitations around data collection. You may be thinking it’s a hard, slow, and expensive process. Impact measurement doesn’t have to be complicated to be effective. When done right, the task can be simple, attainable, and useful.
In this guide, you’ll:
You’ll hear from Julia Mensink, Acumen’s Head of Impact, from Sasha Dichter, Co-Founder of impact measurement company 60 Decibels, and from our community of social entrepreneurs including John Nyagwencha, CEO of Aqua Clara Kenya (ACK), Carol Kariuki, Co-Founder and CEO of GreenPot Enterprises Limited, and others who use impact measurement tools to drive actionable change.
Effective impact measurement goes beyond reporting. When you focus your data collection on the customers you serve and the change you’re effecting, impact measurement can help guide your social enterprise to the next level of change-making.
According to Julia, “What we typically mean when we talk about impact is a positive, lasting effect on people and planet. It’s a change in people’s wellbeing or in the natural environment caused by an organization.”
Social impact measurement is the collection of data to measure, assess, and report on your impact performance. For social innovators, it means thinking through how a product or service is impactful for people and the environment.
Read below about four reasons why impact measurement matters and tips to get you started:
Impact measurement can teach you a lot about your work and your customers. It can help you make tough decisions between different strategies, and answer important business questions.
When trying to make decisions, your data can help you answer questions such as:
For example, Justine Nyaruri Abuga is the CEO and Founder of Ecobora, a pioneer within the clean energy space. When the social enterprise introduced a solar boiler for school kitchens in rural Kenya, Justine and his team developed a lean and rapid customer discovery experiment. Working together with impact measurement company 60 Decibels, their goal was to test a new pay-as-you-go method of financing.
They reflected on a key customer hypothesis: schools found their hardware costly. Determined to learn more about their customer’s pain points, the team expanded their interviews to include students, teachers, cooks, and board members to explore more affordable ways for schools to access their solar boiler.
As a result of the experiment, Ecobora decided to shift to a pay-as-you-go financial model. “Our customer discovery experiment was an eye-opener. The feedback was overwhelming! 90% preferred the pay-as-you-go for energy approach so that they didn’t have to own the boiler hardware,” said Justine.
This new opportunity is mutually beneficial for schools and Ecobora. The schools can pay only for what they need, while Ecobora can use their success to pursue new partnerships such as The World Bank, which is interested in learning more about the company’s technology and impressive impact thus far.
Without focusing on capturing the voices of many stakeholders through customer interviews, and designing their business model with customers at the center, Ecobora wouldn't have been able to make such a clear decision to offer pay-as-you-go and open new avenues for success.
Collecting valuable data to make decisions starts with listening to your customers. The insights you gather from them will help you understand how your business creates meaningful change in the lives of the people you serve.
“Most impact investments aim to reach underserved populations, and yet impact investors and businesses struggle to get reliable data on exactly who these customers are. This matters a lot if you’re trying to understand things like what price points will be affordable for a new solar home system, whether financing will be required for a water-irrigation solution, and how to interpret the financial returns of two competing impact investing funds,” says Sasha Dichter.
Acumen Academy’s Social Impact Analysis course teaches how to define your impact hypothesis, also known as an impact model. An impact hypothesis outlines your theory of the positive social impact that takes place when people engage with your product or service.
Design your hypothesis by defining the focus, depth, and breadth of your impact:
Before speaking to your customers, you should identify the data that is most useful to you. This can be determined by breaking down the process into three steps:
Don’t get intimidated by big words. You’re doing the work and you have the experience. You already have a sense that the work you’re doing — the service or product you’re providing — makes a difference in people’s lives.
It’s about having a customer-centered approach to measuring your social impact: starting with your customers, learning to understand them, and using the everyday tools you have to contact them to try and understand what’s going on in their lives and the changes they experience as a result of your products or services.
By Masha Lisak and Carmel Farzaneh, Learning & Impact at Omidyar Network
IkamvaYouth staff were stumped. Why were high numbers of learners unable to meet the minimum 75% attendance requirement of their after-school tutoring programs? It was a recurring topic at team meetings, and they were struggling to understand the situation.
The group had brainstormed possible reasons, and had two hypotheses: first, that learners didn't find the program valuable and therefore stopped attending, and second, that compulsory after-school sessions were causing schedule clashes.
Omidyar Network is a philanthropic investment firm which has supported IkamvaYouth since 2013 and has seen how its programs strive to provide learners from the most disadvantaged South African communities with the knowledge, skills, networks, and resources to access tertiary education or employment opportunities.
To shed light on the dilemma, Acumen’s Lean Data team (now 60 Decibels) joined forces to gather new insights to help the IkamvaYouth team through experimentation principles and low-cost technology to gather data directly from customers.
With research questions clearly defined, the team created and got in touch with those who left the program to learn more about why. Through an online survey with tutors, in-depth phone interviews with learners, dropouts, tutors, and parents, and IVR temp-check trials which allowed learners to give feedback within 24 hours of their tutoring sessions, the IkamvaYouth and Lean Data team finally had some clues.
The results surprised the IkamvaYouth team, who found that across both active learners and dropout learners, their original hypothesis that dropouts did not find value in the program was incorrect. With their original hypothesis debunked, the team dug deeper into identifying why learners drop out.
A requirement to participate in the program had been to attend 75% of their tutoring sessions, and when learners didn't meet this bar, staff assumed this was mainly due to a lack of interest. To the contrary, surveys found that many learners were missing the requirement due to legitimate conflicts rather than a lack of interest.
Inspired by these findings, IkamvaYouth is now taking two important steps. First, they’re carefully considering distance from the centers when enrolling new students. Second, they've developed an "Ikamva Lite" model to provide a less intensive program with fewer attendance requirements that can serve learners with scheduling conflicts.
As a next step, the Lean Data team suggested evaluating tutor quality to understand how the volunteer tutors might be affecting student learning. To assess this important piece of their program, they piloted a phone survey to ask learners to evaluate their last session. While many responses were positive, some showed that learners were disappointed by the sessions, leading IkamvaYouth to more actively evaluate their tutors. From this learning, they now have a process for learners to give feedback to tutors right after each session, resulting in greater accountability and strengthening the relationship between tutors and learners.
Bolstered by the findings of the Lean Data project, IkamvaYouth is continuing to learn better ways to support and encourage the next generation of South Africans.
A lean data approach to measuring social impact, like the one used by IkamvaYouth, doesn’t have to be expensive to be effective. Acumen Academy’s course on Lean Data Approaches can help you identify the right technology to easily collect data and still gain high-quality customer insights that help you drive important business decisions.
Impact measurement can feel overwhelming, but building a system to collect data and assess your impact doesn’t have to be complicated. In fact, it can help you stay focused on what matters most: your mission and the people you serve.
Acumen Fellow John Nyagwencha is the CEO of Aqua Clara Kenya (ACK), a social enterprise producing water purification tech for rural communities. In a country where 40% of the population has no access to clean water, John explains how customer discovery helped the company stay true to its mission of serving rural populations.
“One of the biggest challenges we face is how to stay focused on this target market. It’s much easier for me to run Aqua Clara targeting middle-income urban families and institutions. If you’re running a business you have to stay afloat and pay your bills, so there are temptations to target higher-paying customers” explains John.
As a participant of the Acumen and Ikea Social Entrepreneurship East Africa Accelerator, John worked with 60 Decibels and designed customer interviews to understand the kind of impact they were having on rural communities.
Following the interviews, he learned that 60% of customers are accessing ACK’s solution for the first time, and 91% said if they didn’t have this option, they would have no other alternative for water sanitation products. For a company focused on providing access to those who otherwise wouldn’t have it, this insightful measure of impact assured John that his company was reaching the right customers.
John’s advice to social enterprises looking to validate whether they’re on the right path is to ask: “What have you been set up to do, and are you able to do that? You could be successful from a monetary perspective, you could be profitable, but are you solving the problems you set out to solve?”
To nail these questions, consider designing a value proposition canvas which maps out your customers’ pains and gains, and helps you determine where your organization can be most effective. Acumen Academy’s free Lean Startup Principles course teaches how to create and test hypotheses to refine your value proposition and stay focused on what matters most.
Like John, being intentional by collecting small insights can help you stay focused on your primary mission.
Head of Research Operations at 60 Decibels Ashley Speyer says find the intentionality of your data collection by asking yourself:
Once you define your internal reasons for collecting data, design simple questions using a lean survey approach:
For inspiration on designing the right questions for your survey, check out 60 Decibels’ Remote Survey Toolkit, which groups tried and tested questions into themes — making it easier for you to ask intentional questions based on your business needs and customer experiences.
Being intentional also means strategizing the most effective way of embedding your survey into your existing operations or touchpoints with customers, making the process seamless for both your business and your customers.
One approach is to design a customer journey: identify all of the touchpoints where you already interact with or collect data from customers — for example an intake form, at checkout, a post-sale call. Can you incorporate some of your lean survey questions into these existing touchpoints?
Tracking your impact helps you pitch and communicate more effectively to nonprofit funders or impact investors, but social enterprises often make the mistake of complicating their data because they think it’s what funders want to hear. What should matter most is collecting the data that can effectively communicate the impact you're having on the ground.
Acumen Fellow Farhana Rashid is the co-founder of Bhumijo Limited, a social enterprise which provides well-maintained, hygienic, and financially sustainable public toilets in Bangladesh. When COVID-19 hit, she received the Acumen Emergency Fund to build hands-free, pedal-operated toilets and wash stations for frontline healthcare workers and hospital patients.
Farhana and her team collected the following impact data:
Bhumijo didn’t burden themselves with complicated data — they gathered insights based on what was happening on the ground. They plan to use it as evidence to prove the need for a hands-free system across institutions, and scale opportunities for growth across the private and public sectors.
When pitching to busy funders, you want to quickly show how your impact will unfold. Keep your summary of impact simple and to the point. Show how you’ll get from A to B and deliver your promise of creating positive change.
The process shouldn't be complicated. Funders know that impact doesn’t just happen; there are various factors at play that need to be considered and addressed. Sharing your thinking in advance through a theory of change can help funders see your vision in a simple way.
A theory of change is a useful approach for mapping out not just how an intervention or program will work, but why it will work. It uncovers the proposed story for how you will create change, for whom, over time. It shows:
Between these 4 components come your assumptions: the things that will have to happen in order for your impact to be achieved. For example, students will need to be available after school to meet with mentors.
Mapping out these assumptions will help you identify the kinds of data you should be measuring — such as the number of students staying after school — to know you’re on track towards achieving your longer-term impact.
Simple data collection helps you be nimble, allowing you to assess what’s happening quickly, and take action right away.
Carol Kariuki is the Co-Founder and CEO of GreenPot Enterprises Limited, Kenya’s first fully integrated bamboo company. The enterprise has big plans to grow their network of smallholder farmers, but Carol had hesitations about surveying and collecting data from farmers at the start.
“We thought that until we started buying their bamboo we really shouldn’t engage with them too much or ask them for feedback because there was no income going back to them,” she explains.
It wasn’t until Carol joined the Acumen and Ikea Social Entrepreneurship East Africa Accelerator that she learned about building customer hypotheses and customer surveys.
“The process of coming up with a hypothesis, and structuring the survey around getting proper feedback from the farmers was really an eye opener for us,” she says.
Following the survey, the team found that at least 60% of farmers were willing to increase their bamboo establishment. This came as a surprise, as the team had always felt the farmers’ lands were too small and they wouldn’t be interested in growing more bamboo until they received income from current plots.
“We got valuable insight in terms of how to better engage with the farmers, what was important to them, what they needed from us in order to get more comfortable with the idea, and what kind of training they needed to improve their own expertise for the next phase of bamboo growth,” says Carol.
Most significantly, these insights have moved Greenpot into taking action. “After asking for honest feedback, and doing something about it, we’ve come up with ways to improve our work — refining our impact strategy and the future of what our organization will be about.”
The key to taking action is to be flexible, and understand that a theory of change and other frameworks are meant to be iterative.
While it’s easy to get stuck in the data collection process, taking a human centered approach will help you design new solutions informed by feedback. This lets you adapt quickly and make decisions that lead to greater impact.
Take the theory of change and list of assumptions in the mentorship program example above: If you measure and notice that the number of students staying after school is low, then you’ll know it’s time to take action and fix what’s not working — why aren’t students staying after school? Do you need to do better outreach to families? Do you need to make the mentorship program more engaging?
Another way to stay flexible is through a systems practice approach, which helps you map complex systems to gain clarity and adapt to changes in your work. For example, it can help you better understand the communities in which these students live, identify specific points where you can drive change in their lives, and make adjustments based on your discoveries.
Getting data about some of your main assumptions, or mapping out systems to better understand your customers, can help you identify the most critical places where your overall model might need to be changed or redesigned in real time — before you get months or years down the line.
Impact measurement is a broad and deep topic. As you continue learning, here is a list of courses and resources we recommend you explore.
To learn more about existing frameworks and to gain inspiration from other social innovators, sign up for our Social Impact Analysis course and discover more ways to effectively collect and measure your impact.
Further explore lean data methodologies, access available resources, and read the latest research from impact measurement company, 60 Decibels or consider enrolling in our Lean Data Approaches to Measure Social Impact course to learn about Acumen's Lean Data framework and how to apply it to your work.
Hear from IDEO.org, the pioneers of human-centered design, in our Introduction to Human Centered Design course and learn about how to gather insights, pivot projects based on community needs, build a prototype, and put your plan in place to create lasting impact.
For help defining your theory of change, check out our free course Theory of Change for Brand Communications.
For more ways to grow your knowledge as a social innovator, explore Acumen Academy’s Course offerings.