As a social innovator, you’re likely measuring and reporting on the social impact of your interventions in some capacity — but how meaningful is the information you’re collecting?

According to the authors of The Power of Lean Data, the practice of impact measurement in the social sector is coming up short. While we emphasize reporting our outputs — such as the number of lives reached, or number of jobs created, we largely fail to capture deeper insights about our work — how customers experience our goods and services, if our products are making a difference, and whether we’re reaching the people who need us most. 

If you’re a social enterprise or nonprofit with limited resources, you may have hesitations around data collection. You may be thinking it’s a hard, slow, and expensive process. Impact measurement doesn’t have to be complicated to be effective. When done right, the task can be simple, attainable, and useful.

Impact measurement doesn’t have to be complicated to be effective.

In this guide, you’ll:

  • Learn what impact measurement is and why it’s important 
  • Overcome your hesitations around impact measurement   
  • Build frameworks for implementing impact measurement tools that drive action

You’ll hear from Julia Mensink, Acumen’s Head of Impact, from Sasha Dichter, Co-Founder of impact measurement company 60 Decibels, and from our community of social entrepreneurs including John Nyagwencha, CEO of Aqua Clara Kenya (ACK), Carol Kariuki, Co-Founder and CEO of GreenPot Enterprises Limited, and others who use impact measurement tools to drive actionable change.

Close

Impact Measurement: What it is & Why it Matters

Effective impact measurement goes beyond reporting. When you focus your data collection on the customers you serve and the change you’re effecting, impact measurement can help guide your social enterprise to the next level of change-making.  

According to Julia, “What we typically mean when we talk about impact is a positive, lasting effect on people and planet. It’s a change in people’s wellbeing or in the natural environment caused by an organization.” 

Social impact measurement is the collection of data to measure, assess, and report on your impact performance. For social innovators, it means thinking through how a product or service is impactful for people and the environment. 

We’re in the business of trying to serve customers and improve their lives. What we need is the data to understand if we are, in fact, pulling that off.

Sasha Dichter, Co-Founder of 60 Decibels

Actions to boosts your impact

 

Here are five attributes of effective impact measurement:

  • Bottom-up: Listen to customers and provide actionable insight based on their needs and interests
  • Useful: Yield sufficient quality data to support decision-making
  • Iterative: Allow for learning, adaptation, and replication
  • Light-touch: Use low-cost tools and technologies that require minimal time and money
  • Dynamic: Enable rapid data collection within a fast-changing environment

Read below about four reasons why impact measurement matters and tips to get you started:

Reason 1: It Helps You Make Decisions

Impact measurement can teach you a lot about your work and your customers. It can help you make tough decisions between different strategies, and answer important business questions.

Reason 1: Making Decisions

When trying to make decisions, your data can help you answer questions such as: 

  • What services or programs should I keep investing in, versus what should I abandon because it isn’t having the impact I intended? 
  • How can I listen to end-customers and beneficiaries, take their views into account, and improve products, services, and programs?
  • How can I systematically collect impact data and make decisions based on it?
  • What decision will put my customer, and serving their needs, at the center? 

For example, Justine Nyaruri Abuga is the CEO and Founder of Ecobora, a pioneer within the clean energy space. When the social enterprise introduced a solar boiler for school kitchens in rural Kenya, Justine and his team developed a lean and rapid customer discovery experiment. Working together with impact measurement company 60 Decibels, their goal was to test a new pay-as-you-go method of financing. 

They reflected on a key customer hypothesis: schools found their hardware costly. Determined to learn more about their customer’s pain points, the team expanded their interviews to include students, teachers, cooks, and board members to explore more affordable ways for schools to access their solar boiler. 

As a result of the experiment, Ecobora decided to shift to a pay-as-you-go financial model. “Our customer discovery experiment was an eye-opener. The feedback was overwhelming! 90% preferred the pay-as-you-go for energy approach so that they didn’t have to own the boiler hardware,” said Justine.

This new opportunity is mutually beneficial for schools and Ecobora. The schools can pay only for what they need, while Ecobora can use their success to pursue new partnerships such as The World Bank, which is interested in learning more about the company’s technology and impressive impact thus far. 

Without focusing on capturing the voices of many stakeholders through customer interviews, and designing their business model with customers at the center, Ecobora wouldn't have been able to make such a clear decision to offer pay-as-you-go and open new avenues for success. 

GET STARTED WITH YOUR CUSTOMERS

Collecting valuable data to make decisions starts with listening to your customers. The insights you gather from them will help you understand how your business creates meaningful change in the lives of the people you serve. 

“Most impact investments aim to reach underserved populations, and yet impact investors and businesses struggle to get reliable data on exactly who these customers are. This matters a lot if you’re trying to understand things like what price points will be affordable for a new solar home system, whether financing will be required for a water-irrigation solution, and how to interpret the financial returns of two competing impact investing funds,” says Sasha Dichter. 

Acumen Academy’s Social Impact Analysis course teaches how to define your impact hypothesis, also known as an impact model. An impact hypothesis outlines your theory of the positive social impact that takes place when people engage with your product or service.

Design your hypothesis by defining the focus, depth, and breadth of your impact:

  1. FOCUS: Who are you serving?
  2. DEPTH: What changes are happening in their lives because of your product or service? How much do you think your customers’ lives are changing because of your product or service? For how long?
  3. BREADTH: How many total lives are you changing in a positive way?

Before speaking to your customers, you should identify the data that is most useful to you. This can be determined by breaking down the process into three steps:

  1. Write your hypothesis statement (answer the three questions above)
  2. Identify what data is needed to validate (or confirm) that your hypothesis is true
  3. Determine how this new knowledge will inform your upcoming business decisions

Don’t get intimidated by big words. You’re doing the work and you have the experience. You already have a sense that the work you’re doing — the service or product you’re providing — makes a difference in people’s lives.

It’s about having a customer-centered approach to measuring your social impact: starting with your customers, learning to understand them, and using the everyday tools you have to contact them to try and understand what’s going on in their lives and the changes they experience as a result of your products or services. 

lean data image-1

USING LEAN DATA TO UNDERSTAND THE LEARNING EXPERIENCE AT IKAMVAYOUTH

By Masha Lisak and Carmel Farzaneh, Learning & Impact at Omidyar Network

IkamvaYouth staff were stumped. Why were high numbers of learners unable to meet the minimum 75% attendance requirement of their after-school tutoring programs? It was a recurring topic at team meetings, and they were struggling to understand the situation.

The group had brainstormed possible reasons, and had two hypotheses: first, that learners didn't find the program valuable and therefore stopped attending, and second, that compulsory after-school sessions were causing schedule clashes. 

The problem was they had no way of confirming whether these were correct, or the extent to which they impacted learner attendance.

Omidyar Network is a philanthropic investment firm which has supported IkamvaYouth since 2013 and has seen how its programs strive to provide learners from the most disadvantaged South African communities with the knowledge, skills, networks, and resources to access tertiary education or employment opportunities.

To shed light on the dilemma, Acumen’s Lean Data team (now 60 Decibels) joined forces to gather new insights to help the IkamvaYouth team through experimentation principles and low-cost technology to gather data directly from customers.

With research questions clearly defined, the team created and got in touch with those who left the program to learn more about why. Through an online survey with tutors, in-depth phone interviews with learners, dropouts, tutors, and parents, and IVR temp-check trials which allowed learners to give feedback within 24 hours of their tutoring sessions, the IkamvaYouth and Lean Data team finally had some clues.

The results surprised the IkamvaYouth team, who found that across both active learners and dropout learners, their original hypothesis that dropouts did not find value in the program was incorrect. With their original hypothesis debunked, the team dug deeper into identifying why learners drop out.

A requirement to participate in the program had been to attend 75% of their tutoring sessions, and when learners didn't meet this bar, staff assumed this was mainly due to a lack of interest. To the contrary, surveys found that many learners were missing the requirement due to legitimate conflicts rather than a lack of interest.

One learner said: "The problem is I live far from Ikamva, so I did not have transportation to attend all the time, and during winter it was too cold to walk..." 

Inspired by these findings, IkamvaYouth is now taking two important steps. First, they’re carefully considering distance from the centers when enrolling new students. Second, they've developed an "Ikamva Lite" model to provide a less intensive program with fewer attendance requirements that can serve learners with scheduling conflicts.

As a next step, the Lean Data team suggested evaluating tutor quality to understand how the volunteer tutors might be affecting student learning. To assess this important piece of their program, they piloted a phone survey to ask learners to evaluate their last session. While many responses were positive, some showed that learners were disappointed by the sessions, leading IkamvaYouth to more actively evaluate their tutors. From this learning, they now have a process for learners to give feedback to tutors right after each session, resulting in greater accountability and strengthening the relationship between tutors and learners.

Bolstered by the findings of the Lean Data project, IkamvaYouth is continuing to learn better ways to support and encourage the next generation of South Africans.

 

A lean data approach to measuring social impact, like the one used by IkamvaYouth, doesn’t have to be expensive to be effective. Acumen Academy’s course on Lean Data Approaches can help you identify the right technology to easily collect data and still gain high-quality customer insights that help you drive important business decisions. 

Reason 2: It Helps You Stay Focused

Impact measurement can feel overwhelming, but building a system to collect data and assess your impact doesn’t have to be complicated. In fact, it can help you stay focused on what matters most: your mission and the people you serve. 

Reason 2; Stay Focused

Acumen Fellow John Nyagwencha is the CEO of Aqua Clara Kenya (ACK), a social enterprise producing water purification tech for rural communities. In a country where 40% of the population has no access to clean water, John explains how customer discovery helped the company stay true to its mission of serving rural populations. 

“One of the biggest challenges we face is how to stay focused on this target market. It’s much easier for me to run Aqua Clara targeting middle-income urban families and institutions. If you’re running a business you have to stay afloat and pay your bills, so there are temptations to target higher-paying customers” explains John. 

[Impact measurement] enables us to stay on the path to ensure the people that really need the solutions that we have are being reached by our business.

John Nyagwencha, Aqua Clara Kenya

As a participant of the Acumen and Ikea Social Entrepreneurship East Africa Accelerator, John worked with 60 Decibels and designed customer interviews to understand the kind of impact they were having on rural communities.

Following the interviews, he learned that 60% of customers are accessing ACK’s solution for the first time, and 91% said if they didn’t have this option, they would have no other alternative for water sanitation products. For a company focused on providing access to those who otherwise wouldn’t have it, this insightful measure of impact assured John that his company was reaching the right customers.

John’s advice to social enterprises looking to validate whether they’re on the right path is to ask: “What have you been set up to do, and are you able to do that? You could be successful from a monetary perspective, you could be profitable, but are you solving the problems you set out to solve?”

To nail these questions, consider designing a value proposition canvas which maps out your customers’ pains and gains, and helps you determine where your organization can be most effective. Acumen Academy’s free Lean Startup Principles course teaches how to create and test hypotheses to refine your value proposition and stay focused on what matters most. 

GET STARTED BY BEING INTENTIONAL

Like John, being intentional by collecting small insights can help you stay focused on your primary mission. 

Head of Research Operations at 60 Decibels Ashley Speyer says find the intentionality of your data collection by asking yourself: 

  • What questions do you want to ask your customers?
  • Who is this useful for? 
  • Are you asking questions to inform your marketing, product design, or other aspect of your business? 

Once you define your internal reasons for collecting data, design simple questions using a lean survey approach:

  1. Drive towards decisions: What do you want to know, and why? Make sure your questions are relevant and actionable: what will you do with the information you collect? 
  2. Focus on your end user: Get clear on who you want to take your survey. Who is your target customer? What might their prior experiences be with surveying and data collection? How can you make the process of taking your survey enjoyable?
  3. Make it rigorous: Focus on gathering reliable data. This will require you to check your biases and remove them from any survey questions. Avoid asking leading questions or overly broad questions. 

For inspiration on designing the right questions for your survey, check out 60 Decibels’ Remote Survey Toolkit, which groups tried and tested questions into themes — making it easier for you to ask intentional questions based on your business needs and customer experiences. 

Being intentional also means strategizing the most effective way of embedding your survey into your existing operations or touchpoints with customers, making the process seamless for both your business and your customers. 

One approach is to design a customer journey: identify all of the touchpoints where you already interact with or collect data from customers — for example an intake form, at checkout, a post-sale call. Can you incorporate some of your lean survey questions into these existing touchpoints?

Reason 3: It Helps You Attract Funders

Tracking your impact helps you pitch and communicate more effectively to nonprofit funders or impact investors, but social enterprises often make the mistake of complicating their data because they think it’s what funders want to hear. What should matter most is collecting the data that can effectively communicate the impact you're having on the ground. 

Reason 3: Attracting Funders

Acumen Fellow Farhana Rashid is the co-founder of Bhumijo Limited, a social enterprise which provides well-maintained, hygienic, and financially sustainable public toilets in Bangladesh. When COVID-19 hit, she received the Acumen Emergency Fund to build hands-free, pedal-operated toilets and wash stations for frontline healthcare workers and hospital patients. 

Farhana and her team collected the following impact data:

  • At least 450 doctors and nurses used the toilet units per day 
  • Almost 900 patients and attendants mostly from low income communities benefited from 3 toilet units per day
  • 2500 users benefited from 10 hands-free washing units per day

Bhumijo didn’t burden themselves with complicated data — they gathered insights based on what was happening on the ground. They plan to use it as evidence to prove the need for a hands-free system across institutions, and scale opportunities for growth across the private and public sectors.

GET STARTED BY KEEPING IT SIMPLE

When pitching to busy funders, you want to quickly show how your impact will unfold. Keep your summary of impact simple and to the point. Show how you’ll get from A to B and deliver your promise of creating positive change.

The process shouldn't be complicated. Funders know that impact doesn’t just happen; there are various factors at play that need to be considered and addressed. Sharing your thinking in advance through a theory of change can help funders see your vision in a simple way. 

A theory of change is a useful approach for mapping out not just how an intervention or program will work, but why it will work. It uncovers the proposed story for how you will create change, for whom, over time. It shows:

  • Inputs: the resources you'll contribute, such as a mentor training program
  • Outputs: what your inputs will produce, such as trained mentors
  • Outcomes: an effect your program has on the people or issues you serve, such as increased mentorship for rural youth  
  • Impact: the long-term or indirect effects of your outcomes, such as increased student retention rates as a result of mentorship

Between these 4 components come your assumptions: the things that will have to happen in order for your impact to be achieved. For example, students will need to be available after school to meet with mentors. 

Mapping out these assumptions will help you identify the kinds of data you should be measuring — such as the number of students staying after school — to know you’re on track towards achieving your longer-term impact. 

Reason 4: It Helps You Take Action

Simple data collection helps you be nimble, allowing you to assess what’s happening quickly, and take action right away. 

Reason 4: Taking Action

Carol Kariuki is the Co-Founder and CEO of GreenPot Enterprises Limited, Kenya’s first fully integrated bamboo company. The enterprise has big plans to grow their network of smallholder farmers, but Carol had hesitations about surveying and collecting data from farmers at the start. 

“We thought that until we started buying their bamboo we really shouldn’t engage with them too much or ask them for feedback because there was no income going back to them,” she explains.

It wasn’t until Carol joined the Acumen and Ikea Social Entrepreneurship East Africa Accelerator that she learned about building customer hypotheses and customer surveys. 

“The process of coming up with a hypothesis, and structuring the survey around getting proper feedback from the farmers was really an eye opener for us,” she says. 

Following the survey, the team found that at least 60% of farmers were willing to increase their bamboo establishment. This came as a surprise, as the team had always felt the farmers’ lands were too small and they wouldn’t be interested in growing more bamboo until they received income from current plots. 

“We got valuable insight in terms of how to better engage with the farmers, what was important to them, what they needed from us in order to get more comfortable with the idea, and what kind of training they needed to improve their own expertise for the next phase of bamboo growth,” says Carol. 

Most significantly, these insights have moved Greenpot into taking action. “After asking for honest feedback, and doing something about it, we’ve come up with ways to improve our work — refining our impact strategy and the future of what our organization will be about.” 

GET STARTED BY BEING FLEXIBLE

The key to taking action is to be flexible, and understand that a theory of change and other frameworks are meant to be iterative.

While it’s easy to get stuck in the data collection process, taking a human centered approach will help you design new solutions informed by feedback. This lets you adapt quickly and make decisions that lead to greater impact. 

Take the theory of change and list of assumptions in the mentorship program example above: If you measure and notice that the number of students staying after school is low, then you’ll know it’s time to take action and fix what’s not working — why aren’t students staying after school? Do you need to do better outreach to families? Do you need to make the mentorship program more engaging?

Another way to stay flexible is through a systems practice approach, which helps you map complex systems to gain clarity and adapt to changes in your work. For example, it can help you better understand the communities in which these students live, identify specific points where you can drive change in their lives, and make adjustments based on your discoveries.

Getting data about some of your main assumptions, or mapping out systems to better understand your customers, can help you identify the most critical places where your overall model might need to be changed or redesigned in real time — before you get months or years down the line. 

Resources to Grow Your Knowledge of Impact Measurement

Impact measurement is a broad and deep topic. As you continue learning, here is a list of courses and resources we recommend you explore. 

To learn more about existing frameworks and to gain inspiration from other social innovators, sign up for our Social Impact Analysis course and discover more ways to effectively collect and measure your impact.

Further explore lean data methodologies, access available resources, and read the latest research from impact measurement company, 60 Decibels or consider enrolling in our Lean Data Approaches to Measure Social Impact course to learn about Acumen's Lean Data framework and how to apply it to your work.  

Hear from IDEO.org, the pioneers of human-centered design, in our Introduction to Human Centered Design course and learn about how to gather insights, pivot projects based on community needs, build a prototype, and put your plan in place to create lasting impact.

For more ways to grow your knowledge as a social innovator, explore Acumen Academy’s Course offerings.