As a social innovator, you’re likely measuring and reporting on the social impact of your interventions in some capacity — but how meaningful is the information you’re collecting?

According to the authors of The Power of Lean Data, the practice of impact measurement in the social sector is coming up short. While we emphasize reporting our outputs — such as the number of lives reached, or number of jobs created, we largely fail to capture deeper insights about our work — how customers experience our goods and services, if our products are making a difference, and whether we’re reaching the people who need us most. 

If you’re a social enterprise or nonprofit with limited resources, you may have hesitations around data collection. You may be thinking it’s a hard, slow, and expensive process. Impact measurement doesn’t have to be complicated to be effective. When done right, the task can be simple, attainable, and useful.

In this guide, you’ll:

• Learn what impact measurement is and why it’s important 
• Overcome your hesitations around impact measurement   
• Build frameworks for implementing impact measurement tools that drive action

You’ll hear from Julia Mensink, Acumen’s Head of Impact, from Sasha Dichter, Co-Founder of impact measurement company 60 Decibels, and from our community of social entrepreneurs including John Nyagwencha, CEO of Aqua Clara Kenya (ACK), Carol Kariuki, Co-Founder and CEO of GreenPot Enterprises Limited, and others who use impact measurement tools to drive actionable change.

Here are five attributes of effective impact measurement:

• Bottom-up: Listen to customers and provide actionable insight based on their needs and interests
• Useful: Yield sufficient quality data to support decision-making
• Iterative: Allow for learning, adaptation, and replication
• Light-touch: Use low-cost tools and technologies that require minimal time and money
• Dynamic: Enable rapid data collection within a fast-changing environment

Read below about four reasons why impact measurement matters and tips to get you started:

When trying to make decisions, your data can help you answer questions such as: 

• What services or programs should I keep investing in, versus what should I abandon because it isn’t having the impact I intended? 
• How can I listen to end-customers and beneficiaries, take their views into account, and improve products, services, and programs?
• How can I systematically collect impact data and make decisions based on it?
• What decision will put my customer, and serving their needs, at the center? 

For example, Justine Nyaruri Abuga is the CEO and Founder of Ecobora, a pioneer within the clean energy space. When the social enterprise introduced a solar boiler for school kitchens in rural Kenya, Justine and his team developed a lean and rapid customer discovery experiment. Working together with impact measurement company 60 Decibels, their goal was to test a new pay-as-you-go method of financing. 

They reflected on a key customer hypothesis: schools found their hardware costly. Determined to learn more about their customer’s pain points, the team expanded their interviews to include students, teachers, cooks, and board members to explore more affordable ways for schools to access their solar boiler. 

As a result of the experiment, Ecobora decided to shift to a pay-as-you-go financial model. “Our customer discovery experiment was an eye-opener. The feedback was overwhelming! 90% preferred the pay-as-you-go for energy approach so that they didn’t have to own the boiler hardware,” said Justine.

This new opportunity is mutually beneficial for schools and Ecobora. The schools can pay only for what they need, while Ecobora can use their success to pursue new partnerships such as The World Bank, which is interested in learning more about the company’s technology and impressive impact thus far. 

Without focusing on capturing the voices of many stakeholders through customer interviews, and designing their business model with customers at the center, Ecobora wouldn't have been able to make such a clear decision to offer pay-as-you-go and open new avenues for success. 

GET STARTED WITH YOUR CUSTOMERS

Collecting valuable data to make decisions starts with listening to your customers. The insights you gather from them will help you understand how your business creates meaningful change in the lives of the people you serve. 

“Most impact investments aim to reach underserved populations, and yet impact investors and businesses struggle to get reliable data on exactly who these customers are. This matters a lot if you’re trying to understand things like what price points will be affordable for a new solar home system, whether financing will be required for a water-irrigation solution, and how to interpret the financial returns of two competing impact investing funds,” says Sasha Dichter. 

Acumen Academy’s Social Impact Analysis course teaches how to define your impact hypothesis, also known as an impact model. An impact hypothesis outlines your theory of the positive social impact that takes place when people engage with your product or service.

Design your hypothesis by defining the focus, depth, and breadth of your impact:

1. Focus: Who are you serving?
2. Depth: What changes are happening in their lives because of your product or service? How much do you think your customers’ lives are changing because of your product or service? For how long?
3. Breadth: How many total lives are you changing in a positive way?

Before speaking to your customers, you should identify the data that is most useful to you. This can be determined by breaking down the process into three steps:

1. Write your hypothesis statement (answer the three questions above)
2. Identify what data is needed to validate (or confirm) that your hypothesis is true
3. Determine how this new knowledge will inform your upcoming business decisions

Don’t get intimidated by big words. You’re doing the work and you have the experience. You already have a sense that the work you’re doing — the service or product you’re providing — makes a difference in people’s lives.

It’s about having a customer-centered approach to measuring your social impact: starting with your customers, learning to understand them, and using the everyday tools you have to contact them to try and understand what’s going on in their lives and the changes they experience as a result of your products or services. 

A lean data approach to measuring social impact, like the one used by IkamvaYouth, doesn’t have to be expensive to be effective. Acumen Academy’s course on Lean Data Approaches can help you identify the right technology to easily collect data and still gain high-quality customer insights that help you drive important business decisions. 

Acumen Fellow John Nyagwencha is the CEO of Aqua Clara Kenya (ACK), a social enterprise producing water purification tech for rural communities. In a country where 40% of the population has no access to clean water, John explains how customer discovery helped the company stay true to its mission of serving rural populations. 

“One of the biggest challenges we face is how to stay focused on this target market. It’s much easier for me to run Aqua Clara targeting middle-income urban families and institutions. If you’re running a business you have to stay afloat and pay your bills, so there are temptations to target higher-paying customers” explains John. 

As a participant of the Acumen and Ikea Social Entrepreneurship East Africa Accelerator, John worked with 60 Decibels and designed customer interviews to understand the kind of impact they were having on rural communities.

Following the interviews, he learned that 60% of customers are accessing ACK’s solution for the first time, and 91% said if they didn’t have this option, they would have no other alternative for water sanitation products. For a company focused on providing access to those who otherwise wouldn’t have it, this insightful measure of impact assured John that his company was reaching the right customers.

John’s advice to social enterprises looking to validate whether they’re on the right path is to ask: “What have you been set up to do, and are you able to do that? You could be successful from a monetary perspective, you could be profitable, but are you solving the problems you set out to solve?”

To nail these questions, consider designing a value proposition canvas which maps out your customers’ pains and gains, and helps you determine where your organization can be most effective. Acumen Academy’s free Lean Startup Principles course teaches how to create and test hypotheses to refine your value proposition and stay focused on what matters most. 

GET STARTED BY BEING INTENTIONAL

Like John, being intentional by collecting small insights can help you stay focused on your primary mission. 

Head of Research Operations at 60 Decibels Ashley Speyer says find the intentionality of your data collection by asking yourself: 

• What questions do you want to ask your customers?
• Who is this useful for? 
• Are you asking questions to inform your marketing, product design, or other aspect of your business? 

Once you define your internal reasons for collecting data, design simple questions using a lean survey approach:

1. Drive towards decisions: What do you want to know, and why? Make sure your questions are relevant and actionable: what will you do with the information you collect? 
2. Focus on your end user: Get clear on who you want to take your survey. Who is your target customer? What might their prior experiences be with surveying and data collection? How can you make the process of taking your survey enjoyable?
3. Make it rigorous: Focus on gathering reliable data. This will require you to check your biases and remove them from any survey questions. Avoid asking leading questions or overly broad questions. 

For inspiration on designing the right questions for your survey, check out 60 Decibels’ Remote Survey Toolkit, which groups tried and tested questions into themes — making it easier for you to ask intentional questions based on your business needs and customer experiences. 

Being intentional also means strategizing the most effective way of embedding your survey into your existing operations or touchpoints with customers, making the process seamless for both your business and your customers. 

One approach is to design a customer journey: identify all of the touchpoints where you already interact with or collect data from customers — for example an intake form, at checkout, a post-sale call. Can you incorporate some of your lean survey questions into these existing touchpoints?

Acumen Fellow Farhana Rashid is the co-founder of Bhumijo Limited, a social enterprise which provides well-maintained, hygienic, and financially sustainable public toilets in Bangladesh. When COVID-19 hit, she received the Acumen Emergency Fund to build hands-free, pedal-operated toilets and wash stations for frontline healthcare workers and hospital patients. 

Farhana and her team collected the following impact data:

• At least 450 doctors and nurses used the toilet units per day 
• Almost 900 patients and attendants mostly from low income communities benefited from 3 toilet units per day
• 2500 users benefited from 10 hands-free washing units per day

Bhumijo didn’t burden themselves with complicated data — they gathered insights based on what was happening on the ground. They plan to use it as evidence to prove the need for a hands-free system across institutions, and scale opportunities for growth across the private and public sectors.

GET STARTED BY KEEPING IT SIMPLE

When pitching to busy funders, you want to quickly show how your impact will unfold. Keep your summary of impact simple and to the point. Show how you’ll get from A to B and deliver your promise of creating positive change.

The process shouldn't be complicated. Funders know that impact doesn’t just happen; there are various factors at play that need to be considered and addressed. Sharing your thinking in advance through a theory of change can help funders see your vision in a simple way. 

A theory of change is a useful approach for mapping out not just how an intervention or program will work, but why it will work. It uncovers the proposed story for how you will create change, for whom, over time. It shows:

• Inputs: the resources you'll contribute, such as a mentor training program
• Outputs: what your inputs will produce, such as trained mentors
• Outcomes: an effect your program has on the people or issues you serve, such as increased mentorship for rural youth  
• Impact: the long-term or indirect effects of your outcomes, such as increased student retention rates as a result of mentorship

Between these 4 components come your assumptions: the things that will have to happen in order for your impact to be achieved. For example, students will need to be available after school to meet with mentors. 

Mapping out these assumptions will help you identify the kinds of data you should be measuring — such as the number of students staying after school — to know you’re on track towards achieving your longer-term impact. 

Carol Kariuki is the Co-Founder and CEO of GreenPot Enterprises Limited, Kenya’s first fully integrated bamboo company. The enterprise has big plans to grow their network of smallholder farmers, but Carol had hesitations about surveying and collecting data from farmers at the start. 

“We thought that until we started buying their bamboo we really shouldn’t engage with them too much or ask them for feedback because there was no income going back to them,” she explains.

It wasn’t until Carol joined the Acumen and Ikea Social Entrepreneurship East Africa Accelerator that she learned about building customer hypotheses and customer surveys. 

“The process of coming up with a hypothesis, and structuring the survey around getting proper feedback from the farmers was really an eye opener for us,” she says. 

Following the survey, the team found that at least 60% of farmers were willing to increase their bamboo establishment. This came as a surprise, as the team had always felt the farmers’ lands were too small and they wouldn’t be interested in growing more bamboo until they received income from current plots. 

“We got valuable insight in terms of how to better engage with the farmers, what was important to them, what they needed from us in order to get more comfortable with the idea, and what kind of training they needed to improve their own expertise for the next phase of bamboo growth,” says Carol. 

Most significantly, these insights have moved Greenpot into taking action. “After asking for honest feedback, and doing something about it, we’ve come up with ways to improve our work — refining our impact strategy and the future of what our organization will be about.” 

GET STARTED BY BEING FLEXIBLE

The key to taking action is to be flexible, and understand that a theory of change and other frameworks are meant to be iterative.

While it’s easy to get stuck in the data collection process, taking a human centered approach will help you design new solutions informed by feedback. This lets you adapt quickly and make decisions that lead to greater impact. 

Take the theory of change and list of assumptions in the mentorship program example above: If you measure and notice that the number of students staying after school is low, then you’ll know it’s time to take action and fix what’s not working — why aren’t students staying after school? Do you need to do better outreach to families? Do you need to make the mentorship program more engaging?

Another way to stay flexible is through a systems practice approach, which helps you map complex systems to gain clarity and adapt to changes in your work. For example, it can help you better understand the communities in which these students live, identify specific points where you can drive change in their lives, and make adjustments based on your discoveries.

Getting data about some of your main assumptions, or mapping out systems to better understand your customers, can help you identify the most critical places where your overall model might need to be changed or redesigned in real time — before you get months or years down the line.