Small Slices of a Bigger Pie

There is increasing pressure in public services for providers to prove the impact they are having, and for commissioning to stimulate competition among providers as to who can deliver the most results.

However change in the real world often happens by lots of organisations working in partnership, with the input of people who use their services. Most evaluation techniques ignore the role of partnership working or who claims the credit for change happening. In Social Return on Investment (SROI) it is included, and known as attribution. However, it is a concept that is not well understood, and often poorly calculated. This report develops the methodology and looks at the implications for commissioners of services.

SROI: From Outcomes to Impact

Social Return on Investment is an evaluation technique that is designed to capture social, economic and economic value. It has some roots with cost-benefit analysis but it has a stronger emphasis on social impacts. This means it can focus on what matters to people that use services and measure change in a way that can inform decision making at an organisational level. Whilst SROI has received a lot of attention of late, much of this is focused on how to define and value outcomes. Another important aspect of SROI is that it tries to isolate the impact of a particular intervention or organisation on the outcomes. Important factors in this measurement are:

• Deadweight: A measure of the outcome that would have happened even if the activity had not taken place.
• Attribution: An assessment of how much the outcome in question was caused by the contribution of other organisations or people
• Displacement: An assessment of how much the outcome displaced other outcomes.

This report focuses on one aspect of calculating impact – attribution. However this can only be properly understood by considering it alongside the other elements of impact. Thinking about attribution is sometimes thought of as slicing up a pie. Trying to establish who can claim the credit for the changes that have been achieved.

How do you estimate attribution

Many evaluation techniques do not include any attempt to calculate attribution. Whilst it is challenging to isolate how much of a change one organisation is attributable for, failing to account for it at all means you are effectively assuming that it is 100%. Calculating impact can be less difficult than many people expect. At nef, our experience is that when you ask people, they are actually very good at thinking at what worked for them, and who helped them most.

We break estimating attribution down into four steps that can be summarised below:

Step 1: Understand who is contributing to the change: To get an idea of what your attribution rate can be you need to become an expert in your subject area, and talk to stakeholders.

Step 2: Collect data: Whilst you can get an estimate from talking to a few stakeholder, you need to go further and evidence this without outcomes data.

Step 3: Estimate the attribution rate: Once you have outcomes data you will be in a good place to estimate the attribution rate.

Step 4: Strengthen your understanding of attribution: You may be able to get a more accurate estimate of attribution by varying the rate by outcome, by stakeholder group, or over time.

What does this mean for commissioning?

It is tempting to ignore attribution. Organisations may have a natural tendency to over-claim. It can be difficult to admit that other people have a role to play in achieving impact. However, in reality, organisations that work in partnership and coproduce outcomes may be generating larger outcomes. Public services work better when they fit together as a coherent network, not a multiplicity of rival agencies, none of whom feel responsible for the whole person. They are taking smaller slices, but the size of the pie may be bigger.

Another aspect to consider is how services involve the people that they are trying to affect change in. There is a tendency to try and pay one provider to try and ​‘fix’ people without using the hidden skills of ordinary people. No provider that ignores the resources of the people that use their services can be efficient. Thus one vital aspect of attribution is looking at the links an organisation has with the community and the role of the people that use the service.

Thinking about attribution can be a challenge for commissioners. It is certainly easier for commissioners if they can contract with one organisation who can do it all. Whilst this might work for some goods and services, it will fail in others. If you contract with a printing company, you can guarantee that this company will be able to singlehandedly deliver the results. When the service in question is preventing homelessness, then lots of things need to happen at the same time. Whilst it is easier for one organisation to take all the credit, and for them to be paid for delivering these ​‘results’, change doesn’t usually work like this in the real world.