In our first SIMNA Award winner Q&A we speak to Anne Hampshire, Head of Research and Advocacy at The Smith Family, about how and why The Smith Family measure their social impact. The Smith Family received the SIMNA Award 2016 for Excellence in Social Impact Measurement. Award winners were announced in April 2016 at the Think Outcomes conference and the Excellence in Social Impact award was sponsored by Think Impact.

Why does The Smith Family bother measuring outcomes? What does it mean to the organisation?

As an organisation whose mission is to improve the educational outcomes of disadvantaged children and young people, it’s essential that we’re able to assess what difference we’re making in the lives of the children, young people and families we support. We owe it to those we’re working with. We also raise a lot of funds from individual Australians, philanthropy, corporates, and other institutions, as well as being contracted by governments. We owe it to all these stakeholders that we’re able to measure outcomes and show that funds are being wisely spent.

In a nutshell, how does The Smith Family measure social impact?

We use Outcomes (Results) Based Accountability for all of programs, which involves collecting data which allows us to answer three key questions: How much did we do? How well did we do it? Is anyone better off?

We track a range of short term outcomes, such as reading age, self-efficacy, knowledge of careers, social networks and these are the foundations for the three long term outcomes we’re tracking. The latter are school attendance and completion and post school engagement in employment, education and training. We have a longitudinal data base which collects administrative, demographic and outcomes data for the 34,000 disadvantaged children and young people we’re supporting through our Learning for Life scholarship program. This data is analysed to understand progress over time that is made by individuals, sub-groups, and at the aggregate level.

Has measuring outcomes made a difference to your organisation and stakeholders? How?

Over the past four years The Smith Family has seen year on year improvements in the three longer term outcomes of school attendance and completion and post school engagement in employment, education and training. This is in response to what we’ve learnt from analysing data.

Outcomes measurement is driving program refinements and continuous improvement, for example:

• More tailored support for particular groups of students and at particular times (e.g. those struggling with school attendance, transitioning to high school, Aboriginal and Torres Strait Islander background students)
• Changes to frequency and nature of engagement with families
• Re-defining roles of staff, role specialisation
• Greater focus on supporting students to complete Year 12
• Training for staff on working with highly disadvantaged families
• Refined induction program for new staff

Measuring outcomes and showing the effectiveness of our work has also contributed to securing new funding to allow for program expansion.

How did you start measuring outcomes?

We identified an outcomes map – including short and longer term outcomes – using the OBA approach. We enhanced internal capacity – people, systems and processes – to measure outcomes and we embedded a focus on program effectiveness as central to our work, including through our 5 year strategic plan.

Has anything emerged from the process of measuring outcomes that has surprised you?

The level of nuanced understanding that can be gained from having a large data set that is systematically analysed over time has meant that we are now able to provide more appropriate support for different groups of children and young people on our program. It has been a delight – but not a surprise – to see how the whole of the organisation has been focused on measuring outcomes. This has included for example program and operations staff, research, IT, human resources, fundraising and at every level of the organisation – from the Board, Executive and right through to the frontline.

How do you share your results with stakeholders and the public?

We report results in a range of ways. This includes in our annual report, our research reports, at conferences and public forums, through fundraising material and in the media.

What is the next phase of outcome measurement for The Smith Family?

To continue our unique longitudinal work and to use the increasingly rich data we are collecting to better understand what works to improve the educational outcomes of disadvantaged young Australians. We are very keen to use our data and the new knowledge we’re gathering to inform not only our work but to share it widely so that it can inform public policy and the work of others.

What advice do you have for organisations that are thinking about measuring their impact?

For mission focused organisations, the work of measuring impact is critical. We owe it to those we are working with that we know what difference we’re making. The work of measuring impact works best I think when it’s a whole of organisation focus – not just that of a research or data team. Build it into the systems, processes and communication of the organisation. Regular communication with staff and training and supporting them to be involved in the process is critical – staff want to know they’re making a difference but measuring impact will take some of their time and effort. Don’t think of measuring impact as something that you do for 3 or 6 months and then you move on. It’s got to be for the long haul. Be prepared to have to change some of your way of working off the back of what you learn from measuring outcomes. A focus on measuring outcomes should go hand in hand with continuous improvement. It can be done and if done well will bring great energy and satisfaction.

How does it feel to have received the Excellence in Social Impact Measurement Award for the work you are doing – and how did you celebrate this success?

The Smith Family was delighted and humbled to be acknowledged by this Award and we shared this news widely across the organisation and with our stakeholders as they had all contributed to this achievement.

Would you encourage other organisations to nominate themselves for the next SIMNA Awards?

Yes. Measuring outcomes is a key part of the work of a range of organisations and being able to document efforts being made in this area is an important recognition of that. Being part of such a movement will also hopefully encourage others to become involved too.

Congratulations to The Smith Family for taking home the Excellence in Social Impact Award and thank you to award sponsors Think Impact.