This session was facilitated by Associate Professor Jo Barraket and presented by Dr Belinda Luke, Associate Professor Aziz Islam, Dr Wendy Scaife, and Dr Craig Furneaux of Queensland University of Technology.[1] The purpose of the session was to consider academic research on social impact measurement, with a specific focus on different stakeholder needs.

The session began with Belinda presenting a brief overview of the measurement tools available, and those commonly adopted, including SROI, social audit and accounting, and logic (or logframe) models.  While there are similarities and differences among these approaches, two recurring features are the internal learning processes associated with undertaking these approaches, but also the lack of comparability (both within organisations over time, and across organisations), in adopting these approaches.  Valuable internal learning was an issue highlighted from previous SIMNA Brisbane Branch meetings which focused on cases where SROI or social audit and accounting had been adopted, and highlighted the feedback and guidance available both to those within the organisation (internal stakeholders), as well as specific external stakeholders (e.g. funders). However an issue which does not seem to have received sufficient attention is the lack of comparability relating to these reports, which limits their usefulness to a broader audience.  This is particularly concerning given the increasingly competitive environment for funding amongst third sector organisations, and the significant resources (time and money) which are required to undertake SROI or social accounting and auditing processes. Reference was made to a Linked-in discussion started by Martina regarding the importance of language in measuring impact.  Avoidance of ‘converging the wrong shorthand’ and ensuring metrics remain language to describe rather than just distil impact were acknowledged as important.  However concerns regarding interpretation within measurement, and the question regarding translation being an art versus technical process also raise important issues.  By way of comparison, ‘rigour’ has become an emerging term in social impact measurement, yet in accounting we focus on reliable measurement, and if something can’t be measured reliably, we don’t put a value on it.  In an annual report for example, items which can’t be measured reliably appear in the notes to the financial statements.  Accountants also don’t consider profit and loss calculations, a one page summary of performance, an ‘art’.    Hence, the lack of comparability and absence of succinct summaries of performance and position are two issues which require attention for third sector organisations. Specific issues to be considered in future approaches to make reporting of social impact more meaningful include a structured, comparable format; reliable measurement; analysis at different levels (e.g. micro or local, meso or regional, macro or national); and practical guidelines accessible for third sector organisations of all sizes, given many are small and resource constrained. Essentially what makes measurement of social impact complex is the dual objectives of money and mission, together with the wide range of stakeholders involved.  This latter issue was considered by Aziz from an external stakeholder perspective, Wendy from a philanthropic perspective, and Craig from an internal stakeholder perspective. Aziz provided an overview of his research relating to disclosure of social value within three social enterprises – Grameen Bank, Charity Water, and the Bill and Melinda Gates Foundation.  While each organisation provided narrative accounts of the work undertaken and associated outcomes and impacts, formal structured reporting processes varied.  Statements of financial, social, and environmental (where relevant) performance were provided.  Social audit reports were not consistently provided (over a three year review period from 2007 to 2009), and tools such as SROI, GRI, AA1000, and ISO guidelines were not adopted by any of the three organisations.  Given each organisation is unique in terms of mission and specific forms of value creation, comparison of performance was difficult.  However addressing this issue and expressing performance in some common terms is an important next step for the development of social enterprise capital markets, where investors who want to make social investments can do so via efficient and informed decision-making processes. Wendy discussed the challenge of communicating impact in an environment where many not-for-profits and foundations have traditionally relied on an established reputation of ‘doing good’ rather than communicating actual performance outcomes and impacts.  A changing landscape of funding and competition for funds means many third sector organisations are re-thinking their approach to measuring and communicating performance. But an interesting prior issue for the supply side of funding is where do these pressures for measurement come from in the philanthropy sector? Are foundations measuring more and if so why.  Issues arise as well for philanthropic funding sources in terms of the purpose of measurement and communication they ask grantees to undertake.  For example, demonstrating accountability may involve quite different processes to replicating measurement for longitudinal data so that changes in performance can be evaluated over time.  The costs versus benefits of measurement remains an important issue, particular for organisations with limited funds and a priority on addressing social issues (rather than investing in and refining their reporting issues).  Another issue to consider is the implications or consequences of measurement, positive and negative; intended and unintended.  A final issue raised was that of who or what is actually being measured and whether philanthropic funders measure their own processes and outcomes as well as those of their grants and grantees. Craig examined contexts where measurement could provide valuable information to internal stakeholders, whilst also serving the needs to external stakeholders.  Tenders for contracts which require a demonstration of ‘value’ from a social perspective, and social procurement where services are sourced with a specific consideration of both the economic and social value being generated are two areas gaining attention.  However, they also require a degree of comparability in order for evaluations to be made across projects and / or organisations.  One way of doing this is through a cost-benefit analysis – a process which is widely adopted in both the public and private sectors.  Focusing specifically on employment-related costs and benefits, Craig presented an overview of research examining this issue, noting that the methodology and formula will be made publicly available, as well as more ‘user friendly’ cases describing how government purchased services from social enterprises in order to generate employment. As part of the open forum at the end of the session, comments referred to the presentations as provocative (hopefully in a good way), and raising issues which will bring maturity to what remains a relatively young and evolving area for both practitioners and researchers.  Comments were raised on the value of rich narratives through case studies, and while this richness is acknowledged, structured summary information is perhaps necessary alongside case study data to develop comparability.  Jo highlighted the potentially valuable role of universities in consolidating and disseminating different approaches to measurement through an online repository, as knowledge sharing is an important process to develop this area, and universities are well placed to undertake this role, in co-operation with practitioners. The opportunity for new topics and presenters for future sessions was raised with the group.  If you would like to nominate a topic or presenter for future SIMNA Brisbane Branch meetings, please contact info@simna.com.au