Background

The measurement culture within the Western Australian Community, Neighbourhood and Learning Centre sector (WA CNLC) could previously be described as output as opposed to outcomes focused. Data was collated for the purpose of satisfying contractual requirements, not for the purpose of service improvement.

Linkwest observed that there was a need for WA CNLC to adopt a methodological approach to planning, evaluating and documenting their performance against preferred customer outcomes. A previously established relationship with two sister organisations in New South Wales and South Australia introduced Results Based Accountability (RBA) as a potential model of impact assessment.

Linkwest was successful in their application to the WA state government for $80,000 to trial the suitability of the RBA framework for WA CNLC. Nine pilot participants, including Linkwest, utilised the Performance Accountability component of RBA to review programs in information and referral, community engagement, service model diversification, social integration, and accredited and unaccredited capacity building. This was the first time there had been a methodological, outcomes based approach to performance review in the WA CNLC sector.

Initial obstacles

A core finding of the pilot project was the discrepancy between the data thought to exist and that actually available for analysis. Consequently, every organisation involved with the pilot project had to implement changes to their data collection processes. Inconsistent data entry and ineffective data management practices were also identified as obstacles to overcome.

An in-house Management Information System database was established within Linkwest, which subsequently helped with the issue concerning data entry. The amalgamation of all previously analysed data into one system has helped to ease reporting requirements. An external company has since been commissioned to develop a bespoke reporting system in line with Linkwest’s requirements.

All pilot participants made changes to their customer satisfaction questionnaires. Revisions were made to ensure sufficient evidence regarding the impact of their service was attained.  Several services became more effective at employing data collection practices due to an improved understanding of their importance.

Early successes

A number of generic processes were introduced to ensure that the required data was available for impact measurement. Examples include:

• Regular team meetings with a specific RBA agenda. These provided a dedicated space for reflection for any activity or action related to the program area under review; and
• The creation of master templates that reflected the core program outcomes. Use of these templates ensured that any future activities align with these outcomes. Each template has the flexibility to be revised, ensuring it remains fit for purpose and that the underlying purpose of outcome review and documentation is fulfilled.

Importantly, a measurement culture has evolved in the WA CNLC sector, which has been the greatest success of all. A number of factors have contributed to this:

• The adoption of a whole team approach to service evaluation and improvement;
• Frequent team meetings to embed a measurement culture;
• The production and availability of good practice procedural documentation, information sheets and master templates designed around RBA principles; and
• The creation of a generic bank of potential WA CNLC performance measures for pilot participants to assist in the transition from outputs to outcomes based thinking.

An internal ‘lessons log’ has also been created. This is regularly updated and reviewed by two of Linkwest RBA ‘experts’ to ensure that the measurement culture within the organisation and across the sector continues to reflect good practice.

Moving forward

Both expected and unexpected changes have occurred within Linkwest as a result of using a model of impact assessment. Expected changes include improvements to the core elements of a program relating to RBA’s ‘how well’ measure. Unexpected changes include a reduction in the frequency of programs, but an increased scope in the remaining provision offered. This unexpected change has resulted in increased opportunities for positive customer outcomes.

The RBA framework has allowed new methods of analysis to be instigated by services, as data is no longer taken at face value. Methods originally perceived as fit for purpose were found not to be measuring their intended purpose. Staff knowledge on research and evaluation has increased exponentially, alongside their ability to apply this knowledge to inform service improvement.

For the first time, Linkwest is using internal benchmarking to appraise service performance. Workplace practices have been adapted to ensure that data is collated, evaluated and reported in the earliest instance. This revised reporting process, also used by other pilot participants, has helped to develop a measurement culture, due to an ability to quickly attain a visual representation of current practices against customer outcome achievement.

Linkwest’s members have benefited directly from this clarity due to the ability to more clearly articulate Linkwest’s service structure and programs. The most important improvement is the understanding Linkwest now has of whether service provision is having an impact on those they support in the WA CNLC sector.

Rebecca Davies, Centre Development Manager, LinkWest