This is the fifth and final chapter of the toolkit. This chapter and the accompanying video look at:
- how to measure program impact
- how to use sponsorship as a lever for change.
Evaluation vs. impact measurement
It is important to note that evaluation and impact measurement are two different mechanisms.
Evaluation is about tracking and monitoring key determinants of success which lead to an outcome. For a sponsorship program, this includes:
- the effectiveness of the EOI and matching processes
- the effectiveness of the capability sessions and check ins
- the experiences of participants.
Impact measurement is about measuring outcomes. This refers to assessment of the change caused by a particular initiative, policy or program. For a sponsorship program, outcomes include:
- increased career mobility, progression, or improved pathways for CALD employees
- increased visibility of CALD talent
- capability uplift around inclusive leadership for executive sponsors.
How to measure impact of a sponsorship program
To understand how to measure impact, it is important to understand key components of impact:
- the dimensions of impact
- the types of impact
- the principles of impact.
Impact measurement needs to be planned and prioritised before commencing with programming, similarly to evaluation.
The dimensions and types of impact assist with measuring impact from a structured perspective, while the principles of impact show how impact is created and experienced from a qualitative perspective.
Below are the five dimensions of impact, with examples relevant to sponsorship.
- What is the intended outcome?
- Improved career mobility, progression, and pathways for sponsees
- Increased understanding of barriers to progression for sponsors
- Increased pool of CALD talent ready for promotion or other mobility
- Who experiences it?
- Sponsees: The OPSC’s Elevate and Advocate sponsorship program targeted CALD employees with leadership ambitions, from Clerk Grade 7/8 – 11/12 (or equivalent level) or with 10-15 years of experience
- Sponsors: senior executives, mainly non-CALD with some CALD representation
- How much of the outcome is experienced and by when?
- Over the program duration, a CALD sponsee pool is established, with their aspirations and goals clarified
- At the program close, sponsees are ready and supported to move towards their career goal
- Post program, sponsees continue moving towards their goals and attain more senior roles
- What contribution do sponsors or the organisation make toward that outcome
- Sponsors use their voice to promote, encourage and influence other senior leaders to buy in and support the program or become sponsors themselves
- The organisation promotes the program internally and externally, making it visible and accessible to all employees and external stakeholders
- What are the risks to realising this impact
- What does this mean for reporting on impact? It may mean the program achieves only a low or limited positive impact
- Note: Have plans to mitigate program risks, including:
- sponsors not adequately supporting CALD sponsees
- sponsees not clarifying career goals or having a clear career plan
- sponsees not being amplified in professional networks
- sponsors being unable to address barriers to progression
Source: Impact Frontiers (Formerly the Impact Management Project), Five Dimensions of Impact
Thinking about the types of impact can also help with evaluation. There are 6 well known types of impact:
- Cultural Impact:
- Sponsorship relationships contribute to the understanding of shared ideas, realities, values and beliefs
- CALD sponsorship improves cultural competence, respect of different cultures, and increases understanding of cultural safety
- Economic Impact
- Sponsorship can impact economic and financial equity. CALD participants may experience underemployment and career plateaus
- CALD sponsorship can contribute to addressing individual career progression and in some cases, underemployment. However, there is limited scope to impact this on an institutional level within a sponsorship program.
- Wellbeing Impact
- Sponsorship can create a positive impact on wellbeing for sponsors and sponsees
- CALD sponsees are more visible and valued in their workplaces, creating a significant positive impact
- For sponsors, the act of giving and creating impact in another person’s career journey has a positive impact on inclusivity and practice-based inclusive leadership
- Social Impact
- Sponsorship relationships provide a benefit to civil society and social cohesion
- Qualitative surveys show that sponsors and sponsees have a positive impact in their broader communities as result of their sponsorship experiences
- Policy Impact
- Sponsor experiences of advocating for a CALD sponsee improves their understanding of inclusive policy making. This leads to policies which better consider diverse needs, both internally and for the community.
- System Impact
- Sponsors learn to use their influence in the systems of recruitment, selection, onboarding, development and advancement
- As sponsors support a CALD sponsee, they develop a keen and personal understanding of systemic barriers and how to reduce or eliminate these barriers
There are eight principles of impact, which look at impact from a qualitative and experiential perspective.
- Participation
- Is participation fair, equitable, accessible, and respectful?
- Transparency
- Was the process of selecting the cohort transparent?
- Are the expectations of the program and timing clearly outlined?
- Security and safety
- What are the key design elements and protocols to govern participant safety?
- Accountability
- How are sponsors held to account?
- How are sponsees held to account?
- How is the overall Executive Sponsor of the program held to account?
- Credibility
- What gives the program its credibility? Is there expert and evidence-based design, delivery and support?
- Cost efficiency and effectiveness
- What is the return on investment of the program? Consider this against the duration of the program, and any budget or other resource investments.
- Flexibility
- How flexible and agile is the program? For example, how did the program respond to machinery of government changes, internal restructures, sponsor-sponsee fit issues, personal issues or professional issues?
- Practicality
- In practice, how implementable are the ideas, concepts, and ways of working to the agency’s everyday context?
How to influence change in the system
A CALD sponsorship program is a lever for change. Sponsorship positively impacts representation and visibility of CALD leaders over the long term. Program participants and the program’s executive sponsor also become key change agents in addressing structural bias and racism within the workforce.
Insights and data from a CALD sponsorship program can be leveraged to influence broader structural change in the following ways:
- Use evaluation data and impact measurement data to form the basis of a report. The executive sponsor can share this report with senior leadership to highlight the systemic and structural barriers experienced by CALD people in their careers and steps that can be taken to address barriers.
- Collect and de-identify positive impact stories from the program to highlight the power of sponsorship. The outcomes of program participants are data points which can be aggregated to show collective impact. In addition, encourage sponsors to use their learnings from the program to create ongoing impacts within their own areas.
- Use the data and experiences of the cohort to work tactically at hiring, development and promotional processes. This data can also be used to improve recruitment and selection methods and uplift the capability of decision makers. Improving these processes creates scalable, sustainable change.
- Use the data and de-identified experiences of CALD people to highlight risks to cultural safety and psychological safety.
- Align sponsorship programs with other senior leadership programs or a CALD target to amplify and accelerate impact.