This page is available as a printable brochure: How CSIRO ensures it delivers impact.
For a hundred years, CSIRO has been conducting research to address the scientific problems facing industry and the nation. The CSIRO Strategy increases the focus on the translation of science and technology into products and services to enhance the wellbeing and prosperity of Australia. This has deepened the need for the organisation to improve our understanding and management of how we can follow impact pathways to translate our research into environmental, social and economic benefits.
While CSIRO has always planned its research and evaluated its impact, it is only recently that CSIRO has taken a consistent, organisation-wide approach. Since 2010, CSIRO has developed and implemented an organisation-wide impact framework to consistently plan, monitor and evaluate the impact of its research. This approach allows for a more comprehensive and complete view of the impact being delivered and supports the management of impact of a large portfolio of research.
The main drivers behind CSIRO's increased interest in measuring research impact are to improve its:
- Analysis – the opportunity to better understand and maximise research impact
- Allocation – to enable more informed decision making
- Adaptation – to inform shifts and changes required to pathways, culture and capability maturity
- Alignment – to ensure strategic connection between goals and delivery
- Accountability – the ability to provide defensible, robust evidence of impact
- Advocacy – an increased capacity to articulate future and delivered impact, and
- Acclaim – to compare and recognise the value that is delivered.
Impact and evidence based planning, monitoring and evaluation is an intrinsic element of the CSIRO's Planning and Performance Framework. Impact management relies on a two-way engagement with all factors along the impact pathway to ensure the research is relevant, realistic and that risks are identified and mitigated.
CSIRO's Impact Framework
CSIRO’s approach to impact management assumes that in order to understand the value of research, it must be possible to track the process by which research translates into benefits in the real world. Planning, monitoring and evaluating impact is based on a theory of change model called program logic.
The CSIRO’s Impact Framework is based on the hypothesis that the process of creating impact begins with deploying inputs, to conduct research activities and produce outputs, which themselves are translated through short to medium term outcomes into long term impact.
The input to impact model (see Figure 1) involves a systematic grouping of information types. For the sake of simplicity the model depicts a simple linear process to aid discourse, but it is understood that science is serendipitous and agile in execution, with multiple feedback loops and engagement at all stages. Therefore the framework is operationalised in the same manner.
Figure 1: CSIRO’s Impact Management Framework
Impact planning and management hierarchy
CSIRO’s Business Units are the core vehicle for research impact delivery. They integrate all activities, from capability development, through to science delivery, and client and commercial engagement. Business Unit goals specify the nature of science to be undertaken within the context of broader national challenges and research priorities. A Business Unit refines how it will achieve its goal through the articulation of the broad areas of impact it is planning to deliver.
Within CSIRO the articulation of a Business Unit’s future intended impact, characterised using the program logic and described in Impact Statements, is undertaken at the Program level. Impact Statements (which summarise several impact pathways developed at a lower level closer to the research) are aggregated up to describe achievement of Program and Business Unit goals.
Programs typically have between three to five impact statements. Programs are responsible for developing and managing the impact pathways for the portfolio of projects they manage (see Figure 2). This approach provides a clear link between projects and Impact Statements, and between Impacts Statements, Impact Areas and Business Unit goals. A simplified ‘Impact Strategy on a Page’ is also created annually for each Business Unit, allowing visualisation of these impact connections.
Figure 2: CSIRO’s Impact planning and management hierarchy
Impact evaluation
To complement its planning hierarchy, CSIRO has developed an Impact Evaluation Guide that provides a series of impact evaluation principles, along with a standardised cost benefit approach, to be used for both ex ante (forecasting) and ex post (after the fact) impact evaluations. Use of the Impact Evaluation Guide ensures the comparability of impact evaluation findings and reports, regardless of where in CSIRO the evaluation is being conducted, the field of science or the area of impact.
Impact culture
Supporting impact management practice, CSIRO is also designing and delivering an enterprise wide and enduring cultural program to support long term improvement in our outcome focus, customer service and impact delivery. Impact management is to be understood, valued, prioritised and rewarded as “the way CSIRO does business”. CSIRO’s impact culture program includes considerations of the ‘people’ aspects of how science is managed for impact, including:
- Strong leadership and accountability requirements relevant to role.
- Capability building (including training and learning and development courses).
- A focus on evidence of outcomes in performance conversations at all levels – not just inputs or compliance.
- Linking organisational efforts to corporate outcomes through setting specific, measurable, achievable and time‑bound outcomes and ensures teams and individuals are accountable for, and own, them.
- Individual and group training in CSIRO’s Impact Framework is available to all Business Units and Enterprise Support Services groups.
The benefits of CSIRO’s approach to impact
CSIRO
- Systematic, one-CSIRO approach to impact planning, monitoring and evaluation.
- Capability building to enhance and accelerate the delivery of impact from research.
- Better informed resource allocation.
- Streamlined internal reporting and communication.
Business Unit Leadership teams
- Better informed decision making.
- Improved transparency.
- Improved understanding of the portfolio.
- Systematic, strategic management framework.
Researchers
- Improved line of sight from the research bench to the intended impact.
- Increased awareness of CSIRO’s strategic direction for delivering and demonstrating impact.
- Opportunities to develop skills and competencies around impact.
External stakeholders
- Improved understanding of CSIRO’s impact.
- Greater understanding of the results of investment into research and development .
- Improved transparency.
This page is available as a printable brochure: How CSIRO ensures it delivers impact. PDF (318 KB)
For a hundred years, CSIRO has been conducting research to address the scientific problems facing industry and the nation. The CSIRO Strategy increases the focus on the translation of science and technology into products and services to enhance the wellbeing and prosperity of Australia. This has deepened the need for the organisation to improve our understanding and management of how we can follow impact pathways to translate our research into environmental, social and economic benefits.
While CSIRO has always planned its research and evaluated its impact, it is only recently that CSIRO has taken a consistent, organisation-wide approach. Since 2010, CSIRO has developed and implemented an organisation-wide impact framework to consistently plan, monitor and evaluate the impact of its research. This approach allows for a more comprehensive and complete view of the impact being delivered and supports the management of impact of a large portfolio of research.
The main drivers behind CSIRO's increased interest in measuring research impact are to improve its:
- Analysis – the opportunity to better understand and maximise research impact
- Allocation – to enable more informed decision making
- Adaptation – to inform shifts and changes required to pathways, culture and capability maturity
- Alignment – to ensure strategic connection between goals and delivery
- Accountability – the ability to provide defensible, robust evidence of impact
- Advocacy – an increased capacity to articulate future and delivered impact, and
- Acclaim – to compare and recognise the value that is delivered.
Impact and evidence based planning, monitoring and evaluation is an intrinsic element of the CSIRO's Planning and Performance Framework. Impact management relies on a two-way engagement with all factors along the impact pathway to ensure the research is relevant, realistic and that risks are identified and mitigated.
CSIRO's Impact Framework
CSIRO’s approach to impact management assumes that in order to understand the value of research, it must be possible to track the process by which research translates into benefits in the real world. Planning, monitoring and evaluating impact is based on a theory of change model called program logic.
The CSIRO’s Impact Framework is based on the hypothesis that the process of creating impact begins with deploying inputs, to conduct research activities and produce outputs, which themselves are translated through short to medium term outcomes into long term impact.
The input to impact model (see Figure 1) involves a systematic grouping of information types. For the sake of simplicity the model depicts a simple linear process to aid discourse, but it is understood that science is serendipitous and agile in execution, with multiple feedback loops and engagement at all stages. Therefore the framework is operationalised in the same manner.
Infographic showing CSIRO's Impact framework an the flow of engagement through:
Figure 1: CSIRO’s Impact Management Framework
Impact planning and management hierarchy
CSIRO’s Business Units are the core vehicle for research impact delivery. They integrate all activities, from capability development, through to science delivery, and client and commercial engagement. Business Unit goals specify the nature of science to be undertaken within the context of broader national challenges and research priorities. A Business Unit refines how it will achieve its goal through the articulation of the broad areas of impact it is planning to deliver.
Within CSIRO the articulation of a Business Unit’s future intended impact, characterised using the program logic and described in Impact Statements, is undertaken at the Program level. Impact Statements (which summarise several impact pathways developed at a lower level closer to the research) are aggregated up to describe achievement of Program and Business Unit goals.
Programs typically have between three to five impact statements. Programs are responsible for developing and managing the impact pathways for the portfolio of projects they manage (see Figure 2). This approach provides a clear link between projects and Impact Statements, and between Impacts Statements, Impact Areas and Business Unit goals. A simplified ‘Impact Strategy on a Page’ is also created annually for each Business Unit, allowing visualisation of these impact connections.
Diagram shows a flow chart from Business Unit Goal (aggregate primary outcome domains, scale and timeframe) at the top level, through to Impact Areas; then to Programs including Lead programs and Supporting programs through to Impact statements down to Impact Pathway Planning and finally to Projects level.
Figure 2: CSIRO’s Impact planning and management hierarchy
Impact evaluation
To complement its planning hierarchy, CSIRO has developed an Impact Evaluation Guide that provides a series of impact evaluation principles, along with a standardised cost benefit approach, to be used for both ex ante (forecasting) and ex post (after the fact) impact evaluations. Use of the Impact Evaluation Guide ensures the comparability of impact evaluation findings and reports, regardless of where in CSIRO the evaluation is being conducted, the field of science or the area of impact.
Impact culture
Supporting impact management practice, CSIRO is also designing and delivering an enterprise wide and enduring cultural program to support long term improvement in our outcome focus, customer service and impact delivery. Impact management is to be understood, valued, prioritised and rewarded as “the way CSIRO does business”. CSIRO’s impact culture program includes considerations of the ‘people’ aspects of how science is managed for impact, including:
- Strong leadership and accountability requirements relevant to role.
- Capability building (including training and learning and development courses).
- A focus on evidence of outcomes in performance conversations at all levels – not just inputs or compliance.
- Linking organisational efforts to corporate outcomes through setting specific, measurable, achievable and time‑bound outcomes and ensures teams and individuals are accountable for, and own, them.
- Individual and group training in CSIRO’s Impact Framework is available to all Business Units and Enterprise Support Services groups.
The benefits of CSIRO’s approach to impact
CSIRO
- Systematic, one-CSIRO approach to impact planning, monitoring and evaluation.
- Capability building to enhance and accelerate the delivery of impact from research.
- Better informed resource allocation.
- Streamlined internal reporting and communication.
Business Unit Leadership teams
- Better informed decision making.
- Improved transparency.
- Improved understanding of the portfolio.
- Systematic, strategic management framework.
Researchers
- Improved line of sight from the research bench to the intended impact.
- Increased awareness of CSIRO’s strategic direction for delivering and demonstrating impact.
- Opportunities to develop skills and competencies around impact.
External stakeholders
- Improved understanding of CSIRO’s impact.
- Greater understanding of the results of investment into research and development .
- Improved transparency.