A framework for measuring success
The Monitoring, Evaluation, Reporting and Improvement (MERI) Framework provides the structure and processes for understanding, improving and communicating the impact of managing the region’s natural resources. A MERI Framework is vital for understanding the successes (and failures) of the RCS and its implementation, and how to best learn from them. It also provides the information that is required to adaptively respond and manage the delivery of programs and projects.
RCS MERI is based on the following principles:
- During the life of the RCS new information and learnings will become apparent; wherever possible this information should be incorporated into implementation of the RCS by taking an adaptive management approach.
- Not all impacts of the RCS implementation will be known or measurable during the life of the strategy.
- MERI success is reliant on available and repeatable data sources.
- Strong regional partnerships and collaboration are important for data sharing and creating a whole-of-region story about NRM.
Outcomes framework
An outcomes framework has been developed to provide a consistent statewide approach to monitoring and reporting on condition and management, using agreed outcome indicators. The framework also enables Victorian CMAs to demonstrate how regional outcomes align with statewide policies and outcomes, improving the way that RCSs reinforce, promote and support government policy and objectives.
The outcomes framework is consistent with the Department of Premier and Cabinet’s ‘Outcome Architecture’ hierarchy by having a core set of themes and associated long-term outcomes in all RCSs. Additional regionally derived outcomes and outcome indicators are at the discretion of each CMA. The architecture of the RCS outcomes framework is shown below.
The outcomes framework is based on the following assumptions:
- The RCS is a high-level strategy – it focuses on the vision for the catchment, and the desired long-term outcomes.
- The RCS is a place-based strategy – it articulates the vision of the catchment community and reflects community priorities.
- The RCS delivers integrated catchment management (ICM) – actions can contribute to many outcomes (a linear attribution between outcomes and a government outcome or target cannot be assumed).
- The outcomes framework is not an accountability framework – accountability for delivery of strategy outcomes is aligned to the investment mechanisms (e.g. corporate plans, projects and programs) that government utilises to fund delivery.
Regional outcomes
While guidance has been provided on the application of a statewide outcomes framework, it is at the discretion of each Victorian CMA how success is articulated across landscapes and/or themes. The Glenelg Hopkins RCS was developed using an integration of an asset-based and an adaptation pathways approach to planning. Outcomes and indicators are identified for each asset theme, with priority management directions identified across different adaptation pathway stages for each of these themes.
Long-term outcomes have been identified to be achieved in 20+ years as well as medium-term outcomes (‘stepping-stone’ outcomes) to be achieved in the 6-year life of the strategy. For each medium-term outcome, a number of priority management directions have been identified that will influence the success of the RCS outcomes. They have been identified through extensive engagement processes with regional partners and community representatives and prioritised based on:
- Location and condition of natural resources.
- Threats to, or impacts on, natural resources.
- Prioritisation methods for determining the most cost-effective management actions.
- Methodologies for assessing the effectiveness of management actions.
Additional information on the relationship between outcomes and priority management directions is provided below as a program logic.
Program logic
Program logic is central to this MERI Framework and expectations about the influence of the RCS. Program logic depicts the rationale behind the strategy and the anticipated relationships between the implementation of priority management directions and outcomes. The program logic structure in this strategy comprises the vision, long and medium-term outcomes and priority management directions.
Assumptions about success
Assumptions are the expectations, based on current knowledge and experience, of what is important for a strategy’s success. The following broad assumptions have been applied:
- Funding programs and resources will be available and at a scale that is required to achieve the desired outcomes.
- Baseline data will be available from which progress will be measured.
- Data for measuring change will be available and where required will be repeatable.
- Our knowledge about cause and effect relationships between actions and outcomes is correct and is based on research or experience.
- There will be regional capacity to support the successful implementation of the RCS.
Key evaluation questions
Key evaluation questions form the basis for assessing whether the strategy has been implemented as required and whether it is being effective in bringing about the desired change. The RCS will be evaluated against the themes of effectiveness, appropriateness, impact, efficiency and legacy.
| Themes | Key evaluation questions |
|---|---|
| Effectiveness | To what extent have the management actions been implemented? |
| To what extent were the 6-year outcomes achieved? | |
| Do the management actions remain the best way to achieve outcomes, or are there alternative actions that should be considered? | |
| Appropriateness | Do the priority management directions and outcomes still meet the needs of the region? |
| Are the priority management directions consistent with government direction; including legislation, policies and strategies? | |
| Impact | What is the status and trend in asset condition in the region? |
| What progress has been made towards achieving the 20-year outcomes? | |
| In what ways and to what extent has the RCS contributed to changing management practices? | |
| Efficiency | With reference to project delivery and funding agreements, has RCS delivery been achieved within budget and achieved the highest value from available resources? |
| Legacy | How are the effects of RCS implementation expected to continue over time? |
Reporting schedule
Annual reporting
Glenelg Hopkins CMA reports annually on the condition and management of land and water resources in the region. A component of this reporting considers the status of RCS implementation and will also consider statewide indicators. Each year the results will be presented in the Glenelg Hopkins CMA Annual Report.
Mid-term review
A mid-term review will be prepared on the RCS at the mid-point of its implementation in 2024. Condition will be monitored and reported in the mid-term review, with reference to the statewide consistent set of indicators.
Full-term review
The full-term review will occur once the RCS has reached the end of its implementation on 30 June 2027. The scope of this review will include consideration of the key evaluation questions detailed above. Condition will be monitored and reported in the full-term review, with reference to the statewide consistent set of indicators.
