This evaluation policy provides direction for the evaluation of programs, projects, strategies, policies and initiatives across the Department of Education to improve education and training outcomes for children and students, promote strong, inclusive, active communities and support the effective, efficient and appropriate use of resources.
Policy statements in Sections 1 to 5 have been updated to reflect the Treasury Circular TC18-03 Program Evaluation that supersedes and replaces the previous DPC Circular C2016-01 - Program Evaluation. Changes have also been made in response to state government evaluation practices, organisational changes within the department and the Administrative Arrangements (Administrative Changes—Public Service Agencies) Order 2019.Superseded Documents
Evaluation Policy 2010; Evaluation Policy January 2014, Evaluation Policy November 2014Document History
1. Policy statement
This policy reflects a commitment to rigorous, systematic and strategic evaluation as part of the NSW Government Program Evaluation Guidelines. The department is a learning organisation that supports consistent and transparent evaluation of government programs to foster evidence-based decision-making and a culture of continuous improvement.
A program is a set of activities managed together over a sustained period of time that aim to deliver outcomes for a client or client group. ‘Program’ is sometimes used interchangeably with policy, strategy, project, initiative or intervention. Programs may include one or more projects that aim to deliver a specific product or output and achieve a strategic outcome within a specific timeframe and budget. Programs are discrete sets of funded activities that are differentiated from the day-to-day delivery of services, business activities or operational functions that are core business as usual. Core business activities and services and capital projects would generally be outside of the scope of this policy.
Evaluation is a rigorous, systematic and objective process to make judgments about the impacts and merits or worth of a program, usually in relation to its effectiveness, efficiency and appropriateness.
This policy provides direction for program evaluation. The purpose of this policy is to support the role of quality evaluation in improving the education and training outcomes for children, students, and other stakeholders such as employers in workplace training, parents and carers, promoting strong, inclusive, active communities, and supporting the effective, efficient and appropriate use of resources.
Evidence from evaluations can be used to plan and develop policies and programs, improve implementation, make judgments about outcomes and impacts, and inform the allocation of resources. Evaluation supports evidence-based decision-making, systems improvement, accountability and successful innovation.
Contexts for evaluation include evaluations initiated by schools, divisions and entities, as well as evaluations required by funding bodies or external program requirements. Each context should be guided by the general principles of evaluation.
1.2. General principles of evaluation
Evaluation will be planned early during the design of programs to ensure that programs can be evaluated and to increase the robustness of the evaluation.
Evaluation will be appropriately resourced as part of program design, taking into account what is feasible and realistic to achieve within time and budget constraints. A robust evaluation design may use both quantitative and qualitative methods including the use of existing administrative data sets and information systems and collecting data from clients.
Evaluation will be rigorous, systematic and objective, with appropriate scale and design. Evaluations should be methodologically sound and replicable in accordance with the program’s size, risk, priority and significance.
Evaluation will be conducted with the right mix of expertise and independence from program managers. Evaluation teams may need to include relevant expertise such as experts and be balanced in composition, taking into account gender and cultural diversity where appropriate. The involvement of program managers, stakeholders and subject matter experts in the planning and governance of evaluations and reviewing draft evaluation reports can enhance the potential learning from an evaluation and acceptance of its results. Responsibility for the final content of evaluation reports will rest with the evaluators.
Stakeholders are to be identified and actively involved in the evaluation process. This will ensure that the definition of outcomes, activities and outputs, as well as what is important to measure in assessing program success, is determined in a collaborative way. Stakeholders are vital in contributing to the interpretation of evaluation information and in formulating recommendations.
Evaluation will be timely and strategic to influence decision-making. Providing valid, reliable information requires a balance of technical and time requirements with practical considerations to ensure the evaluation supports evidence-based decision-making.
Evaluation will be transparent and open. Comprehensive information on all aspects of the evaluation should be systematically recorded, including choice of methods, analyses, and conclusions. Factual findings and conclusions should be explicitly justified and clearly distinguished from value judgments and recommendations.
1.3. Key requirements
Program evaluations are to be conducted in accordance with the general principles and guidelines in the department’s Evaluation Framework. The scale of an evaluation should be proportionate to the size and significance of the program using the four scale model in the NSW Government Program Evaluation Guidelines. Staff should refer to these frameworks to consider the extent to which they could evaluate programs of varying size, importance and complexity.
Program evaluations and reviews will be conducted according to annual evaluation schedules that will be approved by the department’s Executive and submitted to the Cabinet Expenditure Review Committee each financial year. All programs should undergo some form of periodic review and/or evaluation.
Sound evaluation is an integral component of any program that represents a major government investment, including significant initiatives that are funded through internal reallocation of resources. Such programs should be evaluated even if not identified in the department’s annual evaluation schedules.
Programs that are central to the achievement of state, corporate and national goals, involve large-scale investment or are resource intensive must be evaluated as part of the department’s annual evaluation schedules. These program evaluations have the potential to support evidence-based decision-making, improve reporting on performance and inform improved resource allocation and policy choices.
Programs that are wholly or partly funded by other government agencies and/or non-government organisations must be evaluated collaboratively and included in the department’s annual evaluation schedules if the department is the lead agency.
Major initiatives of strategic significance and trial programs that may be part of a broader rollout must incorporate an evaluation plan with appropriate budget in their implementation plan.
An evaluation plan will be required when developing the business case for a new policy proposal, or a recurrent proposal to expand or significantly reform existing programs for submission to the Cabinet Expenditure Review Committee. Proposals must specify and quarantine an evaluation budget and an explicit date for review or evaluation. The department’s Centre for Education Statistics and Evaluation(CESE) and Finance directorate must be consulted on all such proposals.
Evaluation supports transparency and accountability. Evaluation reports will be used to inform decision-making where appropriate, to improve outcomes and resource allocation. The department uses evaluation to identify impacts and benefits, to expand successful programs, to redesign the delivery of existing programs, or to reprioritise resources from existing programs should they no longer be considered a government priority, or effective and efficient in achieving expected benefits.
Evaluation findings must be made publicly available, except where there is an overriding public interest against disclosure, in line with the Government Information (Public Access) Act 2009 (GIPAA). Evaluation reports should be released in a range of forums to enhance accountability and promote evidence-based policy development. These forums include the department’s website and an evaluation repository on the Centre for Education Statistics and Evaluation (CESE) website.
2. Audience and applicability
These requirements apply to all department staff including those in schools, divisions and entities.
Evaluation, along with innovation and evidence-based practice, are key elements of the department’s corporate planning and performance reporting. Evaluation should take place across the lifecycle of a program, from initial design and trialling through to implementation and ongoing delivery.
This policy should be read in conjunction with the principles and procedures provided in the department’s Evaluation Framework, the NSW Government Program Evaluation Guidelines and the Treasury circular, TC18-03 Program Evaluation. Access to resource materials and expertise is provided to assist in the design and implementation of evaluation. A whole-of-government online evaluation toolkit has been developed to support the implementation of the guidelines.
The department’s executive, divisions, schools and entities will provide strong, positive leadership to promote and embed a culture of evaluation. Specialist evaluation units such as the Centre for Education Statistics and Evaluation will provide advice, expertise and support for program evaluation across all education sectors.
Evaluations are also developed and implemented in the context of the NSW state priorities, Council of Australian Governments (COAG) national partnerships, Commonwealth–State bilateral agreements, the NSW Quality Framework for vocational education and training, and the National Quality Framework.
4. Responsibilities and delegations
The Secretary has overall responsibility for the direction of evaluation in the department.
The department’s Executive will approve an annual 12-month evaluation schedule for submission to the Cabinet Expenditure Review Committee.
Steering committees for major strategic evaluations will be chaired by a member of the department’s Executive or an officer nominated by the department’s Executive.
In other contexts, business unit managers have responsibility for ensuring compliance with this policy including:
- schools – principals
- divisions – Deputy Secretaries
- state office directorates – executive directors, chief executive officers and directors
- Centre for Education Statistics and Evaluation – executive director.
The proactive and public release of government information should be undertaken in accordance with the delegations of authority under GIPAA (intranet only).
The Centre for Education Statistics and Evaluation will coordinate the implementation of this policy for the department and will report, as required, to the Executive Director, Centre for Education Statistics and Evaluation.
5. Monitoring and review
The Centre for Education Statistics and Evaluation in the Educational Services division will monitor and review the implementation of this policy and will report, as required, to the Secretary.
Executive Director, Centre for Education Statistics and Evaluation, (02) 7814 1527
Director, Evaluation, (02) 7814 1354, email@example.com