Evaluations should be designed to serve a specific need. To serve the need, the findings must be accurate and useful so they can guide the program owners to improve the processes and outcomes of the program.

Stakeholders should to be consulted throughout the evaluation to ensure that the purposes, activities and findings of the evaluation address the needs of all interested groups.

Consider the efficiency and cost effectiveness of the evaluation. The benefits of the evaluation must outweigh the costs involved in its implementation.

The composition of an evaluation team needs to be carefully considered, to ensure a suitable mix of expertise and independence. This will help reduce the possibility of bias.

Practical strategies for achieving neutrality in a school context include:

• ensuring interviews with teachers are undertaken by senior staff who they don’t report to, e.g. the head teacher from a different faculty, or different stage coordinator
• ensuring focus groups with students are undertaken by a staff member who doesn’t teach the subject or group of students
• masking the identity of student work samples, so that familiarity with the students doesn’t influence the assessment.

For larger evaluations, it may be appropriate to establish a separate reference group to provide neutral advice on the methods to be used and a critique of the analysis. A reference group may include experts and stakeholders, as well as members of the evaluation team. A reference group will determine procedures or ‘ground rules’ that guide the conduct of the evaluation.

All participants in an evaluation need to be well informed about what will happen during the evaluation, as well as how the findings will be used. Participation needs to be voluntary.

• Obtain informed consent before anyone participates in the evaluation. Participants need the opportunity to weigh up the benefits and risks of taking part before they give consent. It is the responsibility of the evaluation team to make sure that all participants understand all parts of the evaluation and the nature of the consent they are giving.
• In all but exceptional circumstances, the department requires ‘active consent’ before students can participate in an evaluation.
  • Active consent requires a written agreement to participate, both from the student and their parent or carer.
  • There may be circumstances in which the form needs to be available in languages other than English.
  • Consent is required regardless of age. Students who are preliterate must still understand what they are agreeing to. In some cases, parents or carers may sign on behalf their child. A suitable statement may be worded: “I have explained this evaluation to my child. They understand what they are expected to do and their right to withdraw.”
  • Consent forms must indicate that participation is voluntary and that consent can be withdrawn at any time without consequence for the participant and without providing a reason.
  • Keep written consent forms on record for a minimum of five years from the date of collection.
• Participants should not feel compelled to participate in an evaluation, and have the right to withdraw at any time. For example, a student might feel compelled to participate when asked by a teacher, or a teacher may feel compelled because their supervisor requested it. Reassure participants that there will be no personal consequence for them (positive or negative) if they choose to participate or not.
• Schools may use a process of ‘standing consent’, where participants acknowledge their general willingness to participate in various evaluations in the school during the year. However, for each evaluation, participants should still be informed so they have the opportunity to opt out if they wish.
• Important changes to the evaluation process must be communicated to participants. If the change has implications for participants, the evaluation team may need to arrange new consent releases.
• If the evaluation design involves qualitative data collection, think carefully about how participants will be grouped. Neutral group dynamics are important for ensuring that the data is credible and that participants are comfortable revealing all relevant information. See the notes on interviews and focus groups for more on this.
• As a general rule, incentives should not be offered to students or staff for their participation in evaluations. There are some contexts in which an evaluation team might feel the need to offer incentives to secure sufficient participation, such as holding a free barbeque after school to encourage community participants to attend a discussion group. Where this is offered, the evaluation team should disclose and justify this in their reporting.

Removing students and teachers from their usual timetables, duties and responsibilities to take part in evaluation activities may interrupt learning opportunities or break times for students, and disturb the usual business of the school. The choice of methods, and the logistical arrangements for them, should seek to minimise disruption.

Plan your evaluation activities to ensure the wellbeing of all participants, while still providing credible data for the evaluation. Particularly where an evaluation addresses topics that may be upsetting for participants to discuss (such as experiences of stress or bullying), the evaluation team should have an agreed strategy for what to do if a participant becomes distressed. This includes a protocol around stopping the interview or discussion, if that’s what the participant wants to do, and an immediate referral pathway if required.

Evaluation teams working with children must have all relevant clearances and comply with Child Protection Procedures. At times, students may disclose issues to the team that must be reported. If at any time an evaluator identifies that a student may be at risk of harm, they must report this information, including the identity of the student, to the principal.

Evaluation teams need to ensure all participants or groups of participants have an opportunity to contribute fully to the evaluation.

• Language can be a barrier for some participants. Where necessary, consider using interpreters to assist participants in understanding the evaluation and communicating their responses. Interpreters are a free resource offered by the department. Auslan interpreters are available for participants who have hearing difficulties.
• All evaluation instruments and information about the evaluation must be age-appropriate.
• The evaluation team may need to organise additional assistance, such as support personnel or a parent, to enable students and other participants to understand the evaluation or provide responses.

All evaluations must be conducted with sensitivity to cultural, religious and other differences amongst participants. School staff will usually alert external evaluation teams to particular circumstances, but it is up to evaluation team members to make sure they are fully aware of potential issues. Below are some significant groups to note:

• Aboriginal and Torres Strait Islander students
  The Australian Institute of Aboriginal and Torres Strait Islander Studies (AIATSIS) has published Guidelines for Ethical Research in Australian Indigenous Studies (2012). All evaluation teams are encouraged to refer to these guidelines prior to conducting evaluations involving Aboriginal and Torres Strait Islander participants. This may require a separate ethical approval process, which should be considered when planning. A local Aboriginal or and Torres Strait Islander representative should be consulted to advise on the conduct of the evaluation and to provide support to students or other participants, where necessary.
• Cultural, religious and social groups
  Australia is a diverse society. Participants in an evaluation may come from a wide range of cultural, religious and social groups that may each present factors to be considered. Consult with school staff or with expert organisations to seek advice about any requirements or constraints that may affect individual participation.
• Gender
  Evaluations may sometimes investigate issues to which girls and boys, or women and men, relate differently or in particular ways. It is worth considering which evaluators might work with which participants. For example, it may be more appropriate to have male evaluators with male students on issues regarding boys’ wellbeing, or conducting separate group interviews with girls and boys if the evaluation requires particular points of view from each. Special consideration should be given to participation of students who identify as LGBTQIA+.
• Dealing with sensitive issues
  Some evaluations deal with issues that are particularly sensitive for some participants, for example, refugee families, victims of bullying or sexual abuse. Consulting with relevant organisations or experts is important to gain an understanding of any potential issues and how to work with them.

It is important to protect the privacy and anonymity of participants. As a general rule, all data collected in school-based evaluations (for individual participants, groups and schools) should be de-identified for reporting.

Some school data, in particular, is confidential. Attendance, suspension and student performance data can be sensitive. Wherever possible, evaluations should only use data that is aggregated across the whole sample group. In cases where the sample size is small (fewer than ten people, for example), aggregated data will not necessarily protect the identity of participants and should not be used.

In multi-school evaluations, evaluation teams, stakeholders and reference groups need to decide whether it is necessary to report school names.

The department’s Enterprise Data Policy outlines the processes and systems relating to data governance. Alongside this policy is the Data Release Protocol that provides information about department-wide protocols for accessing and using data. Where necessary, seek advice from the policy contact.

Evaluation teams must take an objective view of the programs evaluated and ensure that findings are reported accurately.

It is important to ensure that all findings are included. It can sometimes be easy to overlook data and findings when they do not fit a pre-conceived or particular view of the program. Evaluation reports must remain objective and address all findings.

Findings should be reported in a timely manner to ensure that they remain relevant and useful. Delays in reporting may render the findings less useful, especially if the program has changed or progressed since the time of the evaluation.

Responsible reporting of evaluation findings includes being sensitive to and respectful of cultural, religious and other differences amongst research participants, and the impact that publication could have on participants.