Capturing and measuring student voice
This report was originally published 01 November 2016.
'Student voice' refers to the views of students on their experiences of school and learning. Listening to student voice is a powerful way for teachers, school leaders and education researchers to systematically look inside the ‘black box’ of learning from the perspective of the learner (Black & Wiliam 1998). Student voice enables schools (and the broader education community) to learn from students how they see school, and to build a better understanding of factors that affect student learning.
This Learning Curve focuses on capturing and measuring student voice as a way to gain insights into, and improvements in, student engagement. It deals with four key research questions: Why measure student voice? How and when should student voice be measured? What questions can and should be asked? And how should student voice be interpreted? This Learning Curve can be read in conjunction with a series of case studies that look at how NSW public schools have been using the Tell Them From Me survey to measure student voice, assist with school decision-making and improve school outcomes.
What is student voice?
Student voice refers to the perspective of students on their experience of education. In its simplest form, this can mean asking students’ opinions about playground procedures or school uniform. The concept of student voice, nonetheless implies more than simple consultation with students. It is also about recognising that students have distinctive views on their schooling; and affording students the opportunity to influence their own school experience by listening and responding to student feedback (Cook-Sather 2006).
The concept of student voice, when thought about in this way, places value on the diverse thoughts, beliefs and perspectives of students and incorporates these into school decision-making and the cyclical processes of planning, self-assessment and ongoing improvements. For example, a school may be interested in how to improve classroom practices, therefore they may survey students to gain an understanding of their experiences. This data may be considered alongside other evidence the school holds about effective classroom practice and then used to inform change. Student voice thus becomes not only an understanding of the values, beliefs and opinions of students, but a tool that can be used to improve student outcomes and facilitate school change (Mitra 2003; Rudduck & Flutter 2000; Rudduck & Fielding 2006; Rudduck 2007). This is true across the education spectrum, starting from early childhood (see, for example, ACECQA 2016) through to further education, where student voice can be used both as an indicator of quality practice and a strategy to inform improvement.
Vision for student voice
The NSW Strategic Plan for Children and Young People (Office of the Advocate for Children and Young People 2016) observes that children and young people wish to have their voice heard in all aspects of their lives and they want their opinions to be heard and taken seriously.
The NSW Department of Education has a strong commitment to student voice both as a means to allow students to engage, participate, lead and learn; and as a tool to provide data to support many of the Department’s major initiatives. These include the School Excellence Framework, the Wellbeing Framework for Schools and the Creating Futures Together 2015-2017 strategy.
The Department recognises that students have unique perspectives on learning, teaching and schooling; that they can actively shape their own education and schooling experience; that they can actively participate in their schools, communities and the education system; and that students contribute to decision making processes and collectively influence outcomes by putting forward their views, concerns and ideas.
Why measure student voice?
Student voice is considered to be one of the best ways to measure student engagement (Willms 2014). While it is possible to measure student engagement by collecting objective data on behavioural indicators such as attendance or homework completion rates, listening to student voice can provide a better understanding of student experiences (Appleton et al. 2006). Student voice can be particularly useful for measuring emotional and social engagement which may not be directly observable by teachers or through other methods (Fredericks & McColskey 2012).
Using student voice as a means of measuring engagement can also lead to improvements in engagement. The very act of giving all students an opportunity to provide input on their engagement via student feedback may have the added benefit of increasing student effort, participation and engagement in learning (Cole 2006). It allows students to realise that they can have an impact on things that matter to them at school (Fielding & Rudduck 2002), thus raising motivation and engagement (Toshalis & Nakkula 2012). Using student voice to measure engagement may also foster a sense of inclusion, citizenship and school attachment among the student body (Devine 2002), and increase the involvement of historically disengaged and underachieving students (Mohamed & Wheeler 2001).
Capturing student voice can be a powerful and effective tool for school planning and improvement (Manefield et al. 2007). For example, Hattie (2009) tells us that student feedback helps to make teaching and learning more visible and this can lead to discussion and debate among teachers about their teaching practice. In response to student feedback, teachers may develop new perspectives on what and how they teach and make improvements as a result.
Incorporating student voice into school planning processes can lead to school improvements in student achievement (Wilkerson et al. 2000). According to PISA and TALIS data, academically high-performing countries are more likely to seek formal feedback from students and are also more likely to act on student voice in schools. For instance, 87 per cent of principals of 15-year-olds in Singapore seek written feedback from students for quality assurance and school improvement, compared with 69 per cent of principals in Australia (unpublished PISA 2012 data).
It is possible to quantify aspects of school improvement by measuring student voice over time. For example, re-administering student surveys after responding to student feedback allows school leaders and teachers to examine whether strategies that have been implemented have led to changes in the learning environment from the perspective of students. Bell and Aldridge (2014) collected student’s feedback on the classroom environment and investigated whether teachers’ reflection of this feedback led to improvement in the classroom. They administered student surveys to 560 classes before and after teacher reflection and found differences for eight of 11 scales measured, all showing a positive change in the classroom environment.
Many schools today are making use of student feedback in school planning and improvement. For example, in NSW, the Central Coast Principals Learning Alliance runs a program that engages student voice as a way to improve secondary education learning outcomes and school leadership. This program is part of a broader project that aims to improve teaching and learning by looking at the relationship between student voice, teachers’ professional learning and reflective practice. Schools involved in this program seek to engage student voice in multiple ways: as a data source; via student involvement in school governance; and in reflective practice with teachers.
(Central Coast Principals’ Learning Alliance 2014)
How should student voice be measured?
A number of methods can be used to capture student voice, including student surveys, interviews, students writing about their class experiences in a learning log, classroom observations, student feedback surveys, and 360-degree feedback (Wilkerson et al. 2000; Richardson 2010; Hoban & Hastings 2006).
Each method has pros and cons. For example, student interviews give a very personal account of student’s perceptions, but can also raise confidentiality issues (Hoban & Hastings 2006). Student logs may describe what students learn but usually do not state how they learned or how they were taught (Hoban & Hastings 2006). Classroom observations give students a framework to describe teaching and learning but they can be time-consuming to administer and provide limited information on the quality of effort or participation (Fredericks et al. 2004).
Student surveys are one of the most widely used tools for capturing student voice, both in Australia and internationally (Jensen & Reichl 2011). Student surveys usually ask students a series of questions about their attitudes towards school (including the broader school community such as teachers, parents etc.) and/or themselves. An advantage of student surveys over other means of capturing student voice, is that they are able to assess a large population quickly, easily and reliably, providing measures of student views. They also provide a more confidential alternative to other, more direct methods of capturing student voice, such as classroom observations and student interviews (Hoban & Hastings 2006).
One concern with student surveys, despite confidential administration, is that students may not always answer honestly and therefore survey data may not reflect actual behavior and perspectives (Appleton et al. 2006). This risk is mitigated, however, by the fact that student surveys can be given to a large and diverse sample of students at relatively low cost, making it possible to gather data over several waves and compare results across schools (Fredericks & McColskey 2012). Peterson et al. (2000) found that data from student surveys and questionnaires is in fact highly reliable due to the large number of students responding. Aleamoni (1999) found that student ratings of teaching practices in surveys tend to be consistent over time, and at the classroom level (Richardson 2010; Ferguson 2010), pointing to the reliability of student surveys.
Integrating teacher and parent feedback
One way to enhance the value of student surveys is to gather feedback not just from students, but also from the broader school community, such as teachers and parents. Schools can incorporate this broader school voice into school planning and improvement measures. For example, using surveys to capture parent feedback not only allows schools to reflect on how they relate to both students and parents, but it can also strengthen collaboration between parents and teachers (Peterson et al. 2003). It can help align the goals and priorities of teachers and parents, and help teachers work out the best way to support parental engagement in the home so that the actions of parents and teachers complement each other (Perkins 2014).
When should student voice be measured?
The question of when to measure student voice relates to how student feedback is intended to be used. If the feedback is to be of benefit to the current survey respondents, it is best to measure student voice earlier in the school year or a course of study, rather than only at the end (Richardson 2010). This allows schools to respond to feedback in such a way that the students providing it can experience the resulting change.
It can also be of benefit to measure either successive snapshots or a cohort over time. This is called a longitudinal study. What makes a longitudinal study unique is that instead of a researcher collecting data from varying subjects in order to study the same variables, the same subjects are observed multiple times and often over the course of many years. Longitudinal analyses can therefore help establish causal links, which one-off surveys usually do not do. For example, longitudinal studies can show the extent to which early problems with behavioural engagement have a long-lasting effect on achievement (Alexander et al. 1997) or how a student’s interest and motivation in school may increase their educational persistence and ultimately their high school completion (Janosz et al.2008). A longitudinal measure can also help determine if school improvement measures have been effective, as often the effect of these measures is not seen immediately but rather some time down the track.
What questions should be asked?
When engaging student voice, it is important to be clear about the main objective as this should determine the focus of the survey questions or other feedback. It is important to ensure that questions reflect the breadth of concept the school is interested in, and that this concept is also broken down into discrete, measurable components. For example, student engagement is a broad topic, encompassing intellectual, emotional, behavioural, physical, and social factors that either enhance or undermine learning for students. It can be defined or interpreted differently depending on who is asking the question or interpreting the result (Great Schools Partnership 2015).
It is also important to think about what insights the questions will give, and whether this matches the intended use of the survey. Questions may need to be able to differentiate responses between different students, classrooms and/or different schools. For example, if a question on teaching practice looks the same among different types of classrooms and/or schools, the instrument may not be providing enough scope in terms of responses (MET Project 2012).
How should student voice be interpreted?
Sometimes schools can be overwhelmed by the prospect of analysing and interpreting the data that results from implementing a student voice initiative such as a student survey. Data may be in the form of quantitative or qualitative reports and can necessitate understanding of mathematical terminology such as ‘means’, ‘median’ and ‘standard deviation’; and/or the ability to analyse patterns, themes and trends within the local context (La Trobe University 2015). It is probable that at the school level there will be some people on staff who have these skills and can assist with the interpretation of data. There are also resources online that can assist (see, for example, La Trobe University 2015).
Where possible, different data sources (e.g. informal discussions, learning analytics and other surveys) should also be triangulated to cross-verify or refute interpretations that might be made based solely on the basis of one source of data (La Trobe University 2015).
Schools also need to be open to what the student voice data is saying. In the face of unexpected results, some schools may assume that the survey instrument is flawed, that students are deliberately messing with the surveys, and/or that the results are simply ‘not correct’. While this may sometimes be true, it may also be the case that the school feels challenged by or uncomfortable about what students are saying. Often, organisations go through a ‘cycle of acceptance’ when faced with news that they think is bad or differs from their own view (Thwink.org 2014). For example, Fitzgerald (2010) describes running a student engagement survey in a Canadian junior high school over several years. At first, survey results were not positive. Staff were tempted to dismiss student responses 'as a rant to a captive audience'. They were not comfortable using data and felt it might be used to judge them. Over the course of time, however, the principal worked with staff to develop and share questions; ensure the survey was only used for improvements in learning, not individual staff evaluation; and inducted new staff into the ‘student voice’ culture. This led to the survey being successfully made part of the school culture and an important tool to improve student learning1.
The NSW experience
The Centre for Education Statistics and Evaluation (CESE) is using the self-reporting Tell Them From Me suite of surveys (student, teacher and parent) to capture and measure student voice across NSW public schools.
The Tell Them From Me student survey can help schools understand students’ perspectives on critical aspects of their school experience, such as social, intellectual and institutional engagement, wellbeing and exposure to quality teaching practices. The Focus on Learning teacher survey provides school principals and school leaders with insights into school and classroom effectiveness, from the perspective of teachers. The Partners in Learning parent survey provides information on parents’ perspectives on their child’s experience at school as well as learning at home. Further detail about the full suite of Tell Them From Me surveys can be found at the CESE website.
This paper can be read in conjunction with the Tell Them From Me case study series produced by CESE which highlights how individual NSW public schools have used the Tell Them From Me surveys for school planning and improvement purposes.
1 For more information on how change can effectively be achieved in schools, see Fullan (2006) and his paper on change theory. He highlights the need for seven premises to be in place for continuous improvement: a focus on motivation; capacity building with a focus on results; learning in context; changing context; a bias for reflective action; tri-level engagement; and persistence and flexibility in staying the course.