This report was originally published 27 November 2020.

Read online

What are the Check-in assessments?

The Check-in assessments are optional online reading and numeracy assessments designed to assist schools following the disruptions to schooling in 2020. The assessments cover similar aspects of literacy and numeracy as in NAPLAN reading and numeracy tests.

These formative assessments are offered for schools to:

• supplement existing school practices used to identify how students are performing in literacy and numeracy and
• help teachers tailor teaching to meet student needs.

This document provides a summary of information and data from the Check-in assessments delivered in 2020.

Each assessment in 2020 was designed to be quick and easy to administer, consisting of approximately 40 multiple choice questions. Suggested completion time was 50 minutes, however, teachers could use their discretion based on the needs of their students.

Students in Years 5 and 9 completed the assessments during Term 3, Weeks 5 to 7 (17 August – 4 September). Students in Year 3 completed the assessments during Term 3, Week 10 to Term 4, Week 2 (21 September – 23 October).

Initial results were available to schools within 48 hours of test completion, enabling teachers to rapidly move to use the results in addressing learning gaps.

To assist teachers in using the results, test items were aligned to the NSW syllabus, National Literacy and Numeracy Learning Progressions and teaching strategies.

Student assessment feedback and mapping against the syllabus and learning progressions indicators was made available in the department’s reporting platform, Scout.

Features of the school reports included:

• information at item-level with links to the questions and strategies related to the skill being assessed
• information at syllabus stage and progression level for each student
• feedback on strategies students may have been using if they got the answer correct or incorrect alongside how each student responded.

Records of student achievement of learning progression indicators were also available in the department’s PLAN2 platform, where teachers could monitor student progress and create ‘Areas of Focus’ for targeted teaching and skill development.

Support

Professional learning and assessment support was available to all teachers in participating schools for 2020 assessments. This included how best to make use of the assessment package for each school context, administration of the assessment, how to access and use feedback to help inform planning and strategies for teaching.

As at 10 November, more than 4,700 teachers had accessed:

• online support, including live chat and sessions with other teachers to ask questions and
• share ideas
• a range of online courses including guided professional learning to support the analysis of their students’ assessment information using a ‘data pathway’ (Terms 3 and 4)
• strategies to identify areas to focus attention in aspects of literacy and numeracy
• teaching strategies to address these areas specific to Years 3, 5 and 9.

Participation rates

Participation in the Check-in assessments was high, with 83% (1,775) of department schools participating (of schools with students in Years 3, 5 or 9). Participation was higher among primary schools than secondary schools, with 88%-of all Year 3 students, 86% of all Year 5 students, and 61% of all Year 9 students in NSW government schools participating in the Check-in assessments.

Participation was largely representative across various student and school groups.

Table 1: Number of schools and students participating in Year 3 Check-in assessments

Table 2: Number of schools and students participating in Year 5 Check-in assessments

Table 3: Number of schools and students participating in Year 9 Check-in assessments

*Note (for tables 1-3): Remoteness area is based on ASGC2016 remoteness area classifications. Inner regional and outer regional Australia are combined, as are remote and very remote Australia. Percentages of schools participating are calculated based on the total number of schools with enrolments in the relevant scholastic year, for each school type. Figures are based on the test participation data extracted from the test platforms on 10 November 2020.

Summary results

For each 2020 assessment, a quarter of the test items were NAPLAN items with known psychometric properties and difficulty estimates on the NAPLAN scales. This provided the possibility of linking the Check-in assessments with these scales to assist with further analysis.

After scaling and equating exercises for available results from Year 3, Year 5 and Year 9 tests, five assessments in Year 3 reading, Year 3 numeracy, Year 5 reading, Year 5 numeracy and Year 9 numeracy were able to be equated to the NAPLAN scales. Year 9 reading was not able to be linked to the NAPLAN scale due to a range of factors including test design differences between NAPLAN and the Check-in assessment.

As the Check-in assessments were optional, results were weighted (at student level and by prior performance band in NAPLAN test for Years 5 and 9 or prior performance band in Best Start Kindergarten assessment for Year 3, and remoteness) to arrive at population estimates.

Table 4 presents the estimated proportions of students in NAPLAN bands based on Check-in assessments measured in August – October 2020.

Table 4: Estimated (weighted) proportion of students by band, Check-in assessment (August – October 2020)

Table 5 presents the mean scaled scores for each assessment, for 2019 NAPLAN and as estimated from Check-in assessments measured in August – October 2020. This table shows the August – October results in 2020 were similar to previous years’ NAPLAN results, assessed in May for Year 3 reading, Year 5 reading and numeracy, and Year 9 numeracy. In contrast, Year 3 numeracy Check-in results in September/October were substantially higher than previous years’ NAPLAN results assessed in May (note that NAPLAN did not take place in 2020 due to COVID-19).

Table 5: Weighted mean scores of students in the Check-in assessments, compared to NAPLAN 2019

Note: Due to differences between the Check-in assessments and NAPLAN tests (e.g. test design, purpose of tests), caution is needed when comparing Check-in results to NAPLAN results.

Feedback

The response from schools as to the diagnostic value of the assessments has been overwhelmingly positive. Teachers have commented:

“the rich data gleaned is simply amazing!”

“as a class we use the Check-in assessment feedback to talk about how we solve number problems and what strategies we use”.

Conclusion

The 2020 Check-in assessments demonstrate the feasibility of conducting formative assessments that provide schools with rapid insight and highly targeted support in a short timeframe and reduced administrative complexity. The high take-up and strong support across schools demonstrates the willingness and ability of schools to use formative assessment to support their professional judgments in rapidly identifying gaps in student learning. The inability to equate Year 9 reading also demonstrates to some degree the limitation of a fast deployment in single state context. In the longer term the availability of pre-calibrated assessments for use by teachers would further increase the uptake and usability of check-in type assessments.

At the system level, the comparison with 2019 NAPLAN demonstrates that students were generally performing in August – October 2020 at the same levels previously seen in May (with the exception of Year 3 numeracy). This indicates that on average students have fallen approximately 3-4 months behind in Year 3 reading, and 2-3 months behind in Year 5 reading and numeracy and Year 9 numeracy.