Assessing impact using the Bands over Time report

The Bands over Time report can be used to identify changes in NAPLAN performance over time. Used in conjunction with other forms of information, it is useful for assessing the impact of school programs.

The Bands over Time report can be found in the NAPLAN (DoE) app under the heading ‘School Results’.

Bands over Time report

The report shows the percentage of your school’s cohort within each NAPLAN band across selected years.

The report allows you to track performance over time

This is useful for highlighting any changes or differences in results. Ideally, your school would see a decrease in the percentage of students in lower bands and an increase in the percentage of students in higher bands over time.

The report is useful for assessing the impact of programs

If your school introduces an intervention program, it is useful to compare data from before the program was implemented to data from after the program was implemented. Ideally, you would see improvements within the targeted area of need.

Bands-Over-Time-01 — Image: Percentages of Students in bands by Calendar Year

Scenario: Assessing the impact of a numeracy program for girls

In 2015, your school introduced a Year 4 ‘Maths for Girls’ program, aimed at building mathematics skills and confidence in female students. The program is still running, with 30% of Year 4 girls attending regularly. You are interested to see if the program has had any impact on female students’ results in Year 5 NAPLAN numeracy.

Begin by looking at your school’s historical Year 3 NAPLAN numeracy results

Open the Bands over Time report (found in the NAPLAN (DoE) app under the heading School Results)
Generate the report using the appropriate slicers for:
- Which assessment? NAPLAN 5
- Which domain? Numeracy
- Gender = Female
Look for differences between before the program was introduced (before 2015) and the years after. Differences in data would be most apparent from 2016 onwards.
Cross-check the data by selecting the slicer Gender = Male to see if the trends differ.
Next, investigate the expected growth of girls in Numeracy, comparing years
Open the Growth report (found in the NAPLAN (DoE) app under the heading Student Results)
You want to compare the period prior to the introduction of the program to the period following. Three years is an appropriate period of time to take away the impact of specific cohorts of students.
Generate the report using the appropriate slicers:
- Which year was it? Hold down Ctrl (PC) or Command (Apple) to select 2013, 2014 and 2015
- Which scholastic year? Year 5
- Which assessment? NAPLAN 5
- Which domain? Numeracy
- In the STEP 2 section select Gender: Female
- Change the slicer at - Which year was it? to select 2016, 2017 and 2018.
Pay particular attention to the bar across the top of the report which shows the proportion of students - At or Above Expected Growth (green), Below Expected Growth (yellow), and Negative Growth (red).

Bands-Over-Time-02 — Image: Example of the bar across the top of the report.

Consider

How has the proportion of girls achieving 'At or Above Expected Growth' changed?
How does this relate to our expectations of impact?
What other factors might be contributing to achievement of both Band results and Growth results?

Once you have the data

The data indicates that the program had a positive impact on results, however it is not possible to fully attribute the improvement of female numeracy NAPLAN scores to the program. It would be worth cross-referencing scores from other assessments, to check if those students who participated in the program achieved higher results than before. Other factors, such as changes to the school’s Index of Community Socio-Educational Advantage (ICSEA) should also be considered.