Improvements and alternatives to the disadvantaged Language Background Other Than English (LBOTE) measure

This report was originally published 11 September 2014.

Image: Improvements and alternatives to the disadvantaged Language Background Other Than English (LBOTE) measure

Summary

The report of the Gonski review of funding for schooling recommended a new schooling resource standard that includes “a series of loadings for student- and school-based sources of disadvantage” (p 153). Limited English Language Proficiency (ELP) is one such source of disadvantage. The Gonski report noted that while state systems are able to measure ELP through their own data systems, there is no nationally consistent measure of ELP currently available.

In its modelling of a new schooling resource standard, the Gonski review instead used the only measure which was available nationally, the ‘disadvantaged LBOTE (Language Background Other Than English)’ measure, as a proxy measure of educational disadvantage within the broader LBOTE population. The disadvantaged LBOTE measure was developed by the Australian Curriculum, Assessment and Reporting Authority (ACARA). It combines LBOTE with parental education equivalent to Year 9 or below.

However, it is generally acknowledged1 that the disadvantaged LBOTE measure is a poor approximation for limited English language proficiency.

In September 2012, the Australian Government agreed to fund two projects (endorsed by the Strategic Policy Working Group (SPWG) in July 2012) to:

  1. Investigate any improvements that could be made to the ‘disadvantaged LBOTE’ measure; and,
  2. Conduct a cost-benefit analysis of a trial of a more accurate, nationally consistent ELP measure.

This report presents the findings of the first project, which was completed by the NSW Department of Education and Communities (NSW DEC).

Existing Accuracy and Coverage of Disadvantaged LBOTE

The NSW DEC analysis firstly provides evidence that the disadvantaged LBOTE measure is inadequate as a proxy measure for limited English language proficiency for three reasons:

  1. It underestimates the absolute size of the cohort needing support (low size equivalence)
  2. It fails to identify many of the students needing support (limited coverage)
  3. It includes too many students who do not require support (lack of accuracy)2

These conclusions were reached by comparing disadvantaged LBOTE students to those with limited ELP as gauged by the NSW DEC measure of English as a Second Language (ESL). This section describes the data and analysis that supports this conclusion.

Summary of improvements and alternative measures

A range of possible improvements and alternatives to disadvantaged LBOTE were analysed as potential proxy measures of students with limited English proficiency. Most displayed insufficient size equivalence, coverage and accuracy to be considered as appropriate proxy measures for inclusion in a new schooling resource standard as a loading for limited English proficiency. Table 3 provides a summary of the efficacy of the measures considered.

The measure that showed the greatest improvement was ‘LBOTE and <=2 years in an Australian school’. This measure had good size equivalence (106%) along with reasonable, albeit insufficient, improvement in coverage (56%) and accuracy (52.7%). One other measure showed similar size equivalence, but less of an improvement in the other aspects: ‘Disadvantaged LBOTE, OR LBOTE and <=1 year in Australian school’. However, both these measures included data elements not currently available at the national level (i.e., length of time in an Australian school).

Table 3: Summary of all measures considered

Group Size of group Size equivalence (group size as % of target cohort size) Intersect (no. in group also in target cohort) Coverage (% target cohort represented) Accuracy (% of group accurately identified)

Target cohort: Students with limited English proficiency (44,494)

Disadvantaged LBOTE


21,626


49%


5,475


12.3%


25.3%

LBOTE and not born in Aust


62,086


140%


17,064


38.4%


27.5%

Disadvantaged LBOTE, OR LBOTE and <=1 year in Aust school



42,957



97%



18,400



41.4%



42.8%

Disadvantaged LBOTE, OR refugee students


25,501


57%


7,115


16.0%


27.9%

Disadvantaged LBOTE OR NAP students


25,948


58%


8,365


18.8%


32.2$

LBOTE and <=2 years in Aust school


47,311


106%


24,930


56.0%


52.7%

LBOTE and <=1 year in Aust school


23,916


54%


14,660


32.9%


61.3%

LBOTE and <=2 years in Aust school and not born in Aust


18,005


41%


10,355


23.3%


57.5%

Refugee students


6,409


14%


2,860


6.4%


44.6%

NAP students


5,904


13%


3,372


7.6%


57.1%

Refugee OR NAP students


12,159


27%


6,015


13.5%


49.5%

Conclusion

The aim of the current project was to determine ‘what short-term improvements could be made to the disadvantaged LBOTE measure to improve its accuracy.’

The NSW DEC analysis found that disadvantaged LBOTE should not be used to assess students eligible for the ELP loading because it:

  1. did not identify the right students, and,
  2. bore little relationship to the size of the cohort needing support.

NSW DEC investigated other options to better measure low ELP, either by augmenting the disadvantaged LBOTE measure or replacing it. However, none of the options investigated reached the level of accuracy, coverage or size equivalence required to justify inclusion in the new schooling resource standard, or to warrant the time and cost necessary to develop them as nationally consistent measures.

Trying to improve the disadvantaged LBOTE measure, or find alternatives to it, leads to the conclusion that a new national measure is needed to accurately target funding to low ELP students.

The analysis also presented the results of a NSW DEC trial into the EAL/D Learning Progression which found that EAL/D had high levels of reliability and validity, as well as acceptance by teachers, and was superior to the current, internal NSW DEC measure of ELP, known as ESL Phase 1, 2 and 3, which was itself superior to the disadvantaged LBOTE measure. The full report of the NSW trial of the EAL/D Learning Progression is available at: http://www.dec.nsw.gov.au/documents/15060385/15385042/Report26Februaryfinal.pdf

In conclusion, a new, nationally consistent measure is needed that is both more accurate and has greater validity than disadvantaged LBOTE. This study has shown that the EAL/D measure is a clear candidate for that role. A second project conducted by NSW DEC will present a cost-benefit analysis of developing a nationally consistent ELP measure, focused on EAL/D.

1 Acknowledged in the Gonski Report (p. 118); demonstrated in NSW DEC analysis provided to the SPWG meeting May 24 2012, and acknowledged by SPWG at the July 26 2012 meeting.

2 The latter two criteria for evaluating the disadvantaged LBOTE proxy measure (i.e., coverage and accuracy) are similar in concept to the two statistical measures used in Epidemiology for assessing medical tests – Sensitivity (relating to the rate of true positives) and Specificity (relating to the rate of true negatives).

Return to top of page Back to top