Data analysis resources – guiding advice
The following resource can be used to guide schools in using data analysis practices to monitor and evaluate activities aligned to the phases of curriculum implementation.
Data is defined as: measurements or observations that are collected as a source of information (CESE, 2023). There are many different types of data that can be collected by schools to ascertain information on curriculum implementation, which may include:
- documents: units of learning, teaching and learning programs, scope and sequences, assessment schedules, work samples or assessment tasks
- observations: student achievement, lesson observations, learning walks
- administrative records: student achievement, assessment or attendance records, or staff professional learning records
- survey or focus groups: student exit slips or feedback from teachers on curriculum implementation from professional learning exit slips.
Data becomes evidence when it helps us answer a question or test whether a claim is true. Analysing student data helps teachers to identify areas of student need.
When considering curriculum implementation, evidence serves as a key reference for schools to reflect on and improve teaching and learning for both staff and students. Data supports school leadership teams to plan the next steps by identifying whether implementation practices are ‘on track’ or ‘off track’ and helps to identify when staff or students are ready for the next phase of curriculum implementation.
Target audience
The audience for this resource is school principals, school leadership teams and school staff. Directors, Educational Leadership (DELs), Principals, School Leadership (PSLs), Principals in Residence (PIRs) and Principal Coach Mentors (PCMs) can also use this resource to guide schools with their IPM development and planning for curriculum implementation.
What and why
Curriculum implementation provides an opportunity for schools to place curriculum at the centre of school planning. Effective curriculum implementation drives student growth and attainment, and school improvement. This requires:
- evaluating curriculum implementation activities that focus on the Leading, Teaching and Learning domains within the School Excellence Framework (SEF)
- a focus on curriculum implementation in areas such as educational leadership, staff capabilities, learning and development, improvement goals, policy, data skills and use, classroom practice, and assessment.
This resource was developed as a guide for schools to understand and build data analysis capabilities to effectively evaluate curriculum implementation, highlighting the importance of:
- understanding data types (qualitative and quantitative) including strengths and limitations
- best practice data analysis (bias, collection and collation)
- triangulation.
When and how to use
Understanding the principles of evaluative thinking and applying effective evaluation processes enables schools to investigate their curriculum implementation initiatives in a meaningful way. Effective curriculum implementation is an iterative and continuous improvement process, occurring simultaneously for different syllabuses as they are published.
This resource can be used to:
- implement effective data collection, collation, analysis practices to support school leaders to monitor, review and adjust supports for curriculum implementation within their School Excellence Plans (SEPs) and implementation and progress and monitoring (IPM)
- build staff capability to embed the use of data into their daily practice, supporting a strong culture of evaluation, building improved capabilities of using data for continuous improvement for curriculum implementation across the school.
Contact
Email questions, comments and feedback about this resource to contactcurriculumimplementation@det.nsw.edu.au by using the subject line ‘School planning for curriculum implementation’.
Interpreting and analysing data effectively requires an understanding of the features of data concepts and evaluation tools. Data literacy incorporates the ability to recognise and use visual and numerical displays to describe data, and to apply this understanding to draw conclusions from the information.
To effectively evaluate curriculum implementation, school leaders and teachers may benefit from reviewing data concepts through the following resources:
- CESE Glossary: a glossary of data and evaluation terms
- Evaluation resource hub: professional learning resources for evaluation
- Universal Resources Hub (staff only): evidence guides for school planning.
Defining data types
Data can be classified into the broad categories of qualitative or quantitative data sources.
Information that can be reduced to a set of numbers, for example, where something is counted, measured or assessed. Can be used to determine averages, differences or totals (numerical data, statistics).
Benefits
- Helpful to summarise large amounts of information
- Tracks trends over time and highlights patterns
- Can be useful when measuring change
- Can be presented in an easily comparable form (such as percentage score)
Limitations
- Does not provide specific details about an individual
- Does not provide reasons for trends or patterns
- Can give the illusion of being precise
Analysis considerations
- Variability is normal
- Data can be influenced by unexpected factors
- Triangulate data to ensure validity
- Confidence intervals allow for a more reliable range of values
- Look at the distribution of scores
- Only note the facts
- Ensure consistency of data collection, recording and storage
- Visually plot data to identify trends and patterns
- Identify strengths, weaknesses and gaps
- Sample size is important – a large sample size results in more reliable generalisations
- Classes or cohorts can be used to track data over time
- Compare scores across different assessment tasks to determine individual student growth
- Be mindful when comparing achievement to external benchmarks, as these reflect the ‘average’ student
- Look at growth between assessments (such as pre and post-tests)
Curriculum implementation examples
- Professional learning attendance – syllabus familiarisation (evidence of activity)
- Professional learning exit slips – staff knowledge and confidence to implement the new curriculum (evidence of process quality)
- Pre and post-tests – unit of learning (evidence of impact)
Information from open-ended questions, observations, work samples, pictures, audio or other sources. Can be used to determine the what, how and why of an evaluation question (non-numerical, stories).
Benefits
- Provides context around quantitative data
- Gives insight into thoughts, feelings, opinions and experiences
- Explores emerging issues and provides rich narrative
Limitations
- Can be time consuming to analyse
- Broad generalisations are difficult to determine depending on the specific context
- Self-report data relies on memory and can lead to social desirability bias
Analysis considerations
- Thematic analysis requires identification and examination of patterns and themes in data – consider relevance to the inquiry question, what stories are emerging?
- Identify strengths, weaknesses and gaps.
- Ensure consistency of data collection, recording and storage.
- Consider the statistical significance of data (for example accuracy of respondents and the representation of the wider population).
- Avoid making generalisations from small sample sizes.
- Consider quantifying responses to draw comparisons across results.
- Keep the scope of your analysis tight to avoid being overwhelmed with information.
- Triangulate results with multiple data sources.
- Assign codes for responses to de-identify individual responses from surveys, observations or focus groups.
- Consider visually representing results for ease of comparison and reporting.
Curriculum implementation examples
- Teacher focus groups – usefulness of syllabus familiarisation professional learning (evidence of activity)
- Short answer survey responses – staff knowledge or confidence in implementing the new curriculum (evidence of process quality)
- Document analysis – unit of learning reviewed for the alignment to the scope and sequence (evidence of activity)
Reflective questions
The following questions can be used to reflect on data analysis practices for curriculum implementation.
- What qualitative and quantitative data types are used to evaluate curriculum implementation?
- How often are opportunities available to identify and collect meaningful data?
- How is the information obtained from data routinely discussed and analysed?
- How are you or your colleagues supported to use data to modify teaching practices?
Resources
The following data analysis tools could be used by schools to guide the effective use of data to evaluate curriculum implementation activities.