Turning data into evidence
Data becomes evidence when it helps us answer a question or test whether a claim is true.
There are many different sources of data in schools, from work samples and assessments to administrative records, observations, direct feedback and more. Within this range, there are also different types of data, for example, qualitative and quantitative, each with its own strengths and limitations.
Before we can decide what data we need, we have to be clear about our questions, and what kind of evaluation we are doing.
- evidence of activity - shows what has taken place. This often comes from administrative records, and is helpful for describing the reach and scale of a project or demonstrating compliance with a policy or implementation of a new initiative. This is important scene setting for further analysis, so that the strengths and weaknesses of implementation can be understood, or to assist in identifying barriers and enablers in the operating environment.
- evidence of process quality - not only tells us what we did, but how well we did it. This evidence can come from a number of sources, including participants' feedback about their experiences, comparison of observed practice with recommended practice or something not going as planned.
- evidence of impact - is what allows discussion about the consequences of the program’s actions – the short, medium and long term outcomes that flow from our efforts. These impacts might include changes in teacher practice, student wellbeing, learning engagement or learning outcomes. This kind of evidence requires us to wrestle with the challenges of outcome measurement and attribution, identifying what has changed and why.
- Using Data with Confidence, NSW Department of Education, Centre for Education Statistics and Evaluation