Quantitative and qualitative data

A combination of different types of data is most effective in generating powerful evidence to assess school performance and improve practice. Both quantitative and qualitative data have a place in evaluation.

• Quantitative data is any information that can be reduced to a set of numbers. Information from which you can create averages, differences or totals is quantitative data. This is particularly helpful for summarising large amounts of information in a snapshot, tracking trends over time and understanding patterns and differences from one group of participants to another.
• Qualitative data is data that is not easily reduced to numbers. In a school setting, qualitative data may come from observations, work samples, conversations, written documents and more. Thematic analysis of qualitative data is helpful for exploring emerging issues, providing rich description and context about a complicated issue and building theories about what might explain trends and patterns in quantitative data. Qualitative data tends to help us answer questions about the 'what', 'how' and 'why' of a phenomenon, rather than questions of 'how many' or 'how much'.

To read more about quantitative and qualitative data, go to the School Excellence Framework evidence guide.

Strengths and limitations

Every type and source of data has its own strengths and limitations.

• Quantitative data can only ever tell us part of the story. Even if we drill down through statistical analysis we can only analyse what has been captured, for example undertaking item-level analysis in NAPLAN results or looking at attendance trends for only a select group of students. This is why we often need qualitative data to provide context and insight into issues.
• At the same time, it is important to avoid over-generalising from qualitative data, drawing conclusions about 'all' or 'most' people when we have really only heard from a few.
• Self-report data rely on people expressing their opinions or describing their own experiences, attitudes or behaviours. This method is great for empowering people to tell their story in their own words, but it also relies on memory and runs the risk of 'social desirability bias', which is the natural human desire not to 'say the wrong thing'. In some cases, however, it is the only form of data that may be able to answer the evaluation question.
• Independent observation or assessment avoids some of the problems of self-report data. However, there are some things that are very difficult to observe, such as people's attitudes and feelings, and it can be easy to miss things or misinterpret what we see or hear. There are also times when people will behave differently because they know they are being observed or their work is being assessed. This kind of data also requires consistent analysis from one observer or assessor to the next, much like marking moderation.

Triangulation

Triangulation means working with multiple sources of data, with awareness of their strengths and limitations for specific purposes. The metaphor comes from orienteering: in order to locate ourselves on a map, we need to take our bearings from at least two reference points in the environment. Examples of triangulation in evaluation include:

• surveys that ask a mix of scale questions (quantitative) and open-ended questions (qualitative)
• student assessment tools that include free-text notes as well as standardised quantitative measures, for example, Best Start
• asking different people about the same thing, for example getting feedback from students as well as teachers
• corroborating self-report data with independent observation, for example about students' mastery of a new skill, or teachers' confidence with a new teaching practice.