Issues in assessment of general capabilities

Peer reviewed article

This paper is based on a literature review on the assessment of general capabilities. Kevin Bradburn, Paul Brock, Shantha Liyanage, Gerry McCloughan, Gail Miller, Margaret Mulcahy, Andrew Newman, Melissa Nyholm, Penelope Parsons-Lord and Louise Taggart contributed to an earlier draft of this paper – a literature review on the assessment of general capabilities.

Portrait photo of Rob Stevens
Dr Robert Stevens, Manager, Research/Quality Assurance, in the Policy, Planning and Reporting Directorate

21st century capabilities

Flourishing in the 21st century requires capabilities such as critical thinking, ethical understanding and creativity. With challenges facing developed countries early in the 21st century in mind, the Ministers of Education in Australia have committed to support all young Australians to develop such capabilities (MCEETYA, 2008, pp. 8-9). General capabilities such as these have been included in the Australian Curriculum (ACARA, 2012). These kinds of capabilities tend to elude measurement by standardised tests. Some writers claim that assessments for these capabilities do not exist (Fullan, 2010, p. 20). So, can they be measured at all and, if so, how can they be measured?

In this paper I argue that these general capabilities are able to be measured. I then propose a framework for measuring these capabilities, consider its key features and implications for practice.

The approach to the assessment of general capabilities outlined in this paper is a tool that teachers might use if they wish to include the general capabilities in their planning and programming. Teachers may wish to emphasise general capabilities to tailor their program to the specific needs of individuals or groups. The NSW Department of Education and Communities (the Department) has developed a framework for the assessment of two general capabilities: literacy and numeracy. The framework is based on a progression of learning. The approach considered in this paper may be useful in assessing aspects of other general capabilities, particularly those that involve reasoning or complex knowledge.

General capabilities

General capabilities are capabilities that are not specific to a discipline. The Australian Curriculum is underpinned by seven general capabilities. These are:

  • literacy
  • numeracy
  • information and communication
  • technology (ICT) competence
  • critical and creative thinking
  • ethical behaviour
  • personal and social capability
  • intercultural understanding.

By linking literacy and numeracy with these other general capabilities, ACARA suggests implicitly that the general capabilities are ‘new basics’. The general capabilities are accorded a status equivalent to the old basics – the 3 Rs – or literacy and numeracy. Each of these capabilities has an equal claim to being a basic capability – the foundation for further learning in the disciplines, and for future success in life.

In the NSW syllabuses for the Australian Curriculum , the general capabilities are embedded, along with cross-curriculum priorities, in the ‘Learning Across the Curriculum’ content.

Cultivating general capabilities

How can the general capabilities best be cultivated?

The Organisation for Economic Cooperation and Development (OECD) states that to improve learning in school classrooms (not specifically general capabilities) both ‘direct and student-oriented instruction methods should be used’.

  • Direct instruction is built around problems with clear, correct answers that can be learned quickly.
  • Student-centred instruction is associated with the teacher facilitating students’ own inquiry by allowing them time to find solutions to problems on their own before the teacher demonstrates how a problem is solved.

The OECD observes that, while there is no consensus in the literature on which approach is better, ‘an over-reliance on either approach is not recommended’ (OECD, 2012, p. 138).

The cultivation of general capabilities also requires the balanced application of a range of different pedagogies. Indeed, the cultivation of general capabilities – knowing-how as distinct from knowing-that – may require more emphasis on particular forms of student-centred instruction. The Department’s Quality teaching framework, and the research which informed it, reflects such pedagogical concerns.

Learning by doing

In his 1916 book, Democracy and education, the distinguished American philosopher and educator John Dewey suggested that we learn from experience. For Dewey, experience is not, as it is commonly thought of, a ‘mysterious conduit through which information is conducted from the external world into the mind’. Rather, according to Dewey, experience involves a combination of acting or doing or trying something and ‘feedback’from the world on that action. For Dewey, experience, and learning from experience, is not purely a passive undergoing but an active doing and trying.

Whether we learn to walk, to talk, to read, to write, to ride a bike, to perform, or to think, we learn to do something by trying it, by giving it a go. We refine our ability through regular practice, taking account of the feedback we receive.

How can learning, as Dewey describes it, best be supported? Of particular importance in cultivating those capabilities involving reasoning (such as critical thinking and ethical understanding) is the use of a student-centred pedagogy, such as ‘Socratic pedagogy’.

Philosopher and educator Matthew Lipman developed and refined a dialogue-based inquiry approach to teaching critical thinking called ‘Philosophy for children’(1991). This approach is based on a ‘community of inquiry’ in which children learn critical thinking by working with one another and building on each other’s ideas, questioning each other’s underlying assumptions, and suggesting alternatives. Engagement in a community of learning is a powerful means of learning as doing in the Deweyan sense.

An example of such an approach is the trial of ethics classes in NSW in which students engage in discussing ethical issues, guided both by purpose-built thought provoking teaching resources and questioning from the teacher (Knight, 2010, p. 47).

Assessing general capabilities: the SOLO model

How might general capabilities be assessed?

The Structure of the observed learning outcome (SOLO) taxonomy (Biggs, 1995; Biggs & Collis, 1982; University of Queensland, 2008) provides a systematic way of describing how a learner’s performance grows in complexity when mastering varied tasks. The SOLO taxonomy postulates five levels of increasing complexity in growth or development of concepts or skills:

Table of SOLO stages

Implicit in the SOLO model is a set of criteria for evaluating the quality of a response to (or outcome of) a task utilising general capabilities. The quality (or richness or complexity) of a response to a complex task varies with the ‘relevance’of the considerations brought to bear on the task, the range or ‘plurality’of those considerations, and the extent to which these considerations are ‘integrated’into a whole, and ‘generalised’to or related to, broader contexts.

How might the SOLO taxonomy be used to assess general capabilities?

Critical thinking

Nussbaum (2010) characterises ‘critical thinking’ as the ability to:

  • think and argue for oneself, rather than defer uncritically to tradition or authority
  • stop, reflect and analyse, so that crucial issues are not missed by haste and inadvertence (p 50)
  • probe, evaluate evidence, write papers with well-structured arguments, and analyse the arguments presented in other texts (p. 55).

The following reasoning progression, based on a framework proposed by Songer (2009), captures Nussbaum’s characterisation of critical thinking using the SOLO taxonomy. The levels are as follows.

Table of SOLO critical thinking taxonomy

A teacher using the SOLO framework for assessing an argument proposed by a student might seek evidence of:

  • the reasons advanced in support of a claim. Are they relevant to that claim?
  • the range of reasons advanced in support of a claim. Are a range of relevant issues considered?
  • the integration of relevant considerations into a valid argument. Does the student develop a valid argument for the claim, identify objections and counter-arguments to the claim, and reply to those arguments?
  • the application and testing of principles. Does the student identify a general principle in support of a claim, and test that principle?

Ethical understanding

Ethical understanding involves, in part, the application of reasoning to questions about the good life and how we ought to live.

In his book The Idea of Justice, Sen (2009) asks: ‘What kind of reasoning should count in the assessment of ethical and political concepts such as justice and injustice?’ Sen’s answer, in a nutshell, is reasoning that is ‘open, impartial, objective and public’ – or reasoned scrutiny from different perspectives.

While Sen suggests that the path of reason does not exclude taking note of the value of instinctive reactions ... this is consistent ‘with not giving our unscrutinised instincts an unconditional final say’ (Sen, 2009, p. 51).

The idea of justice stresses the importance of public reasoning and the need to accept a plurality of reasons that may be sensibly accommodated in an exercise of evaluation (Sen, p. 394). Judgements about justice have to take on board the task of accommodating, and ‘integrating’, different kinds of reasons and evaluative concerns (Sen, p. 395).

Sen argues that the encounter of public reasoning should go beyond the boundaries of a state or a region – in a sense, generalised or considered in relation to atypical views from non-local contexts. This is, in part, because of the pertinence of other people’s perspectives to broaden our own investigation of relevant principles, for the sake of avoiding under-scrutinised parochialism of values and presumptions in the local community (Sen, p. 402).

How can ethical understanding be assessed using the SOLO model? Suppose students are invited to develop arguments for or against the continuation of live animal exports. The quality of a response to this task might be assessed by considering:

  • the relevance of the reasons provided in support of the conclusion
  • the range of reasons provided
  • the way these reasons are integrated into a clear argument.

An instinctive reaction for or against continuation of live animal exports (a prestructural response) is a useful first step but it would not be the end point of a high quality response to the task. A higher quality response requires the articulation of good reasons for the conclusion. A response that considers the interests of all those affected by the practice of live animal export, both human and non-human, (a multistructural response) will be of higher quality than a response that argues for or against live animal exports on the basis exclusively of animal or human welfare.

A higher quality (relational) response involves the development of a well-integrated argument for or against live animal exports that identifies objections and counter-arguments to the claim, and replies to those objections and

counter-arguments.

An extended abstract response might consider the reasons that led New Zealand to ban live animal exports in 2003.

Creativity

The SOLO model appears to be appropriate to assessing tasks involving reasoning capabilities. How does it fare as a framework for the assessment of creative capabilities such as composing music or creating visual art?

Daniel Levitin (2006) suggests that music is organised sound. (If so, reasoning might be considered to be ‘organised’thought.) The basic elements of any sound are loudness, pitch, contour, duration, (rhythm), tempo, timbre, spatial location and reverberation. In music, these elements are organised into meter, key, harmony and melody (Levitin, p. 14).

Levitin notes that

The idea of primitive elements combining to create art, and of the importance of relationships between these elements, also exist in visual art and dance ... what makes a set of lines and colours into art is the relationship between this line and that one; the way the colour or form echoes another in a different part of the canvas ... When they combine harmoniously they give rise to perspective, foreground and background and ultimately to emotion and other aesthetic attributes.

Levitin, pp. 18-19

Levitin notes that

The appreciation we have for music is intimately related to our ability to learn the underlying structure of the music we like ... and to be able to make predictions about what comes next. Composers imbue music with emotion by knowing what our expectations are and then very deliberately controlling when these expectations will be met, and when they won’t.

Levitin, p. 111

If Levitin is right, the quality of a piece of art may be assessed by the level of organisation or structure in that work. A teacher may assess the quality of a piece of music composed by a student by assessing:

  • the presence of ‘relevant’elements such as rhythm
  • the extent of the range of musical elements
  • how these elements are ‘integrated’into harmony and melody
  • how these elements combine harmoniously in a song
  • to what extent does the piece of music stretch the boundaries of the tradition and confront our musical expectations in a satisfying way?

The way in which the quality of musical composition is structured or organised appears to render it assessable in terms of the SOLO framework.

Features of assessment of general capabilities using the SOLO model

What are the features of the approach to assessment of general capabilities using the SOLO model?

A developmental perspective

The approach reflects a developmental perspective. The developmental perspective assumes that

… student performance on a given learning progression can be traced over the course of instruction ... Assessing the growth of students’ understanding of particular concepts requires a model of how student learning develops over a certain period of (instructional) time.

Wilson et al, 2012, p. 130

It is clear that the SOLO taxonomy adopts a developmental perspective on student learning reflected in a developmental perspective from prestructural through unistructural, multistructural, and relational to extended abstract.

Theory of learning

The SOLO model is informed by a constructivist view of learning that, in turn, informs inquiry-based pedagogical approaches.

This view of learning emphasises the importance of the active role of learners in developing their understanding through using existing ideas and skills to build ‘bigger’ideas and more advanced skills. Big ideas are ones that can be applied in different contexts (Gardner et al, 2010, p. 19-20).

The approach to assessment proposed in this paper would encourage the development of assessment and learning activities, with what the NSW Quality teaching frameworkcharacterises as of high intellectual quality, focused on producing deep understanding of important, substantive concepts, skills and ideas (NSW Department of Education and Training, 2003, p. 9).

Assessment of performance rather than ability

The SOLO framework is less a framework for assessing ability and more of a framework for assessing human creations such as arguments, artefacts and performances. To that extent, it is de-personalised – assessing performances, not people. The framework can be used to assess group performances just as readily as it can individual performances, or individual contributions to group performances.

Assessment for learning

Gardner argues that assessment of any kind should ultimately improve learning (Gardner, p. 2). The SOLO framework is clearly not just assessment of learning but assessment for and as learning. The kinds ‘of’activities that provide the richest data ‘for’the assessment of general capabilities are learning activities such as:

  • writing papers with well structured arguments
  • analysis of arguments presented in other texts
  • preparing for and participating in discussions and dialogue, either face-to-face or online
  • creating a piece of music
  • researching an issue
  • evaluating evidence.

These assessment activities are equally learning activities.

Making thinking visible

The SOLO framework has the potential to generate assessments that in the words of Binkley (2012, p.25)

provide a window into students’ understandings and the conceptual strategies a student uses to solve a problem. Further, by making students’ thinking visible, assessments thus provide a model for quality practice.

The SOLO framework clearly makes student thinking visible. It can be used to classify the quality of a performance, as represented by the sophistication of the assumed underlying logic of students’ responses to assessment/ learning activities.

Using a variety of assessments

Gardner (p. 42) proposes that

… assessment should be based on information of different kinds, including students’ self assessments, to inform decisions about students’ learning and achievements.

The approach to the assessment of general capabilities described in this paper involves the use of a large variety of alternative, authentic ways to learn what students are capable of. Evidence of student learning may come from a wide variety of written and verbal communications that are part of learning activities, such as:

  • essays
  • papers
  • research reports
  • discussions with the teacher
  • discussions with students
  • student questions
  • responses to questions
  • artefacts that students create
  • discussion of these artefacts
  • student arguments
  • broader student behaviour.

They could also come from standardised tests, such as PISA.

Implications for practice

The SOLO model is a useful theoretical model for the assessment of general capabilities. It also has practical significance. Teachers can use the criteria of relevance, plurality, integration and generalisation to assess the quality of rich tasks involving general capabilities. It can be used, with particularly good effect, in connection a student-centred pedagogy – such as Socratic pedagogy.

Teachers can also use the SOLO framework, in particular the five SOLO levels, to inform the development of rich assessment and learning tasks, to cultivate and assess general capabilities. In particular, the five SOLO levels could inform an assessment rubric that could be used in programming learning and assessment tasks. For example, the SOLO model provided the theoretical underpinning for descriptions of student levels of achievement in the ‘Essential secondary science assessment theoretical framework’. This framework is used to design and select items for the test, and is used as the basis for reporting student achievement in the test. It was developed using the Board of Studies NSW Science Years 7-10 syllabus(2003), SOLO and classroom practice.

Teachers might use SOLO in conjunction with syllabus outcomes and classroom practice in their programming and planning. This could be done using the Board of Studies NSW Program builder, software that uses content from the new NSW syllabuses for the Australian Curriculum, to create scope and sequences and units. It enables teachers to add their own teaching, learning and assessment activities, alongside content.

The SOLO model and Socratic pedagogies merit rigorous testing in schools through action research, or teacher research, that involves investigations being done by teachers for teachers with the aim of improving student outcomes (NSW Department of Education and Training, 2010).

Conclusion

Good assessments of general capabilities do exist, and have existed for thousands of years. So do the corresponding ‘instructional practices’, such as ‘Socratic pedagogy’, to teach general capabilities.

This paper has presented a framework for assessment and learning of general capabilities. Whatever barriers exist to the widespread adoption of these pedagogies and assessments, it is not that they have never been tried, or tried and found wanting. The approach merits further testing through action research in schools.

References and further reading

ACARA 2012, General capabilities in the Australian Curriculum, Australian Curriculum Assessment and Reporting Authority (ACARA), accessed 20 January 2018.

Biggs, J. 1995, ‘Assessing for learning: Some dimensions underlying new

approaches to educational assessment’, The Alberta Journal of Educational Research, vol. 41, no. 1, pp.1-17.

Biggs, J. B. & Collis, K. F. 1982, Evaluating the quality of learning – the SOLO taxonomy, Academic Press, New York.

Biggs, J. B. & Collis, K. F. 1991, ‘Multimodal learning and the quality of intelligent behaviour’ in Helga Rowe (ed.), Intelligence: reconceptualisation and measurement, Australian Council for Education Research (ACER), Camberwell, VIC, pp.57-76.

Bingham, G., Holbrook, T. & Meyers, L. 2010, ‘Using self assessments in elementary classrooms’, Phi Delta Kappan, vol. 91, no. 5 F, pp. 59-61.

Binkley, M., Erstad, O., Herman, J., Raizen, S., Ripley, M., Miller-Ricci, M. & Rumble, M. 2012, ‘Defining twenty-first century skills’, in P. Griffin, B. McGaw, & E. Care (eds), Assessment and teaching of 21st century skills, Springer, Dordrecht.

Cheng, B., Ructtinger, L., Fujii, R. & Mislevy, R. 2010, Assessing systems thinking and complexity in science, SRI International, California, accessed 20 January 2018.

Fullan, M. 2010, All systems go: the change imperative for whole system reform, Corwin Press, California.

Gardner, F., Harlen, W., Hayward, L. & Stobart, G. 2010, Developing teacher assessment, McGraw Hill Open University Press, New York.

Griffin, P., Care, E. & McGaw, B. 2012, Assessment and teaching of 21st century skills, Springer Press, Melbourne Australia.

Holder, J. (ed.) 2006, Early Buddhist discourses, Hackett Publishing Company, Indianapolis.

Levitin, D. 2006, This is your brain on music: understanding a human obsession, Atlantic Books, London.

Lipman, M. 1991, Thinking in education, Cambridge University Press, New York.

MCEETYA 2008, Melbourne declaration on educational goals for young Australians, accessed 20 January 2018.

NSW Department of Education and Training 2003, Quality Teaching Model in NSW public schools. Discussion paper, Professional Support and Curriculum Directorate, Ryde, NSW,

NSW Department of Education and Training 2010, Action research in education, 2nd edn, Professional Learning and Leadership Development Directorate, Ryde, NSW.

NSW Department of Education and Training 2008, Quality teaching to support the NSW professional teaching standards, Professional Learning and Leadership Development Directorate, Ryde, NSW.

Nussbaum, M. 2010, Not for profit: why democracy needs the humanities, The Public Square series, Princeton University Press, USA.

OECD 2012, Equity and quality in education: supporting disadvantaged students and schools, OECD, Paris.

Sen, A. 2009, The Idea of justice, Penguin Books, London.

Songer, N., Kelcey, B. & Gotwals, A., 2009, ‘How and when does complex reasoning occur? Empirically driven development of a learning progression focused on complex reasoning about biodiversity’, Journal of Research in Science Teaching, vol. 46, no. 6, pp.610-631.

University of Queensland 2008, Biggs’ structure of the observed learning outcome (SOLO) taxonomy, Teaching and Educational Development Institute, accessed 20 January 2018.

Wilson, M., Bejar, I., Scalise, K., Templin, J., William, D. & Irribarra, D. 2012, ‘Perspectives on methodological issues’, in P. Griffin, B. McGaw, & E. Care (eds), Assessment and teaching of 21st century skills, Springer, Dordrecht.

Keywords: SOLO taxonomy; assessment; student-centred learning

How to cite this article: Stevens, R. 2013, ‘Issues in assessment of general capabilities’, Scan 32(4)

Return to top of page