Document analysis can be used:

• to determine evidence of activity
  • For example, reviewing stage or faculty meeting minutes to identify how and what information about curriculum implementation (phases, journeys, role responsibilities, KLA/ subject matter details) is discussed.
• to determine evidence of process quality
  • For example, reviewing the faculty, stage or school wide process for developing scope and sequences, teaching and learning programs or assessment materials. This can determine whether steps are relevant, concise and sequential for staff to follow.
• to determine evidence of impact
  • For example, by triangulating professional learning attendance records and feedback with teaching and learning program documents, to demonstrate adequate planning to improve understanding of curriculum changes.

To undertake a document analysis, a three-step process can be undertaken:

• Step one – Identify: determine data relevant to your evaluation question; determine how you will record the relevant data you are looking for in each document, this could be through colour coding and highlighting pre-determined criteria; record in a template or use a consistent method of recording the frequency of statements related to the criteria
• Step two – Read: dig deeper to identify relevant patterns in the text; categorise or code words or phrases according to the agreed criteria
• Step three – Interpret: review the categorised data to reflect on what the documents are showing; summarise key themes or patterns; record any other relevant factors (author of the document, who contributed, who did not to determine any voices that may be missing); work collaboratively with colleagues to review interpretations of any words or phrases that are complex to code; annotate documents where necessary to show clear alignment to the coding process; tally the reference/s made to each criteria; consider overall frequencies or average frequencies of criteria to enable comparisons, and; note the facts and consider whether the data is providing new or emerging themes or aligning with existing knowledge.

• Plan evaluation questions and select relevant and existing documents to review carefully. For example: an open-ended question will result in deeper analysis, rather than a question that requires a yes/no answer.
• Keep the scope tight. Use specific criteria to evaluate relevant document/s that are aligned to your evaluation inquiry or question.
• Plan and allocate time to undertake document analysis with team members.
• Involve those undertaking the document analysis process in the planning to ensure inter-rater reliability (consistency of different individuals in measuring something).
• Collaborate with team members to determine a list of possible documents to review. These may include:
  • scope and sequences
  • learning sequences or units of work
  • lesson plans or teaching and learning strategies
  • work samples
  • minutes of faculty/stage/executive meetings
  • procedure documents (school) or policies (departmental)
  • professional learning
• Consider using a sampling strategy if a large number of documents are to be reviewed. This may involve a representative sample across faculties or stages, or across student groups when reviewing work samples.
• Seek permission from staff prior to analysing documents and inform staff of the intention of the review and how the data will be used.
• Duplicate documents you have chosen to review, so that any annotations or coding is recorded on duplicated copies, not on originals.