Advice on using artificial intelligence tools for student assessment and feedback

Summary

The University is updating its guidelines relating to staff use of new Generative AI (GenAI) tools to assist with assessing students’ work or providing feedback on students’ work. These new guidelines seek to achieve a balance between managing the risks associated with the use of AI tools for feedback and assessment, while supporting innovation in AI use where this provides benefits to students and staff.

From Semester 1, 2024, staff can use new AI systems to support their evaluation of students’ work and to provide feedback to students. However, staff remain responsible for any academic judgements made on students’ submitted work and any feedback provided to students.

The outputs from any AI system used for assessment or feedback must be reviewed by staff and the prompts and inputs the AI system is using must be well understood and managed.

The sole use of new GenAI tools to allocate marks or grades to students is not appropriate.

Staff wishing to use new GenAI tools to support student assessment and feedback should use the University’s internal, secure platform of GenAI tools called ‘Spark’ and should seek the endorsement of their Faculty’s Associate Dean, Teaching and Learning prior to doing so.¹

Staff who are interested in using new GenAI tools that are not within the University’s secure Spark platform or enterprise technology suite² need to seek endorsement from the Deputy Vice-Chancellor, Academic.

Importantly, this advice specifically focuses on how staff might use new AI tools for the assessment of, and feedback on, students work. To avoid confusion, this advice is not related or relevant to:

Student use of artificial intelligence tools (even where they are directed to use such tools in the completion of work in their subjects that will be assessed by staff)
Staff use of artificial intelligence tools to prepare their own teaching, learning and assessment material that is then used in their subject or program

[1] Staff who intend to use technology-based tools that are currently within the University's enterprise technology suite (eg Perusall, Gradescope) do not require endorsement or approval.

[2] See the learning technologies web page.

Background to these guidelines

AI has been used in student assessment and feedback for some time. The University already makes available some AI-enabled tools for student assessment within its enterprise suite (eg automated MCQ marking, Perusall, Gradescope). But GenAI tools like Chat GPT are potentially “game changers” when it comes to student assessment and feedback given their power, accessibility, and conversational interface.

New GenAI tools have the potential to assist with assessment and feedback processes, not only by making them more efficient, rapid and scalable, but also by enabling better analysis, and by fostering more and better dialogue between students and staff. Noting this is an emerging area, some examples of how new GenAI tools could be used in assessment include:

Providing feedback to students on the proposed structure of their essay.
Reviewing students’ generated software code to evaluate its quality.
Assessing students’ knowledge and understanding based on their written responses to questions given a marking rubric.
Expanding on short comments from assessors to give students more detailed and consistent feedback on their work.

AI intervention to assist students with the development of their assessment material is already possible. Cadmus, available within the University’s enterprise suite, can be used to set up an environment within which students can develop written assessments. The AI in this tool prompts students to engage in productive learning behaviours, such as outlining and structuring their work, reviewing the marking rubric, inserting citations and references, etc. To set up Cadmus for your assignment, see this guide.

Staff responsibility, roles and pedagogical use

Assessment and feedback are essential to the ongoing pedagogical relationship between students and academic staff. Feedback provided to students is an essential pedagogical tool to allow both students and academic staff to determine how students are progressing on their learning trajectory in a subject and in a program. While there is certainly great potential, ill-considered deployment of new GenAI tools also has the potential to undermine this pedagogical relationship between academic and student.

A reasonable expectation of both students and others in our communities (staff, parents, government) is that providing feedback to students and the assessment of their work is the primary domain of academic educators of the University.

In line with the University’s new AI principles, staff who use an AI system to support them in making evaluative judgements about students’ work or to provide feedback to students on that work remain responsible for the judgements made and feedback provided. This means that the outputs from any AI system used for assessment or feedback must be reviewed by staff, the prompts and inputs the AI system is using must be well understood and managed, and the sole use of new GenAI tools to allocate marks or grades to students is not seen as appropriate at this stage.

Managing risks

More than other potential teaching and learning uses of GenAI tools (eg asking students to use publicly-available tools to produce artefacts they may critique, and so forth) it is important that educators are aware of risks to be managed when considering using these tools in assessment.

Particularly, these risks relate to:

Protecting student IP: student work remains the IP of the student and should not be uploaded to any open third-party software that are not part of the University’s enterprise suite
Protecting copyright: materials to which we do not own the copyright (eg readings provided to students through ReadingsOnline or other means) cannot be further shared.
Protecting University IP: the teaching materials produced by educators, including lecture slides, assignment rubrics etc. are University IP and should not be further shared. If such materials are found on third party sites there is a takedown process that can be followed.

Staff wishing to innovate with GenAI tools in assessment where the above materials may be involved should use the University’s internal, secure platform of GenAI tools called Spark. Spark provides access to both GPT3.5 and GPT4 but keeps the materials shared with the tool within the University’s enterprise environment.

How do I get started?

From Semester 1, 2024, the following guidelines apply for staff use of AI tools to assist in assessing or providing feedback on student work:

Technology-based tools that are currently within the University's enterprise suite (eg Perusall, Gradescope) can continue to be used for assessment. No specific approval is required before using these tools.
Staff looking to experiment with new GenAI tools to support the assessment of and feedback on student work should use the University’s secure Spark AI platform and take following steps:
1. Contact your faculty’s Associate Dean, Teaching and Learning and make them aware of what you are proposing to do and secure their endorsement, given your local curriculum context.
2. Access the Spark platform.
3. Before using any new GenAI tools to support your assessment of or feedback on student work, provide students with a statement that clearly outlines the tools being used, their intended use in feedback and/or assessment, and how students can opt out ³.
Staff looking to experiment with new GenAI tools to support the assessment of and feedback on student work outside of the University’s secure Spark platform should contact the DVC Academic to discuss the proposed use.

[3] Some students may not wish to have their work uploaded to a GenAI platform even if it is secure. Students should be given the option to opt out and rely solely on the academic judgement of staff for their assessment and feedback.