Artificial intelligence
Artificial intelligence
Artificial intelligence
Emerging AI technologies create challenges and offer new opportunities for innovation in the ways we teach, assess, and undertake research.
As these AI technologies and tools rapidly evolve and their potential application expands, the University, like so many other organisations, will need to carefully consider some of its current approaches to education and research. In the immediate short term, the University will need to maintain vigilance in the use and misuse of tools like ChatGPT to ensure assessment security and the continued integrity of conferred degrees. Following this, the rise of AI technologies and tools presents our academic community with significant opportunities to consider how we can revise, tailor, and enhance our current assessment practices.
Current landscape
The increasing sophistication of generative artificial intelligence (AI) software, including text generators (eg ChatGPT, Quillbot), image generators (eg Dall-E, Lensa, Stable Diffusion), audio generators and music composition (eg AIVA, Amper) and other language tools software (eg translators) has sparked discussion worldwide about how such tools will alter work and education.
The November 2022 release of OpenAI’s ChatGPT provoked public debate and growing concern about the power of these tools to disrupt our current approaches to teaching, learning, assessment, and research. On 14 March 2023, OpenAI launched GPT-4. More powerful than the GPT-3.5 predecessor which powered ChatGPT, GPT-4 has increased capabilities to interpret images and to reason and understand complex scenarios. This guide primarily focuses on ChatGPT and provides advice to the University community on the immediate and future implications of it and other generative AI software for teaching, learning, and assessment. It is expected that this advice will be periodically updated and supplemented by other guidelines as AI and associated technologies and tools evolve, and as our teaching, learning, and assessment practices do so as well.
About ChatGPT
GPT-3.5 is described as a Large Language Model (LLM). The version of ChatGPT powered by GPT-3.5 was trained on sizeable inputs of written text and is capable of processing natural language, understanding context, analysing content and making predictions. Like other LLMs, the version of ChatGPT powered by GPT-3.5 works through modelling the statistical probability that a particular ‘token’ (in this case, words or individual characters or punctuation marks) will appear after another one: it presents information based on the statistical likelihood of a series of words appearing together. Essentially, it’s a super powerful predictive text device.
The technical report for GPT-4 describes the tool as a ‘large multimodal model’. This is because, unlike its large language model GPT-3.5 predecessor, GPT-4 can respond to both image and text inputs (though the image input function is currently only available through the paid version). GPT-4 further has the capacity to process significantly more ‘tokens’: 32,000 compared to GPT-3.5’s 8,000. This is the equivalent of approximately 50 pages of text, placing GPT-4’s processing ability at around a Masters level dissertation.
Both the GPT-3.5 and GPT-4 versions of ChatGPT have the ability to mimic human linguistic style in their presentation of information thanks to the conversational interface. Because of this, many users have fallen into the trap of anthropomorphising the technology-based tool and assuming that it “knows” what it is talking about.
ChatGPT capabilities
ChatGPT draws on a vast repository of data and information to generate new content in response to a prompt. As with other AI generative tools, the prompt could be a question, a request for an explanation, or a request to create an image or a poem. ChatGPT is more powerful than its predecessors because it has been trained on a larger data set. ChatGPT’s inputs included massive amounts of data and information drawn from the internet including books, webpages, coding information, and journal and news articles, as well as sources like Reddit discussion posts, which helped ChatGPT learn the rhythms of human dialogue. ChatGPT was then trained to use this data through a process called Reinforcement Learning with Human Feedback so that it could determine what humans expected when they asked a question.
What this means is, ChatGPT can respond in an authoritative, convincing, and human-like way to any prompt it is given. Among other things ChatGPT can:
- Generate new text based on a given prompt or seed text (this can be used for tasks like creative writing, content generation, and language translation)
- Complete a given text or sentence based on the context provided
- Summarise a given text
- Answer questions based on a given context or a text passage
- Generate responses in a chatbot or virtual assistant.
As a data-to-text model (rather than a text-to-text model), the version of ChatGPT powered by GPT-4 has the ability to interpret complex graphical inputs such as charts, and to interpret and critically analyse images. As the launch video demonstrates, GPT-4 has the capacity to generate a website based on a quick hand-drawn sketch inputted via a photograph. It can also explain what makes something funny.
OpenAI has highlighted the latest model’s improved capacity to turn down inappropriate requests or queries that could generate harmful responses.
GPT-4 also has improved ‘steerability’: users can direct the tool to take on a particular kind of personality. This was an element of GPT-3.5, but is much improved in GPT-4. Examples provided in the launch stream included asking it to act as ‘TaxGPT’ by processing and providing advice on the inputted current US tax code, and requesting the tool to act as a Socratic tutor, responding to a user’s prompts with questions rather than answers.
ChatGPT limitations
ChatGPT is a language model, so its biggest limitation is that it can’t know what it is talking about. The more data ChatGPT consumes, the better its responses become – not because it has digested the information, but because it has more data from which to base its predictions that certain words will go together. In a medical context, journalist Liam Mannix has likened talking to AI generative software like ChatGPT as “talking to an actor playing a doctor”, they may be able to mimic the correct response to a question but they are still “someone who is impersonating a doctor.”
If ChatGPT does not have enough data to formulate a response to a particular prompt, it will tell the user so. More importantly, if it has limited, incomplete, or incorrect data it will provide an incorrect (and potentially offensive) response or simply make something up. Understanding this is important to how educators begin to use such tools in their work, their teaching and learning practices, and their assessment design.
ChatGPT does make mistakes. While it can perform well with relatively simple responses, repetitive tasks, data entry, basic coding, and the like, when it comes to more complex and thoughtful tasks (such as applied coding, a research essay, or interpretation) it still requires oversight and correction by someone who understands the subject matter in order to ensure a good or ‘correct’ response. Many have also noted ChatGPT’s propensity to make up references. This includes book or article titles, journals or other publication outlets, and even DOI links.
Educational concerns
Because ChatGPT produces an original response to a given prompt, it cannot be recognised using existing plagiarism detection software such as Turnitin. As a new technology, ChatGPT has the potential to disrupt current approaches to student learning and assessment. The process of preparing material for assessment is an important part of students’ education and university experience. It allows students to apply their learning and to develop analytical, communication, and presentation skills which are highly valued by employers. This tool has the potential to undermine this process.
University policy and actions
The Office of the Provost and the Academic Board have developed a statement that clarifies the policy relating to the use of AI tools by students to produce assessment materials. The statement makes clear that if a student submits work created and /or significantly modified by AI tools for assessment as if it was their own, then this may constitute academic misconduct and will be subject to the usual academic misconduct procedures of the University.
This does not prevent students from using these tools but if a student does use AI generated material in the preparation of their assessment submission, this must be appropriately acknowledged and cited in accordance with the Assessment and Results Policy (MPF1326). Guidance has been provided to students on how to do this appropriately.
As of Semester 1, 2024 all commencing coursework students are asked to complete academic integrity education modules which include guidance on the appropriate use of generative AI. For undergraduate students this is a requirement and part of the Joining Melbourne Modules. Graduate coursework students are automatically enrolled in and encouraged to complete the Graduate Cornerstones of Good Scholarship module.
As of April 2023, the University has been using Turnitin’s AI detector tool. The tool aims to identify passages potentially generated by AI by looking for highly predictable language patterns. It sits within Turnitin’s similarity report function and is not currently visible to students (a limitation of the tool). As with similarity reports, high AI detector scores are not proof that misconduct has taken place. Please see the University guidance on using the detector tool for more information.
More generally, the Office of Student Academic Integrity is working with the DVC Academic and the Academic Board to review the University’s policies and processes with regard to student academic integrity.
University resources
Academic Integrity at the University of Melbourne – Information for staff and students, including guidance on a range of issues related to AI in the context of academic integrity.
Generative AI in Teaching Community of Practice - The Generative AI in Teaching Community of Practice focuses on the many ways generative AI may impact teaching and learning in the university setting. It is a space for academics to share ideas, learn from each other and explore the technology as it applies to our context.
CSHE - Assessment, AI and Academic Integrity – Practical advice and strategies relating to the use of generative artificial intelligence tools (such as ChatGPT) for assessment and academic misconduct.
The Generative AI Taskforce (GAIT) – The GAIT was established in 2023 to oversee the University’s response to the risks and opportunities associated with generative artificial intelligence (GenAI) tools and systems.
BEL+T Guidance on Generative AI – Discussion of the impact of AI on Built Environment, Learning and Teaching, assessment and academic integrity.
External resources
TEQSA2023 generative AI demo – Video demonstration showing the capabilities and possible applications of AI for producing work for student assessment.
TEQSA resources on artificial intelligence – Resources from TEQSA and across the sector on AI and learning and teaching and research.