Generative artificial intelligence (Gen AI), a subset of artificial intelligence focused on creating new content, has gained significant traction in various industries, including law. Capable of producing text, images, and audio, platforms like ChatGPT are among the most recognized tools in this space. While Gen AI holds great promise, it also presents unique challenges, particularly in legal settings where accuracy and reliability are paramount.
AI in Its Early Stages
Despite its potential, AI technology remains in its infancy, raising concerns about its application in professional and official contexts. The use of Gen AI to generate false or misleading information can have serious repercussions, especially in courts, where decisions hinge on precise and verifiable evidence.
A notable example of these challenges arose in Handa & Mallick [2024] FedCFamC2F 957, a case before Federal Circuit and Family Court Judge Amanda Humphreys. In this family law matter, a lawyer submitted a document listing prior court cases to support their argument. Upon review, the listed cases were found to be fictitious—a stark reminder of the pitfalls of relying on AI-generated content without proper verification.
NSW Supreme Court Practice Note SC Gen 23
In response to such issues, NSW Chief Justice Andrew Bell issued Supreme Court Practice Note SC Gen 23 on 21 November 2024, providing guidelines for the use of Gen AI in the state’s court system. Effective from 3 February 2025, the practice note establishes strict limits on how Gen AI can be applied in legal proceedings while offering general guidance for judges and legal practitioners.
Key Provisions of the Practice Note
- Definition and Scope
- Gen AI is defined as AI capable of creating new content based on training data, including text, images, and audio.
- The note clarifies that tools for spelling correction, transcription, and generating chronologies from existing documents are not considered Gen AI.
- Confidentiality and Data Security
- The practice note warns against entering confidential information into Gen AI systems, as this data may be used to train AI models and could become accessible to others.
- Information subject to non-publication or suppression orders, as well as subpoenaed or statutorily protected materials, is explicitly prohibited from being processed by Gen AI.
- Accuracy and Verification
- Legal practitioners must verify any citations, case law, or legislative references produced by Gen AI to ensure they are accurate and relevant. Responsibility for inaccuracies remains with the user.
- Restrictions on Document Generation
- Gen AI cannot be used to draft affidavits, witness statements, or character references, as these must reflect the authentic opinions of individuals.
- If exceptional circumstances warrant Gen AI use, detailed disclosures are required, including the AI program used and whether confidential or open-source data was involved.
- Expert Reports and Written Submissions
- Expert reports generated using Gen AI require prior court approval, and all references within such reports must be verified.
- Lawyers remain accountable for any Gen AI-generated content presented in court, including written submissions or summaries of arguments.
Judicial Guidelines on Gen AI Use
Chief Justice Bell also issued guidelines for NSW judges, underscoring the cautious application of Gen AI in judicial processes. Judges are prohibited from using AI to draft judgments or assess evidence. If AI tools are employed for secondary tasks, any outputs must be verified manually. Additionally, judges should be vigilant for red flags in AI-generated submissions, such as false citations or jurisdictionally irrelevant references.
Lessons from Handa & Mallick
The Handa & Mallick case exemplifies the dangers of improperly applying Gen AI in legal contexts. A lawyer used an AI-powered tool to produce a list of legal authorities, which was later found to contain fabricated cases. The court adjourned proceedings and ordered the lawyer to explain why their conduct should not be referred for investigation by the Victorian Legal Services Board. This incident underscores the importance of procedural fairness and the need for legal professionals to meticulously verify AI-generated materials.
Navigating the Future of AI in Law
As AI technology evolves, so too will its application in legal practice. The NSW Supreme Court’s guidelines represent a proactive step toward balancing innovation with responsibility. By emphasizing transparency, verification, and ethical use, the legal profession can harness the benefits of Gen AI while safeguarding the integrity of judicial processes.
The journey of integrating AI into the legal realm has just begun, but the clear message from the NSW Chief Justice is that caution and accountability must guide every step.
The information in this publication is of a general nature and is not intended to address the circumstances of any particular individual or entity. Although we endeavour to provide accurate and timely information, we do not guarantee that the information in this publication is accurate at the date it is received or that it will continue to be accurate in the future. We are not responsible for the information of any source to which a link is provided or reference is made and exclude all liability in connection with use of these sources. If you do not wish to receive newsletters from us, please let us know.