Sullivan & Cromwell admits AI hallucinations in US court filing

Sullivan & Cromwell has disclosed to a US bankruptcy court that a recent filing contained inaccurate legal citations, including errors generated by AI tools.

The issue arose in bankruptcy proceedings involving Prince Global Holdings, where the firm is acting for court-appointed provisional liquidators.

In a letter dated 18 April to Chief Judge Martin Glenn in New York, the firm said an emergency motion filed on 9 April included “inaccurate citations and other errors”, some of which were the result of AI hallucinations.

AI failure

The firm described hallucinations as instances where AI systems fabricate case law, misquote authorities or generate non-existent sources, and said we “deeply regret that this has occurred”.

The errors were identified in detail in a schedule attached to the letter, which lists more than 40 corrections.

Policy breach

Sullivan & Cromwell said it maintains “comprehensive policies and training requirements” for AI use, including mandatory training before lawyers can access generative AI tools.

Those policies require lawyers to independently verify all AI-generated output and emphasise the risks of fabricated citations and misinterpretation of legal authorities.

The firm acknowledged that these protocols were not followed in this instance and that the failure extended to its broader review and supervision processes.

Remedial steps

The firm said it has undertaken a full review of the circumstances and rechecked all filings in the matter, confirming that no further AI-related errors were identified.

It has also filed a corrected version of the motion and said it is assessing whether additional enhancements to its training and review processes are needed.

The letter adds that the firm has apologised for the burden on the court and to opposing counsel, US litigation firm Boies Schiller Flexner, who first raised the issue.