Answers

The LLMs we tested performed below the level of a qualified lawyer.

• GPT-4 and LLaMa 2 provided answers that were convincing but were frequently wrong and contained incorrect citations.
• GPT-4 performed better on some types of questions (such as those involving summarising the law) but less well on others.

Overall performance of GPT-4, Gemini 1, Claude 2, LLaMa 2 and Perplexity

Too confident or too eager

The output of all LLMs was frequently inaccurate and requires expert human review. One apparent issue is the 'sycophancy problem'*: the bias towards confirming the user's input. We also found that the 'eagerness' of the models to provide a clear answer leads to an overstated level of confidence in the advice. For example, when asked to describe the Australian competition law risks with sharing certain information disclosed by a reseller to a manufacturer, LLaMa 2 recommended confidentiality arrangements and firewalls. In fact, these are not appropriate as a general response. The correct solution instead needs to be assessed on a case-by-case basis.

A more extreme example comes from the question about former shareholder claims against third parties mentioned above. LLaMa 2 said a shareholder has a right to bring a claim against a third party for breach of contract as a 'reflective loss' claim. In fact, such claims are restricted, due to the well-known 'no reflective loss' rule. Perplexity manufactured an answer that is completely inapposite, perhaps due to a bias towards giving a 'yes' answer, and expressed it with falsely inflated confidence.

* Sharma, M, et al, Towards Understanding Sycophancy in Language Models, 27 October 2023.

GPT-4 achieved 10 out of 10 on this question. The answer given is correct and covers all material issues, and was given 5/5 for substance, 3/3 for citations and 2/2 for clarity.

Why is GPT-4 able to answer so well?

This is a good example of the ability of LLMs to summarise the law, and related guidance, in particular legal areas that are likely to be well known and discussed extensively on the internet (given they operate as an 'internet in your pocket').

The real-world answer would, of course, apply these principles to the specific characteristics of and risks for the client’s business, but this is impossible for GPT-4 to do based on the short prompt. It might also go into further detail around the procedural steps required to complete a financial assistance whitewash or specify the exemptions to the financial assistance prohibition.

There is some variance that might reflect the mix of questions. GPT-4 achieved the highest marks for banking (8.44 out of 10) and the lowest for intellectual property (3.78 out of 10).

Reasons for the variations

The difference in the marks between the various practice areas is interesting and might be partly based on the mix of questions. While all contained a mixture of straight legal questions and questions requiring an application of the facts, the banking questions were mostly 'summarisation' questions, which the LLMs generally find easier to answer.

In contrast, the intellectual property questions required the application of the law to specific facts and touched on issues that are likely to be less 'commonplace', ie less widely discussed on the internet. As the LLMs (at least partly) learn from internet content, they might be able to more easily parrot back the answers to summarisation questions.

Little Australia

Another issue appears to be peculiar to Australia (and other 'smaller' jurisdictions). The October 2023 LinksAI report postulated that the performance of LLMs might be worse for 'less common' legal systems. Our results bear this out, at least in less-harmonised areas of law such as intellectual property, as answers frequently incorporated EU and UK law analysis that is not correct for Australian law. This effect is less severe in areas with greater harmonisation, such as banking.

It is curious, however, that less 'infection' by US law is seen in the responses, compared with EU and UK law. It may be that LLMs have 'learned' that Australian lawyers, judges and governments generally treat UK sources as more appropriate references in an Australian context than US law. This includes pre-Brexit UK jurisprudence that was based on EU law.

No central moderation

Equally, it is possible that the higher marks for some practice areas are because the questions were easier and the marking more generous. There is a degree of subjectivity in both designing the questions and the marking process. However, because the subject matters are technical, we did not attempt to centrally moderate the difficulty of the questions, or the marks applied in relation to substance.