← All articles

Underestimated Potential

Anyone who works as a lawyer with judgments, contracts or written pleadings knows the usual way of proceeding. The potential of generative AI chatbots based on large language models (LLMs), however, is often underestimated in legal work.

Used correctly, such tools can help to extract large volumes of relevant information and to grasp legal texts more quickly. What matters here is not only the tool itself, but above all how it is used and the quality of the prompt. In legal research in particular, this can contribute to a considerably more efficient way of working.

Previous Studies and Research

There are already many approaches that attempt to obtain meaningful results on the use of LLMs in a legal context by means of targeted metrics and benchmarks. For example, the LEGALBENCH project attempted to create a basic classification on the basis of 162 given tasks.1 In essence, it found that different techniques in entering the prompt can change the output considerably. A project that developed a reusable prompting framework2 reached the same conclusion.

It was found, for example, that a context-layered3 or chain-of-thought prompt4 is much better suited to generating an adequate and, above all, more detailed and less superficial result. With a zero-shot prompt or few-shot prompt,5 by contrast, the risk of hallucination was found to be particularly high.6 The reason is obvious: essential information, details or other points of reference are not accessible or are missing altogether. The LLM may fill such gaps with false information. This is highly problematic and risky.

It must be borne in mind, however, that the studies and investigations mentioned were generally developed on the basis of American or British law.7 These legal systems differ fundamentally from the German legal system. German law is precisely not based on the common law. Supreme-court decisions of the Federal Court of Justice (Bundesgerichtshof, BGH) have considerable guiding and orientating effect. They do not, however, replace interpretation and subsumption under the relevant provisions. Decisions of the Federal Constitutional Court, by contrast, have a special binding effect. Under Section 31(1) of the Act on the Federal Constitutional Court (Bundesverfassungsgerichtsgesetz, BVerfGG), its decisions are binding upon the constitutional organs of the Federation and of the Länder as well as on all courts and those with public authority. In contrast to a strict system of precedent, judicial decisions in Germany are therefore, as a general rule, not a binding source of law in the same way. This also means that the way of working in law differs.

When it comes to evaluating prompts and using LLMs correctly, a general, one-size-fits-all guide is therefore not helpful. This also applies to other areas of the legal system, such as the taking of evidence or the hearing and legal argumentation in general. In German law, moreover, depending on the type of proceedings there are separate duties of investigation, of presenting the facts and of clarification, which cannot simply be compared with Anglo-American procedural models.

Common Approach

In the LegalBench project, two metrics were mainly used: first, correctness and, second, the ability to analyse.8 Correctness as meant there corresponds to the output achieved, taking into account logical, arithmetical and other legal errors made by the LLM. Analysis refers to the specific conclusions and propositions on which an answer is based.9

An LLM is generally based on neural networks that operate on the basis of large amounts of training data. Specifically, the AI model is trained to recognise patterns in language and data and, on that basis, to generate probable continuations. This does not necessarily lead to a high-quality result. In law, case-by-case decisions and an exact subsumption are precisely what is required.

It has been found that LLMs can achieve better results if, for example, a rule of interpretation is explained first. The use of simplified language, as compared with technically sophisticated language, can also have a positive effect on performance. Here too, however, there are considerable differences, not only between different LLMs but also between versions within commercial model series.10

Metrics for Evaluating the Results

The different basic features of legal systems, and with them the different ways in which lawyers work abroad as compared with Germany, have already been pointed out. The development of one’s own metrics is of considerable importance here and forms a cornerstone of a meaningful evaluation of this topic.

In the present investigation, two metrics were used. On the one hand, citations of provisions and the correct identification of chains of provisions (Normenketten) were evaluated. This refers to the chain of provisions that is decisive for resolving the case at hand. Admittedly, such chains of provisions may differ from one another depending on the facts of the case and on interpretation. Nevertheless, there are common interpretations of provisions that have become established over time through case law, academic literature and the application of the law. On the other hand, here too the correctness of the answer was evaluated in the sense described above.

The work was mainly done with ChatGPT (GPT-5.5) and Claude (Sonnet 5). The choice of these metrics is based on the following consideration: irrespective of the area of law concerned, a lawyer, judge or public prosecutor must first of all know and name the relevant provisions.

An example: if the public prosecutor is simply not aware of Section 6 of the Compulsory Insurance Act (Pflichtversicherungsgesetz, PflVG), he or she cannot properly examine or investigate a corresponding criminal offence either. If opposing counsel bases a legal argument on a provision that the other party was not previously aware of, this can be an indication of a legally relevant or strong argument. Logically, further factors play a role here. In principle, the correct use of an LLM can represent a considerable opportunity to improve legal work in certain areas in a lasting way.

Findings from Practical Testing

The present investigation is no substitute for a comprehensive empirical study. Rather, it consists of experiments that aim to classify different prompting techniques systematically across different use cases and to evaluate them in terms of their legal quality.

First, a case from the law of sales was used. The case concerns a claim for cure in the form of remedying the defect. It comes from the third semester and is about half a page long. First, a zero-shot prompt was run. It contained the instruction to solve the case and to give an assessment in expert-opinion style (Gutachtenstil).

As a result, the essential provisions, Sections 437 no. 1 and 439(1) alt. 1 of the German Civil Code (Bürgerliches Gesetzbuch, BGB), were correctly identified. The specific subsumption and the approach to the case, however, remained superficial. Different prompting techniques changed the result only to a limited extent. An LLM is suitable, at the very least, for checking a specific legal result or for providing a first entry point into a case in order to identify the essential provisions. In a comparison with further cases from criminal law or public law as well, the LLM had no significant problems identifying individual relevant provisions.

The precise application of provisions, which in an individual case can differ considerably through the change of a single detail, must nevertheless continue to be carried out by a lawyer. This finding is not entirely new. But it once again illustrates the potential and the limits of use.

In a considerably longer case on the owner–possessor relationship (Eigentümer-Besitzer-Verhältnis, EBV), for example, a subsection of Section 990 BGB was cited that does not exist in the BGB. This is a classic hallucination problem. It cannot reliably be inferred from this which training data the model lacked. The passage does show, however, that the model could not reliably verify the structure of the provision and therefore generated a non-existent reference to a provision.11 This does not necessarily mean that an LLM is worthless for such questions. It could, for example, help to search for, structure or summarise interesting judgments and literature on an EBV problem, provided that the underlying sources remain verifiable.12

An Illustrative Example

A further illustrative example is the judgment of the Court of Justice of the European Union (CJEU) of 7 December 2023 in Case C-634/21, SCHUFA Holding (Scoring),13 delivered on a reference for a preliminary ruling under Art. 267 TFEU. The subject of the proceedings was, among other things, the question whether the automated establishment, by a credit information agency, of a probability value concerning a person’s ability to meet payment commitments in the future constitutes “automated individual decision-making” within the meaning of Art. 22(1) GDPR.

The CJEU clarified that this is the case where a third party to which that probability value is transmitted draws strongly on it to establish, implement or terminate a contractual relationship with the person concerned, for example when granting a loan.14

The decision is of considerable importance for the interpretation of Union law by national courts. Its practical relevance is also evident in proceedings before the BGH concerning claims for access against SCHUFA and the question of what information data subjects must receive about the calculation and significance of score values.

For lawyers who advise or litigate in comparable data protection proceedings, the CJEU’s reasoning can therefore be of particular importance.

The decisive point is therefore not only whether an LLM is used, but how it is used. The following aspects may be relevant here:

  • the aim of the legal research, for example the analysis of a contract, a judgment, a written pleading, a correspondence or other documents,
  • the relevant area of law,
  • the form of the expected result,
  • and above all the traceability and derivation of the result. This shows that the metric of the ability to analyse (see above) is particularly valuable.

If the judgment to be examined is provided as source material, an LLM can generally answer the following questions without major problems:

  • What decision did the CJEU reach in Case C-634/21?
  • Which question or questions referred for a preliminary ruling underlay the proceedings?
  • What is the central legal problem of the decision?
  • Who were the parties to the main proceedings?
  • Which further provisions of the GDPR are relevant to the decision?
  • Which essential arguments can be derived from the decision for German proceedings concerning claims for access?

If one now wishes to compare other relevant judgments, an LLM cannot (yet) draw the corresponding connections reliably without further ado. It would, for example, look for similarities if it can find comparable judgments in the course of an in-depth web search. Here too, the serious hallucination problem arises. In the tests carried out here, both ChatGPT and Claude tended to cite judgments that do not actually exist.

To be able to use an LLM sensibly in this context, one could, for example, access a case-law database of the BGH or open databases such as openJur. If such research is also to be automated, further data protection points must be observed. As a rule, the risk of hallucination is lowest when summarising information that has been provided.15

On the internet and on platforms such as YouTube, one comes across various articles and videos that claim to be able to supply the “right guide”. Often a not inconsiderable subscription is attached. Tables with the supposedly “right prompts” are frequently printed there. As already explained, such a presentation cannot be accurate, at least where it promises a universal solution.

Not only the commercial model chosen but also the area of law, the type of task and the user’s understanding of the task significantly change the output. ChatGPT responds less to mere role assignments and abstract ideas than to context-layered prompts. This can also be helpful when filtering contract clauses.16

A prompt can and need not always lead directly to the desired result. Often several attempts, intermediate steps and points of reference are needed to turn an LLM into a truly helpful instrument.

This is not to say, of course, that all guides available online are useless. Caution is simply called for. Such guides can be interesting for someone who is engaging with the topic for the first time and is looking for a starting point. However, where there is no understanding of how LLMs work and of the particularities of legal work, the actual purpose of prompting techniques is often misunderstood.

Outlook

Artificial intelligence can be used sensibly in legal practice to structure legal texts more quickly. Particularly with extensive judgments, contracts or written pleadings, it can offer considerable added value.

At the same time, AI remains a supporting tool. The quality of the output depends on the factors mentioned. It is not suitable for every use case and replaces neither legal subsumption nor the lawyer’s responsibility.

Prompt engineering is therefore a practical skill that should be trained. Its value lies not in supplying a single true formula, but in being able to realistically assess and make use of the possibilities and limits of generative AI in the respective legal context.


  1. Guha et al., LEGALBENCH: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models, 2023. ↩︎

  2. Krishna et al., A Reusable Prompting Framework for Applying Large Language Models to Legal Tasks, 2025. ↩︎

  3. As the name suggests, a context-layered prompt is intended to explain the circumstances of the specific task in more detail. For example, the aim, purpose, persons involved and other aspects can be stated. Cf. Krishna et al., A Reusable Prompting Framework for Applying Large Language Models to Legal Tasks, 2025, p. 3114. ↩︎

  4. A chain-of-thought prompt resembles the human thought process. It provides a series of pieces of information by which the LLM can orient itself. Cf. Krishna et al., A Reusable Prompting Framework for Applying Large Language Models to Legal Tasks, 2025, p. 3114. ↩︎

  5. A zero-shot prompt does not provide any examples or other patterns of action. An output is regularly achieved through several prompts, i.e. few-shot prompts. Cf. Krishna et al., A Reusable Prompting Framework for Applying Large Language Models to Legal Tasks, 2025, p. 3114. ↩︎

  6. Krishna et al., A Reusable Prompting Framework for Applying Large Language Models to Legal Tasks, 2025, pp. 3117, 3121; Microsoft, Prompt Engineering for Lawyers (2nd Edition), 2025, pp. 19, 30 et seqq., 43 et seq.; Yang/Li/Gong, Legal Knowledge Generation Based on LLM Prompt Engineering, CLOUD 2025, pp. 16-31; Guha et al., LEGALBENCH: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models, 2023, p. 5. ↩︎

  7. Cf. also Mielke/Wolff, „KI, LÖS‘ MIR DEN FALL!“, Jusletter IT 2025, pp. 137 et seq.; see for example Policarpio, ChatGPT Prompts for In-House Lawyers & Legal Teams: A Practical Guide, available at: https://www.legalontech.com/post/chatgpt-prompts-for-lawyers (last accessed on 26.06.2026). ↩︎

  8. Guha et al., LEGALBENCH: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models, 2023, pp. 6, 34; cf. also Krishna et al., A Reusable Prompting Framework for Applying Large Language Models to Legal Tasks, 2025, p. 3118; Weidinger et al., AQuAECHR: Attributed Question Answering for European Court of Human Rights, Findings of the Association for Computational Linguistics 2025, p. 6. ↩︎

  9. Guha et al., LEGALBENCH: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models, 2023, p. 10; Krishna et al., A Reusable Prompting Framework for Applying Large Language Models to Legal Tasks, 2025, p. 3121. ↩︎

  10. Guha et al., LEGALBENCH: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models, 2023, p. 10. ↩︎

  11. Weidinger et al., AQuAECHR: Attributed Question Answering for European Court of Human Rights, Findings of the Association for Computational Linguistics 2025, p. 18. ↩︎

  12. Wardas/Matthes, LLMs for Legal Subsumption in German Employment Contracts, 2025, p. 4. ↩︎

  13. Judgment of the Court of Justice of the European Union (CJEU) of 7 December 2023 (Case C-634/21) – infocuria. ↩︎

  14. Judgment of the Court of Justice of the European Union (CJEU) of 7 December 2023 (Case C-634/21) – infocuria, para. 73. ↩︎

  15. Dalerci et al., MÖVE: A Holistic LLM Benchmark for the German Public Sector, 2026, p. 63; Krishna et al., A Reusable Prompting Framework for Applying Large Language Models to Legal Tasks, 2025, p. 3126. ↩︎

  16. Krishna et al., A Reusable Prompting Framework for Applying Large Language Models to Legal Tasks, 2025, p. 3127; cf. also Al Zubaer et al., GradeLegal: Automated Grading for German Legal Cases, 2026, p. 4, pp. 7-8; Mizrahi et al., State of What Art? A Call for Multi-Prompt LLM Evaluation, 2024, Wardas/Matthes, LLMs for Legal Subsumption in German Employment Contracts, 2025, p. 6. ↩︎

Suggested citation

Zierhut, Annika S.: Prompt Engineering for Lawyers, Zierhut Jura, July 5, 2026, https://zierhut-jura.de/en/artikel/prompt-engineering-for-lawyers/.