Retrieval-Augmented Detection of Potentially Abusive Clauses in Chilean Terms of Service

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

Retrieval-Augmented Detection of Potentially Abusive Clauses in Chilean Terms of Service

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

machine learning, deep learning, neural networks

Retrieval-Augmented Detection of Potentially Abusive Clauses in Chilean Terms of Service Christoffer Löffler 1*, Tomás Rey Pizarro 2 , Daniel Ignacio Miranda Vásquez 1 , Andrea Martínez Freile 2* School of Computer Engineering, Pontificia Universidad Católica de Valparaíso, Brasil 2950, Valparaíso, 2340025, Chile. 2 Faculty of Law, Universidad Adolfo Ibáñez, Av. Padre Hurtado 750, Viña del Mar, 2581793, Chile.

arXiv:2605.26019v1 [cs.LG] 25 May 2026

*Corresponding author(s). E-mail(s): [email protected]; [email protected]; Contributing authors: [email protected]; [email protected]; Abstract Online Terms of Service often function as contracts of adhesion, creating asymmetries that may expose consumers to potentially abusive clauses. In Chile, assessing such clauses is legally challenging because some provisions clearly violate mandatory consumer law, whereas others depend on broader standards such as good faith and contractual imbalance. We present a retrieval-augmented generation framework for the automated detection and classification of potentially abusive clauses in Chilean Terms of Service. Designed for local execution, it combines efficient clause detection, hybrid dense–sparse retrieval, reranking, and prompt augmentation to support medium-sized open-weight language models. We also introduce the Chilean Abusive Terms of Service Extended corpus, comprising 100 contracts and 10,029 annotated clauses in 24 legally grounded categories spanning illegal, dark, and gray clauses. Experiments comparing commercial and openweight language models, fine-tuned encoders, and traditional baselines show that retrieval-augmented prompting substantially improves performance and enables local models to approach larger cloud-based systems at lower computational and token cost. The study also contributes a refined legal annotation scheme and a practical design for AI-assisted consumer contract review. Keywords: Consumer protection law, Abusive clauses, Deep neural networks, Retrieval augmented generation

1 Introduction Consumer contracts are marked by a structural asymmetry between providers and consumers (Baraona González 2014; Jara Amigo 1999). In Chilean consumer law, this asymmetry is especially visible in contracts of adhesion, whose terms are drafted unilaterally by the provider and offered on a take-it-or-leave-it basis (Jara Amigo 1999; de la Maza Gazmuri 2003; SERNAC 2021). In digital markets, this contractual form is commonly embodied in online Terms of Service (ToS), which consumers routinely accept without reading or understanding them in detail (SERNAC 2021; Löffler et al. 2025). As a result, consumers may agree to clauses that restrict remedies, alter procedural rights, shift risks, or otherwise undermine the level of protection guaranteed by mandatory law (Barrientos Camus 2019; Löffler et al. 2025). This problem is not merely practical but doctrinal. Chilean Law No. 19.496 on the Protection of Consumer Rights (LPC) regulates abusive clauses in adhesion contracts and establishes consequences for clauses that are contrary to the statute (Barrientos Camus 2019; Löffler et al. 2025). Yet legal assessment is not equally straightforward across all cases. Some clauses are relatively easy to identify because they directly contradict express legal norms. Others are presumed abusive because they generate manifest contractual imbalance. Still others fall within open-textured standards, most notably the requirement of good faith under Article 16(g) LPC, and therefore require contextual legal interpretation (Barrientos Camus 2019; Morales 2018; Löffler et al. 2025). For consumers, and often even for experts, these distinctions are difficult to apply in the face of lengthy, heterogeneous, and constantly changing digital contracts. The practical consequences of this difficulty are significant. Individual consumers typically lack both the legal expertise and the economic incentive to review ToS clause by clause, giving rise to what consumer-protection doctrine has described as a collective-action dilemma (Jara Amigo 1999). Because the transaction costs associated with seeking legal advice or pursuing litigation often exceed the monetary value of the underlying service, consumers are effectively discouraged from defending their rights. This imbalance is therefore not merely informational but also institutional: individuals cannot realistically be expected to rely on private litigation as their primary safeguard against problematic contractual terms. This socio-economic reality helps explain why Chilean consumer law has developed as a special protective regime (Baraona González 2014) and why automated tools may play a useful role in supporting the consumer’s right to information and informed contractual choice (SERNAC 2021). While such tools cannot replace formal judicial review, which remains necessary to determine the legal effect of a disputed clause, they can substantially reduce the practical barriers users face when confronting lengthy digital contracts and help identify provisions that warrant closer scrutiny (Löffler et al. 2025). Prior work has shown that Machine Learning can assist in the detection of potentially unfair clauses in consumer contracts, beginning with supervised approaches such as CLAUDETTE for European contracts (Lippi et al. 2019) and extending to memoryaugmented models (Ruggeri et al. 2022), retrieval-based systems (Dadas et al. 2024), and large language models (Löffler et al. 2025). A previous study on Chilean ToS adapted this line of research to the Chilean legal context by constructing an annotated corpus grounded in domestic consumer doctrine and by comparing fine-tuned 2

encoders with prompted large language models (Löffler et al. 2025). That study also revealed an important limitation: as legal categories become more specific and doctrinally faithful, annotation and classification become more difficult, especially for open-ended categories linked to good faith and other interpretive standards (Löffler et al. 2025; Barrientos Camus 2019). This paper addresses that challenge through two complementary contributions. First, we refine the legal taxonomy used to annotate abusive clauses in Chilean ToS. Building on the previous 20-category scheme (Löffler et al. 2025), we introduce an expanded and revised 24-category framework that better distinguishes between clauses that are illegal, dark, and gray, and that reduces redundancy among partially overlapping labels. Second, we propose a retrieval-augmented generation (RAG) framework (Lewis et al. 2021) that improves the classification of potentially abusive clauses while remaining deployable on local consumer hardware. The framework combines efficient clause detection, hybrid dense–sparse retrieval, reranking (Gao et al. 2024), and prompt construction based on legally annotated examples, thereby enabling medium-sized open-weight models to perform competitively without exclusive reliance on cloud-based systems. The paper therefore makes both legal and technical contributions. On the legal side, we operationalize Chilean consumer-law doctrine in a refined annotation scheme for digital adhesion contracts and make explicit the different levels of legal certainty involved in abusive-clause analysis (Barrientos Camus 2019; Löffler et al. 2025). On the technical side, we show that retrieval-augmented prompting can provide a more efficient and more legally grounded alternative to large static few-shot prompts for this task (Gao et al. 2024; Löffler et al. 2025). The resulting framework is implemented in a browser extension that supports real-time review of ToS by highlighting potentially problematic clauses, their possible legal basis, and analogous annotated examples. Our specific contributions are: • We present a retrieval-augmented framework for the local detection and classification of potentially abusive clauses in Chilean ToS. • This paper introduces the Chilean Abusive ToS Extended corpus, a Spanishlanguage dataset of 100 contracts and 10,029 annotated clauses. • We refine the legal annotation scheme from 20 to 24 categories by clarifying ambiguous labels, reducing redundancy, and adding recurrent Chilean consumer-law issues. • We compare commercial LLMs, open-weight LLMs, fine-tuned encoders, and traditional baselines across detection and classification tasks. • We demonstrate a practical browser-based implementation for privacy-preserving consumer contract review. The remainder of the paper is structured as follows. Section 2 presents the legal background. Section 3 formulates the computational tasks. Section 4 describes the proposed framework. Section 5 introduces the extended corpus and annotation scheme. Section 6 reports the experiments. Section 7 discusses the findings in relation to prior work in AI and law. Section 8 concludes.

2 Legal Background 2.1 Consumer Contracts, Adhesion, and Digital Terms of Service Chilean consumer law is built on the recognition of an asymmetric relationship between providers and consumers (Baraona González 2014; Jara Amigo 1999). Law No. 19.496 on the Protection of Consumer Rights (LPC) defines consumers as the final recipients of goods or services and providers as the entities that habitually offer them in the market. In contrast to the classical civil-law model of negotiated agreement, consumer transactions frequently take the form of contracts of adhesion, in which the provider unilaterally drafts the contractual terms and the consumer can only accept or reject them (Jara Amigo 1999; de la Maza Gazmuri 2003; SERNAC 2021). In the digital environment, Terms of Service are a paradigmatic instance of this contractual form (SERNAC 2021; Löffler et al. 2025). Their practical significance lies not only in their prevalence but also in their opacity: they are often lengthy, technically worded, and accepted without meaningful reading. This creates a risk that consumers will assent to clauses that restrict rights, impose disproportionate burdens, or conflict with mandatory consumer law (Jara Amigo 1999; Barrientos Camus 2019). For that reason, the regulation of adhesion contracts is central to consumer protection (Baraona González 2014). The problem is particularly acute in cross-border digital contracting, where providers often deploy standardized global ToS without adapting them to Chilean consumer law. As a result, contracts may contain provisions on foreign jurisdiction, mandatory arbitration, provider-controlled dispute mechanisms, or other restrictions that are difficult to reconcile with domestic protections afforded to consumers. These features illustrate why the legal analysis of ToS cannot be reduced to abstract contractual interpretation alone, but must be situated within the mandatory framework of consumer law and access-to-justice guarantees (Barrientos Camus 2019; SERNAC 2021; Löffler et al. 2025).

2.2 Abusive clauses under the LPC The LPC addresses abusive clauses primarily in Article 16 and related provisions governing adhesion contracts (Barrientos Camus 2019; Löffler et al. 2025). Some clauses may be regarded as ineffective because they conflict with explicit legal prohibitions; others are presumed abusive because they produce a manifest imbalance to the detriment of the consumer. In addition, Article 16(g) introduces a broader standard linked to good faith and significant imbalance in contractual rights and obligations. This provision is especially important because it functions as an open category capable of capturing abusive contractual practices not exhaustively specified elsewhere in the statute (Barrientos Camus 2019; Morales 2018; Löffler et al. 2025). From an AI and law perspective, this legal structure is important because it entails different levels of interpretive difficulty. Clauses that directly contradict an express rule are comparatively easy to operationalize. By contrast, clauses whose abusiveness depends on good faith, context, or contractual purpose are inherently harder to

annotate and classify (Barrientos Camus 2019; Löffler et al. 2025). Any automated system for consumer-contract review must therefore account for the distinction between rule-like violations and open-textured legal standards.

2.3 Hierarchy of Abusiveness Following the previous work (Löffler et al. 2025), we organize potentially abusive clauses into three groups. • Illegal clauses are provisions that directly contradict an explicit legal norm to the detriment of the consumer. Their legal basis is comparatively determinate, and they can often be identified by contrast with a written prohibition (Löffler et al. 2025). • Dark clauses are not necessarily illegal in the same direct sense, but they are manifestly abusive and are treated by the legislator as presumptively unfair because they reflect a pronounced imbalance in contractual power. Typical examples include unilateral modification, unjustified termination, or anticipatory limitation of liability (Barrientos Camus 2019; Löffler et al. 2025). • Gray clauses are clauses whose abusiveness is less immediate and depends on interpretation, especially under Article 16(g) LPC and the principle of good faith. These clauses are the most difficult both for legal experts and for automated systems because their assessment depends more heavily on context, purpose, and evidentiary argument (Barrientos Camus 2019; Löffler et al. 2025). This hierarchy is not intended to substitute for judicial determination. Whether a specific clause is legally ineffective or abusive in a particular dispute remains a matter for courts (Barrientos Camus 2019; Löffler et al. 2025). Rather, the hierarchy provides a doctrinally grounded structure for annotation and for AI-assisted identification of clauses that may deserve closer legal scrutiny.

3 Problem Formulation We formulate the analysis of potentially abusive clauses as a classification task, where the dataset D = (Xi , yi )N i=1 consists of the textual representation Xi of a clause from a contract and yi is the target vector, with dimensionality equal to the number of classes in the task. We distinguish two tasks: i) the detection of potentially abusive clauses predicts a binary vector yi with either "okay" or potentially "abusive" (two classes or T = 2), and ii) the classification of potentially abusive clauses predicts a multi-class vector yi (with more than two classes T > 2) that can encode multiple labels per instance, with each clause potentially representing at least one form of potential abuse. Several learning paradigms were proposed to approach the ToS problem on different datasets, i.e., supervised classification (Lippi et al. 2019) and a memory-augmented variant (Ruggeri et al. 2022), Nearest-Neighbor clustering (Dadas et al. 2024), and few-shot prompting of generative models (Löffler et al. 2025). We briefly summarize each formulation in the following paragraphs.

Formulated as a supervised classification problem (Lippi et al. 2019; Ruggeri et al. 2022), let f1,θ : X → y be the model f1,θ over trainable parameters θ that outputs the vector of predicted class probabilities ŷ. This model may be, for example, a transformer-based architecture such as BERT (Devlin et al. 2019) that outputs SoftMax probabilities, or a Support Vector Machine (Cortes and Vapnik 1995). For neural networks, we train f1,θ on the dataset D using the Cross-Entropy loss PN PT L = − N1 i=1 t=1 yi,t log (ŷi,t ) for either a detection task or classification task, e.g., T may be six or nine classes for the Chilean Abusive ToS Extended dataset, see Section 5. Formulated as a clustering problem (Dadas et al. 2024), we first project the raw high-dimensional text samples Xi into a lower-dimensional embedding space. Let f2 : X → y be the embedding model f2 that predicts an embedding vector ei that encodes semantic information about the original meaning of the clause Xi . We use the model f2 to generate a database E = (ei , yi )N i=1 for the dataset D. For any new clause Xj ∈ / D, we generate its embedding ej = f2 (Xj ) and then use a distance metric, such as the cosine distance, to find the top-k nearest neighbors in the annotated database E. Embedding models like E5 (Wang et al. 2024b,a) are trained for retrieval tasks and preserve sufficient semantic similarity to permit meaningful retrieval using cosine distance. Because absolute similarity scores are not straightforward to interpret, a majority-vote mechanism over the top-k retrieved instances and their reference labels is used to classify the clause Xj . The formulation as a few-shot learning problem (Löffler et al. 2025) extends the clustering approach but does not rely on a precomputed database E. Instead we adhoc generate the database S = ∪C c=1 (f2 (Xc,i ), yc,i ) for each query Xj . Here, S is the support set for the classes C and their provided annotated samples are the tuples (Xc,i , yc,i ). For example, few-shot learning with K = 5 samples per class is called 5-shot learning. The class prediction of Xj can be performed via a nearest neighbor majority vote using cosine distance. When using LLMs, supervised fine-tuning as usual for smaller language models like BERT is impractical due to cost considerations. Instead, few-shot learning is employed to generate prompts that provide context to classify query clauses at inference time. Let f3 : X → y be the LLM that can be prompted with a query sample Xj and the support set S. Then, the predicted classes ŷtokens = f3 (Xj , S ) are generated as a sequence tokens instead of a class prediction vector ŷ and parsed for interpretation. With K = 0 samples in S, an LLM can only rely on pre-training. As K increases, the sequence length may become infeasible, depending on the model’s context window and cost constraints. On the basis of these definitions, the next section proposes a framework to effectively and efficiently address the problem of detecting and classifying potentially abusive ToS.

4 Methodology We propose a RAG-based framework of locally deployable models for the automated review of online Terms of Service.

We introduce the Chilean ToS Assistant and explain its architecture in Section 4.1. Section 4.2 then describes our enhanced prompting strategy based on RAG, which achieves higher scores while requiring less computation and fewer tokens than cloudbased LLMs.

4.1 Processing Pipeline We propose an efficient Machine Learning framework to locally analyze potentially abusive clauses in Chilean Terms of Service. The architecture consists of three principal modules, i.e., the "User Interface", the "Detection" module and the "Classification" module. Figure 1 shows the data flow inside and between the modules. User Interface

Query Contract

Detection

chunking

Query Query Query Query Clauses Clauses Clauses Clauses

Classification

Predicted Class

Gen. Model

detect

Detection Models retrieve

Flagged Clause(s)

Flagged Prompt Prompt Prompt Clause augment Knowledge prompt Base

Fig. 1: The processing pipeline first detects potentially abusive clauses and then classifies them using RAG-based prompting. The user interface is implemented as a companion side panel for web browsers, see Figure 2. Users can initiate a scan of the currently displayed website, which is performed locally on their device. When processing is complete, the results are presented as a list of potentially abusive clauses and include a reference to the potentially applicable legal provision, a brief explanation, and similar annotated clauses from the knowledge base. To enable processing on standard consumer laptops without reliance on powerful cloud resources, we optimize the efficiency of each processing step. First, in a divide and conquer strategy, we chunk the contract currently under study using hierarchical chunking (Jaiswal et al. 2025; Jain et al. 2025). This takes the HTML website’s structure into account. Furthermore, it avoids issues of ixed-size chunks that may disrupt clause-level semantics, and is computationally much cheaper than semantic chunking, that iteratively groups semantically similar embeddings. Future work may investigate more advanced strategies, such as adaptive chunking via language models (Jain et al. 2025). 7

Next, the "Detection" step filters the individual clauses or paragraphs using an efficient detection model. This model may be a simple SVM with TF-IDF features, which is computationally efficient, or a small language model, such as multilingual BERT (Pires et al. 2019). Only clauses flagged as potentially abusive are analyzed in the next step. The filtering substantially reduces the cost of subsequent steps due to the ratio of "okay" and potentially "abusive" clauses of about 8:2. The "Classification" step requires more powerful language models, that previously needed cloud compute (Löffler et al. 2025). With our framework, we reduce these requirements by constructing an enhanced prompt based on RAG that contains more precisely selected examples and a substantially shorter context length, see Section 6. This enables smaller local LLMs to approach the classification quality of larger cloudbased models. Our RAG-based method retrieves the most similar instances to the flagged query from our annotated knowledge base, see Section 4.2 for the complete description. Next, we augment the prompt to a local LLM to classify the flagged clause. Finally, the resulting analysis is displayed in the user interface together with a brief explanation and a hyperlink to the potentially applicable legal provision, enabling the user to make informed decisions.

Fig. 2: The ToS Assistant is implemented as a browser extension and presents the results of the analysis as a list of potentially abusive clauses in a side panel.

4.2 Retrieval-Augmented Generation This section describes the prompting enhancements that enable the local classification component of our framework. Notably, previous work suggested that reliable classification of potentially abusive clauses requires large, cloud-based LLMs that have to be queried with costly few-shot learning with a high shot count, e.g., OpenAI’s GPT4o (OpenAI 2024) with large prompts containing up to 10 shots (Löffler et al. 2025). However, this requires substantial computational resources and increases inference cost. Instead, we propose to integrate RAG (Lewis et al. 2021) to construct more precise prompt contexts that enable even smaller models to achieve higher accuracy at lower cost and size (Ram et al. 2023). Figure 3 outlines the components in detail. The method works as follows: in an initial step we prepare the "vector store". The second part is at inference time and consists of the three steps "retrieval", "augmentation" and "generation". Vector store

Annotated Contracts embed

Embed. Models store

Vector Store

Retrieve

Augment

Generate

Query Clause

Query Prompt Prompt Prompt Clause prompt

embed

Embed. Models

Annotated Annotated Annotated Clause Clause Clauses

Generative Model respond

Predicted Class

Ranker

Fig. 3: Our framework utilizes a vector store for its knowledge base, paired with a reranker to refine retrieval precision. By merging the query with these retrieved samples, we create an augmented prompt that enables conditional generation via nonparametric memory. In the first step the vector store E = (ei , yi )N i=1 is constructed. The framework supports both dense and sparse embeddings, including a hybrid of both variants. Dense vectors can be generated using neural networks such as the multilingual variant of E5 (Wang et al. 2024b) or OpenAI’s text-embedding-3-large (OpenAI 2024), while sparse embeddings refer to classical term frequency-based methods like BM25 (Robertson and Zaragoza 2009) retrieval function that uses bag-of-words features. Regardless of the variant, the embedding model f2 embeds all annotated contracts’ clauses in D to a lower-dimensional representation and we add them to the database E = (ei , yi )N i=1 . 9

The vector store implements fast similarity search over the embedding vectors, and different products are available, e.g., FAISS (Douze et al. 2025) or Qdrant (Qdrant 2026). E forms the knowledge base supporting the LLM’s predictions and acts as non-parametric external memory. During inference, we first retrieve relevant knowledge, then we augment the query prompt with the most suitable clauses, and finally generate the prediction. In the retrieval step f2 embeds the query clause Xj and we look up the top-P most similar annotated clauses in the vector store, e.g., using cosine distance for dense embeddings. Using a hybrid knowledge base E with both dense and sparse embeddings, we merge the retrieved lists of 2 × P clauses. This list represents a wide selection of candidate clauses. Then, a reranker (Glass et al. 2022) determines the actual relevance of the candidates, predicting a similarity score between the query-text sequence Xj and any retrieved clause text to select the most relevant samples, resulting in a more precise retrieval. Rerankers are often based on Sentence-BERT (Reimers and Gurevych 2019), such as the families of models MS MARCO(Bajaj et al. 2018), or Jina (Wang et al. 2025). These models effectively implement a semantic, that is, neural, variant of the lexical BM25 algorithm (Lu et al. 2025). Mixing embedding types and using hybrid RAG can enhance results, because sparse and dense features capture different relevance features (Gao et al. 2024). They can retrieve complementary samples from the vector store. In our imbalanced dataset, the sparse component handles rare cases better, potentially enhancing robustness in a context with specialized jargon (Gao et al. 2024), and thus improving the classifier’s macro-F1 score. This diversity helps reduce the risk that retrieval will overconcentrate on a narrow conceptual neighborhood in the embedding space. Using rerankers can further increase the relevance of retrieved instances, acting as an enhancer and filter (Gao et al. 2024). In the augmentation step, the query text Xj is combined with the top-k reranked clauses, where k < P , to construct a prompt that is better suited to knowledgeintensive tasks. Finally, we query a generative model f3 : X → y to predict the tokens ŷtokens that encode the predicted labels. In comparison to few-shot learning, RAG (Gao et al. 2024) can be more token-efficient and less susceptible to noisy, unsuitable augmentation (Yao et al. 2025), and improves parameter efficiency (Ram et al. 2023). Few-shot learning uses K samples for each of the C classes in the dataset when constructing its ad-hoc support set S = ∪C c=1 (f2 (Xc,i ), yc,i ), whereas RAG retrieves a more precise context of k samples, where k < K, that does not necessarily contain examples of each possible class. Instead, the LLM is tasked with deciding among the annotations of the most similar instances without being burdened by likely unnecessary context.

4.3 Annotation Scheme Evolution The annotation scheme used in this study developed in three stages. Early work on European contracts, such as CLAUDETTE, employed a relatively small number of broad abuse categories tailored to EU consumer law (Lippi et al. 2019). Previous work adapted this approach to Chilean doctrine and expanded it into a 20-category scheme

suited to the LPC and related sources (Löffler et al. 2025; Barrientos Camus 2019). The present study further refines that scheme into 24 categories. The revision was motivated by legal and computational considerations (Löffler et al. 2025). During the annotation of Chilean ToS, some labels proved too broad, partially redundant, or systematically co-extensive with more specific categories. This created avoidable ambiguity for both annotators and models. We therefore revised the scheme to preserve doctrinal fidelity while improving operational clarity (Barrientos Camus 2019). In particular, categories that had functioned as broad umbrella labels were decomposed or absorbed into more precise labels when repeated analysis showed that a narrower formulation better captured the legally relevant feature of the clause. More generally, the revision involved a recurring trade-off between doctrinal specificity and annotation usability. Some labels could have been preserved as broad legal umbrellas closely mirroring the structure of the LPC, but in practice this often produced overlapping annotations that reduced consistency across cases and made learning more difficult. We therefore revised the scheme so that legally distinct and sufficiently recurrent patterns remained separate, while broad or low-frequency labels were absorbed into more specific or residual categories when separate treatment added complexity without improving interpretive clarity (Barrientos Camus 2019; Löffler et al. 2025). This was especially important for access-to-justice issues, where clauses affecting judicial recourse, burden of proof, and provider-controlled internal dispute mechanisms proved related but not equivalent from either a legal or annotation perspective. A representative illustration concerns clauses affecting access to justice. In the earlier scheme, a broad label captured interference with judicial recourse. In the revised scheme, this concern is distributed across more specific categories, such as inversion of the burden of proof and discretionary internal dispute resolution by the provider. Although these issues are related, they do not describe the same legal defect and are therefore more usefully annotated separately (Löffler et al. 2025; Barrientos Camus 2019). Conversely, some low-frequency issues remain under residual labels when a dedicated category would add complexity without improving reliability. The revised scheme also incorporates categories that became salient during the expanded review of contracts commonly used in Chile, including recurrent clauses linked to unjustified refusal to sell and other consumer-law issues with a clear statutory basis (Löffler et al. 2025; SERNAC 2021). The result is a taxonomy that is both more legally informative and better suited to Machine Learning.

5 Extended Corpus Our new Chilean Abusive Terms of Service Extended corpus consists of 100 online Terms of Service, doubling the previous dataset (Löffler et al. 2025) by adding 50 new contracts. These additional 50 contracts were selected from widely used services in the Chilean market across different sectors and user demographics with the full list being provided in the Appendix A. The refined annotation scheme was applied to the 10, 029 clauses. Next, clauses shorter than 7 words were eliminated, resulting in 8, 755

annotated clauses. Of these, 1, 535 (17.5%) were marked as potentially abusive and the remaining 7, 220 as okay. Following the categorization, we identify 7% of clauses as Illegal Clauses, 6.7% as Dark Clauses and 9% as Gray Clauses. Each clause can be labeled with multiple classes of each category, and also with labels from multiple categories. The processing pipeline consists of the detection of potentially abusive clauses, followed by their classification according to types.

5.1 Detection Task We define four detection tasks, one for each category and an additional joint detection task that includes any potentially abusive clause, see Table 1. The Joint Detection dataset has a total of 1, 535 positive instances from 24 different classes, and the remaining 7, 220 clauses marked as "ok". The mean sequence length is 142.6 tokens per instance, see Figure 5. Next, we generated the three datasets Illegal Detection (788 "abusive" and 7, 967 "ok"), Dark Detection (616 "abusive" and 8, 139 "ok") and Gray Detection (588 "abusive" and 8, 167 "ok"). Their instances of "abusive" clauses consist of only their specific category, with any remaining clause set to the "ok" label. This permits the development of specific detection models for each category as well as a generic detector. Table 1: Class Statistics for the detection tasks of the Chilean Abusive ToS Extended dataset. Dataset Joint Dark Gray Illegal

Label ok abusive ok abusive ok abusive ok abusive

Train 5,056 1,068 5,700 424 5,713 411 5,573 551

Val 720 164 819 65 820 64 804 80

Test 1,444 303 1,620 127 1,634 113 1,590 157

Total 7,220 1,535 8,139 616 8,167 588 7,967 788

5.2 Classification Task We provide the three classification tasks Illegal Clauses, Dark Clauses and Gray Clauses. The classification tasks are highly imbalanced and multi-label, meaning that each clause can have more than one label, posing hard challenges to Machine Learning methods. The Illegal Clauses dataset contains nine classes, but two alone ("ILG NA" and "ILG LPC PRO") make up about 60% of the total count. The rarest classes ("IGL ng" and "IGL COT") account for only around 1%, rendering their correct classification extremely difficult. The Dark Clauses dataset consists of only six classes, that are more balanced. However, the co-occurrence as reported in Fig. 4 is higher, leading to high error rate. The Gray Clauses are both highly imbalanced and multilabel, with five of the nine classes each making up only around 1.8% to 5.8% of the 12

dataset, and the remaining four classes being in the majority with between 10% and 25%. We present co-occurrence matrices for each task in Fig. 4 and the mean sequence lengths of around 212 tokens in Figure 5. Due to the imbalanced nature of the data, we provide fixed splits of 70% train, 10% validation and 20% test instances for a better reproducibility, and use an iterative stratification method (Sechidis et al. 2011) to maintain the split well-balanced despite the large number of samples with more than one class, see Tab. 2. A glossary of the annotation abbreviations used in Table 2 is provided in Appendix B. Table 2: Class statistics for the classification task of the Chilean Abusive Terms of Service Extended dataset. Dataset

Illegal

Dark

Gray

Label ILG NA ILG LPC PRO ILG RC ILG LPC ILG LPC INT ILG LPC JUS ILG acp ILG ng ILG COT Total ltd cr nod ter er ch Total des risk des uni des reser bfe des def des det des inf des lic des us Total

Train 234 118 68 56 54 25 13 7 6 581 197 157 47 29 27 23 480 132 130 127 53 32 27 21 15 10 547

Val 34 17 10 8 8 4 2 1 1 85 28 23 7 4 4 3 69 19 19 18 8 5 4 3 2 1 79

Test 67 34 19 16 15 7 3 2 2 165 57 45 13 8 8 7 138 37 35 36 15 9 8 7 5 3 155

Total 335 169 97 80 77 36 18 10 9 831 282 225 67 41 39 33 687 188 184 181 76 46 39 31 22 14 781

6 Experiments Our experiments evaluate the Detection and the Classification task, and ablate the method. First, Section 6.1 describes the experimental setup. Next, Section 6.2 evaluates the detection and Section 6.3 the classification. Then, Section 6.4 discusses the selection of the framework’s other components, such as the embedding models or reranker. Finally, Section 6.5 discusses the trade-off of model quantization to further increase efficiency for local processing. 13

ILG LPC PRO 21 169 1

ILG RC 2

ILG LPC 1

ILG LPC INT 0

ILG LPC JUS 0

ILG acp 0

ILG ng 0

ILG COT 5

300

250

200

ltd 282

225

nod

ter

250

200

150

100

ILG ILG N LP A CP RO ILG R IL C ILG G LP LP C ILG C IN LP T CJ U ILG S ac p ILG ILG ng CO T

(a) Illegal Classification

ter

d no

ltd

ILG NA 335 21

(b) Dark Classification

des uni 198 3 122 5

des risk 3 188 4

des reser 122 4 181 6

150

125

bfe 5

175

des def 0

100

des det 0

des inf 1

des lic 0

des us 0

de su de ni sr de isk sr ese r bfe de sd de ef sd e de t s in de f s li c de su s

Fig. 4: Label co-occurrence matrices for multi-label classification tasks. The diagonal elements represent the total count of each label, while off-diagonal elements show the frequency of label pairs appearing in the same sample.

6.1 Experimental Setup Our extensive experiments include a broad variety of methods and models. We finetune Small Language Models and explore prompting strategies for open-weight and proprietary Large Language Models. We evaluate fine-tuning of Small Language Models for the detection and classification tasks and base the selection of this group of models on previous work to maintain comparability (Löffler et al. 2025). These models were pre-trained on diverse data regimes, such as English, Spanish and domain-specific legal corpora. We refer to Greco and Tagarelli (Greco and Tagarelli 2024) for a thorough overview of language models for the legal domain. Our experiments evaluate BERT (Devlin et al. 2019), RoBERTa (Liu et al. 2019) and LongFormer (Beltagy et al. 2020), that were pretrained purely on English-language datasets. Next, Legal-BERT (Chalkidis et al. 2020) 14

1400

Task

Detection Classification

1200

Token Count

1000 800 600 400 200 0 Abusive

Illegal

Category

Dark

Gray

Fig. 5: Distribution of token counts (GPT-2 tokenizer) for the detection (Abusive) and classification (Illegal, Dark, Gray) tasks. was fine-tuned on data from the English-language legal domain. The pre-training of M-BERT (Devlin et al. 2019) and XLM-RoBERTa (Conneau et al. 2020) was extended to general multilingual datasets. For purely Spanish-language models, we evaluate BETO (Cañete et al. 2023) as a general model, and RoBERTalex (Gutiérrez-Fandiño et al. 2021) as a legal domain-specific model. All these models are smaller than 1 billion parameters and their context lengths are 512 tokens, apart from LongFormer’s 4096 tokens. We evaluate a diverse set of Large Language Models, that include open-weight models between 8 billion and 686 billion parameters and the most recent proprietary models. All models support large context windows and are multilingual. The openweight models are Alibaba Qwen3 14b and 235b (Yang et al. 2025), Deepseek AI Deepseek-v3.2 685b (DeepSeek-AI et al. 2025), Google Gemma3 27b (Kamath et al. 2025), Meta Llama3.1 8b and Llama3 70b (Dubey et al. 2024), and OpenAI GPT OSS 120b (OpenAI et al. 2025). We include smaller models of 8b, 14b or 27b parameters, that can be executed with relatively moderate compute resources, as well as larger models of 70b, 120b, 235b and even 686b parameters, that are more suitable for cloudcompute environments. The proprietary models are OpenAI’s GPT 5.2, GPT 5 mini and GPT 5 nano (Singh et al. 2025), that represent different capabilities and cost structures, mirroring the open-weight model range. The few-shot prompts are stable per shot-count to retain comparability between models. We create static 1-, 3- and 5-shot prompts. Rather than picking examples at random, we use spaced sampling based on text length, that helps avoid biased selection. Following Löffler et al. (Löffler et al. 2025), the construction algorithm maintains length diversity by sorting candidates by length and picking representative samples for each class, i.e., we avoid picking only the shortest or longest clauses, which helps the model understand the label across different levels of complexity. The prompt also instructs that the task is either binary (detection) or multiclass, multilabel 15

(classification). Finally, the examples are shuffled to reduce LLMs’ recency bias. The prompt is in a pattern completion format: 1. Instruction: What to do and how to behave. 2. Few-Shot Examples: A series of Cláusula: [Text] followed by Etiqueta: [Label]. 3. The Hook: It ends with a final Cláusula: “{{ }} and a trailing Etiqueta:. This dangling label acts as a trigger for the model to complete the text with the predicted class immediately. The RAG-based prompt is a product of the method outlined in Section 4, and contains up to 5 samples selected from 15 retrieved and reranked samples, hyper parameters that were determined in preliminary experiments. Our method requires embedding models or retrievers, and a reranker. For hybrid RAG, we evaluate the lexical retriever BM25 (Robertson and Zaragoza 2009) and the lexical-neural hybrid retriever BM42 1 . As neural embedding models, we evaluate OpenAI’s textembedding-3-large 2 and multilingual-e5-large (Wang et al. 2024b). For reranking, we evaluate the multilingual versions of the neural models Jina Reranker V2 3 , Jina Reranker V3 (Wang et al. 2025) and the efficient cross-encoder (Reimers and Gurevych 2019) ms-marco-MiniLM-L-6-v2 4 , trained on MS MARCO passage ranking (Bajaj et al. 2018). See Section 6.4 for the selection of the best combination of these components. LLMs larger than 70b parameters were commercially hosted. We used the OpenRouter API for querying open-weight models and the OpenAI API for the GPT 5 variants. LLMs up to 70b parameters were quantized to 8-bit to increase throughput without sacrificing performance (Li et al. 2024) and executed using Ollama version 0.14.3. Fine-tuning Small Language Models relied on Mixed-Precision training on NVidia ADA Lovelace GPUs, using PyTorch 2.9.1. Experimental runs and trainings were repeated five times with different random seeds. We use the Adam optimizer (Kingma and Ba∗ 2015) with a learning rate of 3 · 105 for 50 epochs, and set the batch size to 32. For completeness, we also evaluate a Support Vector Machine (Cortes and Vapnik 1995) with TF-IDF features for all task, and train the model with 10-fold cross validation. As a more modern baseline, we also evaluate a RAG-based MajorityVote classifier, that selects the predicted labels among the retrieved samples using a majority vote mechanism. We evaluate a dense and a hybrid RAG variant. Selecting robust performance metrics is essential for the evaluation the approaches for identifying potentially abusive clauses within Terms of Service, given the significant class imbalance inherent in this legal domain, see Section 5. Standard accuracy can be a deceptive indicator of success because it often yields inflated performance 1

Qdrant (2024), https://qdrant.tech/articles/bm42, accessed on Feb. 20th 2026 OpenAI (2024), https://openai.com/index/new-embedding-models-and-api-updates/, accessed on Feb. 20th 2026 3 Jina AI (2024), \protect\tolerance9999\emergencystretch3em\hfuzz.5\p@\vfuzz\hfuzzhttps://jina.ai/ news/jina-reranker-v2-for-agentic-rag-ultra-fast-multilingual-function-calling-and-code-search/, accessed on Feb. 20th 2026 4 Reimers and Gurevych (2021), https://huggingface.co/cross-encoder/ms-marco-MiniLM-L6-v2, accessed on Feb. 20th 2026 2

estimates by masking poor predictive power regarding minority classes. To provide a more rigorous assessment, this study uses the Micro-F1 (µ-F1) and Macro-F1 (M-F1 scores. The following metrics form the basis of our evaluation. Precision p measures the accuracy of positive predictions and recall r measures the ability of the model to find all relevant instances within a class. The F1 score serves as the harmonic mean of precision and recall, providing a single summary statistic as TP + FP TP TP + FN r= TP p·r F1 = 2 · p+r p=

(1) (2) (3)

In this work, we apply two aggregation methods. First, Micro-averaging aggregates the contributions of all classes to compute the average metric globally. It sums the individual true positives, false positives, and false negatives across all classes before calculating the final ratios p and r, as defined in Eq. 1 and Eq. 2. With imbalanced data, rarer classes may get overshadowed by more frequent classes. In contrast, Macroaveraging treats all classes equally by calculating the metric P independently for each N class i and then taking the arithmetic mean as F 1macro = N1 i=1 F 1i . This ensures that performance on rare abusive clauses is better recognized in the aggregated score.

6.2 Detection The "Detection" module in our proposed framework processes large quantities of text, flagging suspicious clauses that may be potentially abusive, acting as a filter before the "Classification" module. It should be computationally inexpensive with high recall and precision. This section tests various models for their ability to detect any type of potentially abusive clause, validating the filter step. We first perform the analysis on the "Abusive Clauses" dataset, that unifies all datasets, and later break down the performance into separate tests on the three individual tasks of Illegal, Dark and Gray Clauses. The Abusive Clauses dataset contains all clause annotated as any potentially abusive class as "abusive" class, and marks all other clauses with the class "okay". The three tasks Illegal, Dark and Gray Clauses retain each of their potentially abusive clauses as "abusive", but mark all other ones as "okay", even those that belong to another category of abusive clauses, see Section 5.1 for details. The evaluation focuses on the less expensive methods that are more suitable for processing large quantities of data locally, i.e., the fine-tuning of small language models and the baselines SVM and RAG-based Majority-Vote. We also report the results of the LLM Qwen3 14b with hybrid RAG for completeness. We present the detection results on the Abusive Clauses dataset in Table 3 and on the other three datasets in Table 4. The first insight is that the integrated detection of any potentially abusive clause using only small models is possible at high rates. Multiple fine-tuned models, such as the multilingual variants of BERT and RoBERTa, or the Spanish and legal pre-trained RoBERTalex, obtain the highest scores (M-F1 17

Abusive M-F1

µ-F1

SVM (TF-IDF) Majority-Vote (Hybrid) Majority-Vote (Dense)

0.75±0.01 0.77±0.00 0.78±0.00

0.88±0.00 0.87±0.00 0.88±0.00

Qwen3 14b (Hybrid)

0.76±0.00

0.86±0.00

BERT cased BERT uncased (legal) BERT cased (multiling.) BERT uncased (multiling.) BERT uncased BETO cased (Spanish) BETO uncased (Spanish) Longformer RoBERTalex (Spanish, legal) RoBERTa RoBERTa (multiling.) RoBERTa (multiling., large)

0.78±0.01 0.61±0.23 0.76±0.00 0.79±0.01 0.77±0.00 0.78±0.01 0.78±0.01 0.77±0.01 0.79±0.00 0.78±0.01 0.79±0.01 0.66±0.18

0.88±0.01 0.85±0.03 0.87±0.00 0.88±0.01 0.87±0.01 0.88±0.00 0.87±0.01 0.86±0.00 0.88±0.01 0.87±0.00 0.88±0.00 0.85±0.03

Method

Table 3: Aggregation of detection results for the Abusive domain.

of 0.79, µ-F1 of 0.88), with BETO, BERT, and the three baseline methods only 1-2 M-F1 points behind. Interestingly, the LLM with RAG obtains lower scores than a simple Majority-Vote (M-F1 of 0.76 vs 0.78, µ-F1 of 0.86 vs 0.88). Second, on the individual datasets, the fine-tuned BETO cased/uncased models (Spanish-language variants of BERT) dominate, scoring the highest on Illegal and Dark Clauses, and the second highest on Gray Clauses, only behind BERT. Surprisingly, the three baseline methods perform poorly. The LLM still does not obtain competitive results, and is between 2-8 M-F1 points and 3-4 µ-F1 points behind. Third, the breakdown of the Abusive Clauses into Illegal, Dark and Gray Clauses indicates a ranking in the difficulty of the tasks, see Table 4. Scores range from between M-F1 of 0.83 for Illegal Clauses over 0.77 for Dark Clauses to 0.72 for Gray Clauses. This is in line with the intuitive ranking of our legal experts, who consider Gray Clauses to be the most open to interpretation. To conclude, these experiments demonstrate that smaller, less costly language models, such as BERT variants, or even RAG-based Majority-Vote methods, can reliably detect potentially abusive clauses in Chilean Terms of Service. This result further validates our aim of processing data locally, as well as our design decisions, separating detection from classification.

6.3 Classification The "Classification" module in our proposed framework only processes the filtered clauses, that are more likely to be abusive. There, we can employ larger and more costly, but also more powerful language models. This section tests a large variety of models for their ability to classify the highly imbalanced, multi-label datasets Illegal, Dark and Gray clauses. First, Section 6.3.1 presents aggregated results in the form of a high score, focusing on the overall M-F1 and its stability. Then, Section 6.3.2 presents 18

Illegal

Dark

Gray

M-F1

µ-F1

M-F1

µ-F1

M-F1

µ-F1

SVM (TF-IDF) Majority-Vote (Hybrid) Majority-Vote (Dense)

0.78±0.01 0.78±0.00 0.79±0.00

0.94±0.00 0.94±0.00 0.94±0.00

0.69±0.02 0.70±0.00 0.69±0.00

0.94±0.01 0.93±0.00 0.93±0.00

0.66±0.01 0.69±0.00 0.68±0.00

0.94±0.00 0.94±0.00 0.94±0.00

Qwen3 14b (Hybrid)

0.75±0.00

0.91±0.00

0.72±0.00

0.91±0.00

0.70±0.00

0.91±0.00

BERT cased BERT uncased BERT uncased (legal) BERT cased (multiling.) BERT uncased (multiling.) BETO cased (Spanish) BETO uncased (Spanish) Longformer RoBERTalex (Spanish, legal) RoBERTa RoBERTa (multiling.) RoBERTa (multiling., large)

0.79±0.01 0.79±0.01 0.77±0.03 0.81±0.01 0.81±0.01 0.83±0.01 0.81±0.00 0.79±0.02 0.80±0.01 0.80±0.01 0.80±0.01 0.66±0.17

0.94±0.00 0.94±0.00 0.92±0.02 0.94±0.00 0.94±0.00 0.95±0.00 0.94±0.00 0.94±0.01 0.94±0.00 0.94±0.01 0.94±0.01 0.93±0.02

0.73±0.02 0.72±0.05 0.75±0.02 0.73±0.04 0.75±0.01 0.76±0.01 0.77±0.01 0.59±0.12 0.75±0.01 0.72±0.01 0.75±0.00 0.48±0.00

0.94±0.00 0.94±0.00 0.94±0.00 0.93±0.00 0.94±0.00 0.94±0.00 0.94±0.00 0.93±0.00 0.94±0.00 0.93±0.00 0.94±0.01 0.93±0.00

0.72±0.02 0.70±0.01 0.67±0.02 0.68±0.00 0.69±0.02 0.71±0.01 0.69±0.00 0.61±0.11 0.70±0.01 0.61±0.11 0.69±0.02 0.48±0.00

0.94±0.01 0.93±0.00 0.92±0.01 0.92±0.01 0.93±0.01 0.93±0.01 0.93±0.00 0.93±0.01 0.93±0.00 0.92±0.01 0.93±0.01 0.94±0.00

Method

Table 4: Detection scores as macro-F1 (M-F1) and micro-F1 (µ-F1). the detailed breakdown, demonstrating the significant improvement of RAG-based prompting over the prior state-of-the-art, such as fine-tuning of few-shot learning.

6.3.1 Aggregate Results This section demonstrates the overall performance on the classification tasks in a compact, aggregated form. This analysis aims to test which method generalizes best among tasks, specifically comparing open-weight and proprietary methods to determine applicability of device-local instead of cloud-based document analysis. We aggregate the evaluation of each method and model over all classification tasks, i.e., Illegal, Dark and Gray Clauses, testing performance with multi-label, highly imbalanced tasks, into a concise ranked high score. Figure 6 shows the p arithmetic mean of the Macro-F1 scores and the combined P 2 σ from all three classification tasks and ranks the classistandard deviation as n1 fiers. We chose the Macro-F1 score to more clearly represent prediction quality on the imbalanced data. Please refer to Section 6.3.2 for a detailed breakdown of each task that also includes Micro-F1 scores. We present the results grouped into RAG-based prompting of commercial or open-weight models, few-shot, fine-tuning and baselines. The most important insight from our results is the closure of the open-weight gap between commercial models like GPT 5.2 (Dense) with 0.613 ± 0.008 Macro-F1 and Qwen3 14b (Dense) with 0.603 ± 0.007 due to the use of RAG-based prompting. Both models perform statistically identical, i.e., the score ranges overlap: 0.596 − 0.610 vs 0.605 − 0.621. We conclude that we can use open-weight Qwen3 models without loosing statistically significant performance. Second, while the ranking is dominated by RAG prompting, fine-tuned smaller language models mostly beating few-shot prompting of LLMs, with the strongest finetuned model RoBERTa (multiling., large) beating GPT 5.2 (3-shot) by a large margin. The RAG-based Majority-Vote classifier variants outperform the SVM with TF-IDF features. 19

GPT 5.2 (Dense) GPT 5 nano (Dense) GPT 5.2 (Hybrid) Qwen3 14b (Dense) Gemma3 27b (Dense) Qwen3 14b (Hybrid) RoBERTa (multiling., large) GPT 5 mini (Hybrid) GPT 5 nano (Hybrid) Gemma3 27b (Hybrid) GPT OSS 120b (Hybrid) Deepseek-v3.2 (Hybrid) GPT 5 mini (Dense) Qwen3 235b (Dense) GPT OSS 120b (Dense) GPT 5.2 (3-shot) Deepseek-v3.2 (Dense) RoBERTa (multiling.) BETO cased (Spanish) Qwen3 235b (Hybrid) Longformer GPT 5.2 (5-shot) Meta Llama3.1 8b (Dense) Meta Llama3 70b (Dense) BERT cased (multiling.) BETO uncased (Spanish) BERT uncased (multiling.) RoBERTa Meta Llama3.1 8b (Hybrid) Majority-Vote (Dense) Meta Llama3 70b (Hybrid) Gemma3 27b (5-shot) Majority-Vote (Hybrid) Qwen3 14b (5-shot) BERT cased BERT (legal) BERT uncased Gemma3 27b (3-shot) Qwen3 14b (3-shot) RoBERTalex (Spanish, legal) SVM (TF-IDF) GPT 5.2 (1-shot) Gemma3 27b (1-shot) Meta Llama3.1 8b (5-shot) Meta Llama3.1 8b (3-shot) Qwen3 14b (1-shot) Meta Llama3.1 8b (1-shot) 0.20

Model Type

Commercial Open-Weight Fine-Tuned Few-Shot Baseline

0.25

0.30

0.35

0.40

0.45

0.50

Average Macro-F1 (Avg of Illegal, Dark, Gray)

0.55

0.60

0.65

Fig. 6: Method ranking by overall Macro-F1 on classification tasks (with stability error bars) Third, stability is an issue when classifying clauses. The results show that GPT 5.2 (nano/mini), Qwen3 and Gemma3 perform more stable with Hybrid RAG than with Dense RAG. This may be due to the merging of two distinct indices in the retrieval step, that may select more diverse options for the prompt, leading to fewer mode collapses in the prompt. Furthermore, GPT 5 Nano should be avoided in practice due

to its high standard deviation of ±0.026 overall and ±0.07 in specific tasks. Its instable performance makes it dangerous for automatic analysis as it is too unpredictable compared to the solid stability of Qwen3 or GPT 5.2. Lastly, the multilingual RoBERTa large model can be used as a safe fallback for computationally weaker devices. It requires much less resources due to its smaller size, while still obtaining competitive Macro-F1 scores compared to larger LLMs. To conclude, our analysis shows that a purely local processing chain is feasible for classifying potentially abusive Chilean Terms of Service. With more precise RAGbased prompting, local LLMs like Qwen3 14b approach the performance of cloud-based solutions like GPT 5.2, and robust fallback options (fine-tuned RoBERTa multiling. large) with slightly lower performance are available for less capable clients.

6.3.2 Detailed Breakdown The detailed breakdown demonstrates the significant improvement of (hybrid or dense) RAG over standard few-shot prompting and fine-tuning small language models, that represent the previous state-of-the-art for the classification of potentially abusive clauses in Chilean Terms of Service (Löffler et al. 2025). We compare the individual Macro- and Micro-F1 scores for RAG- and few-shot prompted LLMs as well as fine-tuned models for our three novel classification tasks. Table 5 shows the models’ classification Macro- and Micro-F1 scores with standard deviation for the three datasets Illegal, Dark and Gray Clauses. The table groups the baseline methods SVM and Majority-Vote variants, the RAG-based prompting of both open-weight and commercial LLMs (GPT 5 nano (Dense) uses the dense RAG variant) the few-shot prompting of LLMs with one, three or five examples per class (Qwen3 14b (3) uses three examples per class), and the fine-tuning of smaller language models. The best result per dataset and metric is highlighted in bold and the best result per group is underlined, e.g., Qwen3 14b (Hybrid) scores the hightest Macro-F1 score for classifying Gray Clauses with 0.77 ± 0.01. In the following we analyze the improvement of RAG over standard few-shot prompting, comparing the RAG variants’ performance against the best few-shot result. First, using RAG yields a massive performance jump, far greater than simply adding more few-shot examples, with an average Macro-F1 improvement of +10% and an average Micro-F1 improvement of +17.75%. Second, the smaller the model, the more it benefits from RAG. Qwen3 14b sees a staggering +31% absolute gain in Micro-F1 compared to its best few-shot variant (from 41% to 72%) and an average improvement of 16.33% overall. The technique effectively transformed it from a weak learner into a state-of-the-art model. The even smaller model Llama3.1 8b showed the highest average improvement at 18.33%, with RAG providing consistent gains across all categories, particularly in the "Dark" and "Illegal" Micro-F1 scores (+24% and +22% respectively). However, its absolute performance numbers remain below the larger models. Our largest model tested is GPT 5.2. It sees a smaller but still significant average improvement of 8.33%, driven by the Illegal Clauses dataset (+11% to +21%), with similar results for 5-shot prompting and RAG on the Dark Clauses. This suggests that, even though larger models already have strong internal representation of the problem compared to smaller models, all 21

Illegal

Dark

Gray

M-F1

µ-F1

M-F1

µ-F1

M-F1

µ-F1

SVM (TF-IDF) Majority-Vote (Hybrid) Majority-Vote (Dense)

0.36±0.02 0.48±0.00 0.46±0.00

0.64±0.01 0.70±0.00 0.71±0.00

0.43±0.02 0.44±0.00 0.51±0.00

0.77±0.01 0.73±0.00 0.76±0.00

0.59±0.02 0.58±0.00 0.57±0.00

0.70±0.02 0.65±0.00 0.67±0.00

GPT 5.2 (Hybrid) GPT 5.2 (Dense) GPT 5 mini (Hybrid) GPT 5 mini (Dense) GPT 5 nano (Hybrid) GPT 5 nano (Dense) GPT OSS 120b (Hybrid) GPT OSS 120b (Dense) Gemma3 27b (Hybrid) Gemma3 27b (Dense) Qwen3 235b (Hybrid) Qwen3 235b (Dense) Qwen3 14b (Hybrid) Qwen3 14b (Dense) Meta Llama3 70b (Hybrid) Meta Llama3 70b (Dense) Meta Llama3.1 8b (Hybrid) Meta Llama3.1 8b (Dense) Deepseek-v3.2 (Hybrid) Deepseek-v3.2 (Dense)

0.55±0.02 0.55±0.02 0.53±0.02 0.48±0.03 0.53±0.05 0.56±0.07 0.52±0.05 0.46±0.02 0.46±0.01 0.50±0.02 0.44±0.01 0.42±0.02 0.55±0.01 0.49±0.01 0.33±0.08 0.44±0.02 0.40±0.03 0.45±0.05 0.46±0.04 0.45±0.02

0.72±0.01 0.71±0.02 0.71±0.00 0.69±0.01 0.67±0.02 0.68±0.03 0.66±0.04 0.68±0.02 0.69±0.00 0.73±0.01 0.68±0.01 0.66±0.00 0.71±0.01 0.72±0.01 0.52±0.09 0.64±0.01 0.56±0.02 0.56±0.03 0.60±0.01 0.63±0.03

0.60±0.01 0.78±0.00 0.61±0.01 0.79±0.00 0.57±0.01 0.76±0.01 0.57±0.02 0.76±0.01 0.58±0.01 0.77±0.01 0.59±0.03 0.79±0.02 0.55±0.03 0.77±0.03 0.55±0.02 0.76±0.02 0.58±0.01 0.76±0.00 0.62±0.01 0.78±0.01 0.52±0.00 0.75±0.00 0.61±0.04 0.79±0.03 0.54±0.01 0.76±0.01 0.62±0.00 0.79±0.00 0.61±0.02 0.75±0.00 0.50±0.14 0.60±0.12 0.51±0.03 0.75±0.02 0.53±0.03 0.74±0.01 0.59±0.04 0.77±0.03 0.56±0.06 0.74±0.03

0.66±0.01 0.69±0.01 0.68±0.01 0.73±0.01 0.65±0.01 0.72±0.01 0.65±0.01 0.73±0.01 0.64±0.03 0.70±0.01 0.67±0.02 0.73±0.02 0.65±0.02 0.70±0.01 0.65±0.01 0.71±0.00 0.68±0.02 0.74±0.01 0.68±0.03 0.74±0.01 0.65±0.02 0.72±0.02 0.65±0.02 0.73±0.00 0.71±0.00 0.75±0.00 0.70±0.02 0.77±0.01 0.59±0.10 0.67±0.09 0.66±0.02 0.73±0.01 0.63±0.03 0.71±0.02 0.62±0.00 0.69±0.01 0.65±0.03 0.71±0.01 0.63±0.01 0.72±0.01

GPT 5.2 (5) GPT 5.2 (3) GPT 5.2 (1) Gemma3 27b (5) Gemma3 27b (3) Gemma3 27b (1) Qwen3 14b (5) Qwen3 14b (3) Qwen3 14b (1) Meta Llama3.1 8b (5) Meta Llama3.1 8b (3) Meta Llama3.1 8b (1)

0.35±0.01 0.44±0.02 0.34±0.01 0.41±0.02 0.43±0.00 0.38±0.04 0.37±0.01 0.41±0.01 0.21±0.01 0.31±0.06 0.27±0.03 0.24±0.05

0.44±0.02 0.51±0.01 0.41±0.01 0.46±0.01 0.50±0.01 0.44±0.03 0.38±0.01 0.41±0.01 0.21±0.01 0.34±0.03 0.31±0.02 0.25±0.03

0.62±0.01 0.58±0.01 0.42±0.02 0.53±0.01 0.49±0.01 0.41±0.01 0.55±0.01 0.50±0.01 0.37±0.01 0.41±0.02 0.38±0.04 0.23±0.04

0.70±0.01 0.64±0.01 0.55±0.02 0.63±0.01 0.55±0.02 0.55±0.01 0.62±0.01 0.59±0.01 0.50±0.01 0.51±0.05 0.45±0.05 0.31±0.03

0.63±0.01 0.62±0.01 0.54±0.01 0.57±0.01 0.56±0.00 0.49±0.02 0.58±0.02 0.56±0.02 0.50±0.02 0.45±0.02 0.45±0.02 0.32±0.05

0.68±0.00 0.61±0.00 0.57±0.01 0.64±0.01 0.61±0.00 0.55±0.01 0.61±0.01 0.58±0.01 0.53±0.01 0.51±0.03 0.45±0.03 0.38±0.03

BERT uncased (multiling.) BERT cased (multiling.) BETO uncased (Spanish) BETO cased (Spanish) BERT uncased BERT cased BERT (legal) RoBERTa (multiling., large) RoBERTa (multiling.) RoBERTalex (Spanish, legal) RoBERTa Longformer

0.43±0.02 0.43±0.05 0.45±0.02 0.47±0.00 0.43±0.04 0.45±0.03 0.44±0.04 0.52±0.01 0.48±0.02 0.38±0.01 0.40±0.02 0.45±0.03

0.71±0.02 0.71±0.01 0.72±0.00 0.73±0.01 0.69±0.01 0.71±0.02 0.70±0.01 0.74±0.02 0.74±0.02 0.68±0.02 0.67±0.02 0.69±0.02

0.55±0.01 0.55±0.02 0.51±0.04 0.58±0.02 0.49±0.00 0.49±0.04 0.48±0.02 0.60±0.02 0.58±0.01 0.51±0.00 0.52±0.02 0.54±0.02

0.79±0.01 0.79±0.02 0.80±0.01 0.80±0.00 0.76±0.01 0.78±0.02 0.76±0.00 0.81±0.03 0.80±0.01 0.79±0.01 0.77±0.01 0.76±0.03

0.58±0.04 0.61±0.01 0.61±0.02 0.57±0.01 0.57±0.05 0.56±0.04 0.58±0.02 0.64±0.01 0.58±0.03 0.52±0.01 0.62±0.01 0.61±0.02

0.70±0.01 0.72±0.01 0.74±0.02 0.73±0.01 0.70±0.02 0.72±0.01 0.70±0.02 0.75±0.01 0.72±0.01 0.71±0.01 0.73±0.01 0.72±0.03

Method

Table 5: Classification scores as macro-F1 (M-F1) and micro-F1 (µ-F1). models benefit from not only domain-specific context in the form of few-shot learning, but the much more sample-specific legal patterns that RAG retrieves. It is logical that RAG-based prompting contains more specific information than few-shot prompting, as it retrieves similar instances compared to the query clause. The retrieved samples are more appropriate for the imbalanced nature of the data

distribution of our datasets. The distributions feature a "long tail" of rare class samples, that are difficult to classify well with few-shot prompting. Due to the imbalanced data, the evaluation focuses more on the Macro-F1 scores, that penalize low performance for rarer classes. For example, Qwen3 14b scores very low on Illegal Clauses with Macro-F1 of 0.37. With RAG-based prompting, the score jumps to 55%. RAG retrieves the most similar clauses from the database, possibly simplifying the LLM’s task. However, the comparison to the Majority-Vote baseline indicates, that the LLMs do not simply copy the reasoning, but infer class labels better than a simple vote mechanism when in doubt. Besides the lower score results of few-shot learners in absolute terms, we also observe a high volatility between different shot-counts. For example, the 3-shot performance of GPT 5.2 of 0.547 is actually higher than the 5-shot performance of only 0.533. This phenomenon is a known issue that may stem from sampling biases of fewshot learning for class imbalanced datasets (Ochal et al. 2021). With five samples per class, the probability to select bad samples that confuse the model is higher than with three samples. In contrast, RAG consistently improves performance across all models without a penalty due to confusion as seen in few-shot learning, because its retrieved samples are more specific to the query. Furthermore, we see a strong grounding effect of RAG, especially in small models like Llama3.1 8b. Here, the Micro-F1 score jumps from an average of 45% to a respectable 67% over all datasets. This indicates that smaller models hallucinate too much to be useful with few-shot learning, while RAG contributes to preventing wrong classification, especially of rare and difficult queries. Finally, the fine-tuned model RoBERTa in its multilingual and large variant performs competitively to RAG-based prompting on some tasks and metrics, especially in Micro-F1, but falls behind overall and especially in the Macro-F1 for rarer and more difficult instances. While models smaller than RoBERTa large were previously still competitive with few-shot learners (Löffler et al. 2025), not even domain specific (LegalBERT) or language specific pre-training (BETO) small language models manage to to obtain competitive Macro-F1 scores for the imbalanced legal datasets compared to RAG. To conclude, the RAG technique is mandatory for compute constrained environments that can only use LLMs with 8b, 14b or 27b parameters. Then, Qwen3 14b with RAG even outperforms GPT 5.2 without RAG (but few-shot prompting). Our results show that the prompted context is more relevant than the model’s absolute parameter count for specific legal classification tasks.

6.3.3 Error Analysis and Reasoning Mode This section analyzes the remaining errors of Qwen3 14b (Hybrid) and evaluates whether enabling explicit reasoning improves classification quality enough to justify its additional cost. We distinguish between "retrieval errors", where the gold label is absent from the retrieved examples, and "generation errors", where the gold label is present in the retrieved set but the model still fails to predict it. We also measure the Pearson correlation (r) between class support and per-label F1 and compare the default setting with the model’s reasoning mode. 23

Table 6: Error analysis of Qwen3 14b (Hybrid) on the three classification tasks. Retrieval (Ret.) errors denote false negatives (FN) for which the gold label was absent from the retrieved examples, and generation (Gen.) errors denote false negatives for which relevant evidence was retrieved but the model still failed to predict the gold label. Pearson correlation measures the association between class support and perlabel F1. Task

FN Ret. Error Gen. Error Gen./Ret. Support–F1 Pearson r

Illegal Dark Gray

46 33 38

10 12 11

36 21 27

3.60 1.75 2.45

0.498 0.728 0.188

Table 7: Frequent wrong-label substitutions and effect of explicit reasoning mode for Qwen3 14b (Hybrid). Panel (a) reports the most frequent label confusions across classification errors. Panel (b) compares the default setting with explicit reasoning mode. Runtime refers to average end-to-end duration per run. (a) Most frequent label confusions Task

Gold Label

Pred. Label

Illegal Illegal Dark Dark Gray Gray

ILG NA ILG LPC PRO er ch bfe des risk

ILG LPC PRO ILG NA ltd cr des reser des reser

Task

Reasoning

Macro-F1

Illegal Illegal Dark Dark Gray Gray

No Yes No Yes No Yes

0.55 0.55 0.54 0.57 0.71 0.58

Count 7 6 6 3 4 3

(b) Effect of explicit reasoning FN / Gen. Error 46 / 36 47 / 37 33 / 21 31 / 20 38 / 27 51 / 37

Runtime: no reasoning ≈ 1–3 min/run; with reasoning ≈ 20–26 min/run.

Table 6 shows that generation errors outnumber retrieval errors across all three tasks. This indicates that many false negatives occur even when relevant examples are retrieved, suggesting that retrieval alone does not fully resolve the classification problem, even though RAG combined with LLM-based inference still clearly outperforms retrieval-only baselines such as majority vote in the overall evaluation. The same table also shows substantial differences in sensitivity to class imbalance: this dependence is strongest for Dark (r = 0.728), moderate for Illegal (r = 0.498), and weak for Gray (r = 0.188). Table 7(a) shows that the dominant mistakes are not random, but cluster around semantically adjacent labels, such as ILG NA versus ILG LPC PRO, er versus ltd,

and bfe versus des reser. A qualitative review of the generation errors reveals three primary failure modes. First, when the retriever fails entirely to fetch relevant clauses, it often causes blind contextual reliance. For example, in a Disney subscription clause involving unilateral price modifications (ch) and consumers charged for provider errors (er ), the retriever failed to find relevant examples for either. The generator, instead of classifying the clause zero-shot, blindly adopted the retrieved distractor labels cr and ter. This suggests that the retriever’s failure not only causes a retrieval error but also biases the generation step by moving the model into irrelevant semantic spaces (Wu et al. 2024). Second, we observe contextual flooding (Shi et al. 2023), where the model is misled by a high volume of distractor labels in the prompt. In Gray clauses, which exhibit the highest generation-to-retrieval error ratio of 2.45:1, the model often ignores the correct minority gold label in the prompt to follow the majority distractor. This is particularly evident in des risk and des uni, which together account for over 50% of the observed generation errors in the Gray task sample. Third, we identify that the model’s reasoning breaks down in partial retrieval in multi-label scenarios. We found cases where the retriever successfully fetched context for one true label (e.g., des risk ) but missed a second true label (e.g., bfe). In these instances, the missing context for the second label appeared to degrade the model’s overall confidence, leading it to faulty reasoning (Zarrinkia et al. 2026) on both labels, even the one for which evidence was present. These confusion patterns are consistent with the error decomposition in Table 6, which suggests that many failures persist even when relevant evidence is available. In rare instances, we also observe that the model successfully predicts the correct label zero-shot despite a complete retrieval failure. However, Table 7(b) shows that explicit reasoning does not reliably correct the errors. While reasoning mode slightly improves Dark, it leaves Illegal essentially unchanged and substantially worsens Gray, while increasing runtime from roughly 1–3 minutes to 20–26 minutes per run. This decline in Gray performance with reasoning suggests that long-form chain-of-thought may actually amplify the semantic ambiguity of the clauses rather than clarifying them. Overall, the error analysis indicates that Hybrid RAG substantially improves contextual grounding, but the remaining failures are concentrated in fine-grained legal distinctions and minority classes. Explicit reasoning mode is therefore not a cost-effective remedy for local deployment in this setting.

6.4 Hyperparameter Optimization We optimized the components of our proposed RAG framework to perform best for the classification of potentially abusive clauses in Terms of Service. Specifically, we selected the best-performing sparse and dense embedding models, and the reranker to generalize across the different datasets via statistical analysis (Demsar 2006). To identify the most robust configuration with highest scores and lowest variance, we perform a random-effects meta-analysis (Borenstein et al. 2010). We assume the observed performance µce for a configuration c on task e is composed of the true

average performance µc , a task-specific deviation δe , and measurement noise ϵce : (4)

µce = µc + δe + ϵce

We explore the configuration space c ∈ C populated with approaches comprising lexical retrieval, semantic embedding, and neural reranking to address the inherent limitations of single-model approaches, i.e., the lexical retriever BM25 (Robertson and Zaragoza 2009) and the lexical-neural hybrid retriever BM42, the semantic embedding models text-embedding-3-large or multilingual-e5-large (Wang et al. 2024b), and the multilingual versions of the neural reranking models Jina Reranker V2, Jina Reranker V3 (Wang et al. 2025) or the efficient cross-encoder (Reimers and Gurevych 2019) msmarco-MiniLM-L-6-v2, trained on MS MARCO passage ranking (Bajaj et al. 2018). For the tasks e, we evaluate three tasks from Chilean Abusive ToS (Löffler et al. 2025) and one from UnfairTOS (Lippi et al. 2019). We utilize the DerSimonian–Laird estimator (DerSimonian and Laird 1986) to quantify the between-task variance (τ 2 , calculated over four different tasks) and the within-task variance (σe2 , calculated over four experimental repetitions). Configurations are then ranked by a weighted pooled mean, where the weight we for each task is inversely proportional to both sources of variance: we =

1 σe2 + τ 2

(5)

This weighting scheme penalizes models that are unstable across different tasks (high τ 2 ). A value of τ 2 ≈ 0 indicates consistent behavior, whereas a large τ 2 implies high task sensitivity. The best configuration is selected based on a composite score (the sum of the random-effects Macro-F1 and Micro-F1) to balance rare-class detection with global accuracy. The performance metrics and variance across the top configurations are summarized in Table 8. Table 8: Performance of evaluated RAG configurations sorted by Random-Effects Macro-F1. The selected optimal configuration with the highest composite score is highlighted in bold. Sparse

Dense

Reranker

RE Macro-F1

Macro 95% CI

RE Micro-F1

Micro 95% CI

2 τmacro

2 τmicro

BM25 BM42 BM42 BM25 BM42 BM42 BM25 BM25 BM42 BM25 BM42 BM25

Text-3-Large Text-3-Large Text-3-Large Multi-E5 Multi-E5 Multi-E5 Text-3-Large Text-3-Large Text-3-Large Multi-E5 Multi-E5 Multi-E5

Jina V2 mMARCO Jina V2 Jina V2 Jina V2 Jina V3 mMARCO Jina V3 Jina V3 mMARCO mMARCO Jina V3

0.7316 0.7315 0.7309 0.7308 0.7302 0.7230 0.7212 0.7212 0.7187 0.7171 0.7170 0.7155

[0.5162, 0.9471] [0.5505, 0.9126] [0.5200, 0.9417] [0.5458, 0.9159] [0.5426, 0.9177] [0.5284, 0.9175] [0.5332, 0.9093] [0.5204, 0.9221] [0.5175, 0.9199] [0.5160, 0.9182] [0.5072, 0.9268] [0.5061, 0.9249]

0.7576 0.7620 0.7577 0.7637 0.7637 0.7422 0.7621 0.7413 0.7424 0.7400 0.7397 0.7349

[0.5801, 0.9350] [0.6079, 0.9160] [0.6116, 0.9038] [0.6143, 0.9131] [0.6318, 0.8956] [0.5649, 0.9194] [0.6164, 0.9078] [0.5765, 0.9062] [0.5819, 0.9028] [0.5674, 0.9126] [0.5704, 0.9090] [0.5604, 0.9094]

0.0483 0.0339 0.0463 0.0355 0.0365 0.0381 0.0367 0.0404 0.0398 0.0421 0.0458 0.0449

0.0328 0.0247 0.0222 0.0232 0.0181 0.0321 0.0221 0.0277 0.0261 0.0310 0.0298 0.0311

As the analysis in Table 8 demonstrates, the combination of BM25 (sparse), Jina V2 (reranker), and Multilingual-E5 (dense) emerges as the optimal configuration with a composite score of 1.4945 (RE-Macro F1: 0.7308, RE-Micro F1: 0.7636). Notably, while the Text-3-Large variant achieved a nominally higher raw Macro-F1, it exhib2 ited significant instability across tasks (τmacro = 0.0483). The random-effects model successfully penalized this volatility in favor of the Multilingual-E5 variant, which 2 proved considerably more robust (τmacro = 0.0355). Further analysis of the discrepancy between Macro-F1 and Micro-F1 reveals crucial trade-offs in the configuration space, illustrating why a composite score is necessary for this domain. For instance, while variants utilizing the Text-3-Large dense embedding model (such as the {BM25, Text-3-Large, Jina V2} configuration) achieved the highest raw Macro-F1 scores (0.7316), they suffered from lower Micro-F1 performance (0.7576) 2 and notably higher task-level instability (τmacro = 0.0483). This suggests a trade-off with higher performance on rare classes within specific tasks at the cost of broader global accuracy and cross-task generalization. Conversely, substituting BM25 for BM42 alongside the Multi-E5 embedding model with Jina V2 reranker yielded a model that perfectly matched the winning configuration’s top-tier Micro-F1 (0.7637) and even achieved the lowest Micro variance in the 2 study (τmicro = 0.0181). However, this BM42 variant experienced a slight degradation 2 in Macro-F1 (0.7302) and a slight increase in Macro variance (τmacro = 0.0365) compared to the BM25 winner. Ultimately, the combination of BM25 with Multilingual-E5 and Jina V2 resides strictly on the Pareto-optimal frontier of the composite score, offering the most rigorous balance: it maintains high global accuracy (Micro-F1) without sacrificing the robust detection of rare abusive clauses (Macro-F1). The optimization of our proposed RAG framework’s components used a randomeffects meta-analysis to robustly select the most suitable models, that represent the most robust configuration of embedding models and rerankers for analyzing potentially abusive Terms of Service. The optimal configuration balances Macro- and Micro-F1 at a low variability. Furthermore, the dense embedding model Multilingual-E5 is a local model, as are the sparse embedding method BM25 and the reranker model Jina v2, supporting the aim of purely local and efficient document analysis.

6.5 Model Quantization The most cost and time intensive component of our proposed RAG framework is the LLM. To achieve reasonably short processing times, the LLM’s massive parameter count quickly becomes a limiting factor of their practical applicability due to the large memory and compute requirements. This experiment tests whether a quantization of the weights’ precision maintains high prediction quality at smaller size and faster processing speeds. In this experiment, we evaluate the quantized versions of the best performing openweight model Qwen3 14b and measure their classification score differences on Illegal, Dark and Gray Clauses. Specifically, we compare our default 8-bit version of Qwen3 14b, that was quantized to an 8-bit integer representation (Dettmers et al. 2022), with the 4-bit version, quantized to 4-bit integers using K-means Medium (Lin et al. 2025; Wu et al. 2025). This quantization method keeps some weights that are considered 27

important at slightly higher precision of 6-bit. The resulting 8-bit model is about 15GB large, compared to about 9.3GB for the 4-bit version. The 8-bit is considered to be performing similarly to a model trained with full 32-bit precision (Dettmers et al. 2022), and the 4-bit variants are considered "almost universally optimal" (Dettmers and Zettlemoyer 2023) wrt. total model size and accuracy. Depending on the compute hardware, a speedup between 3x and 4x of 4-bit vs 16-bit can be expected (Frantar et al. 2023; Lin et al. 2025). Table 9: Model Performance of 8-bit and 4-bit quantized Qwen3 14b with RAG variants on the datasets Illegal, Dark, and Gray Clauses. Illegal Model Qwen3 14b 8-bit (Hybrid) Qwen3 14b 8-bit (Dense) Qwen3 14b 4-bit (Hybrid) Qwen3 14b 4-bit (Dense)

Dark

Gray

M-F1

µ-F1

M-F1

µ-F1

M-F1

µ-F1

0.55±0.01 0.49±0.01 0.49±0.00 0.50±0.01

0.71±0.01 0.72±0.01 0.72±0.01 0.72±0.00

0.54±0.01 0.62±0.00 0.52±0.01 0.62±0.01

0.76±0.01 0.79±0.00 0.75±0.01 0.79±0.00

0.71±0.00 0.70±0.02 0.72±0.01 0.72±0.00

0.75±0.00 0.77±0.01 0.77±0.00 0.78±0.01

The results in Table 9 show that the models using dense RAG maintain or even improve their performance on all three tasks. The scores on Gray Clauses of 0.72 MF1 and 0.78 µ-F1 are even beating the global best. The hybrid RAG model does not exhibit such consistency and differences range between -6 to +1 point M-F1 and +1 to +3 points µ-F1. These surprising results, that we averaged over 6 repetitions, with both performance drops and gains, show that the quantization of models is worth further investigation, that we leave for future work. To conclude, we showed that model quantization has the potential to achieve shorter processing times by reducing memory and compute requirements, enabling local processing on a wider base of installed computers, while maintaining or even increasing prediction quality. However, the inconsistency of hybrid RAG requires further investigation.

7 Discussion This section situates our results in the broader literature on automated analysis of potentially abusive clauses in consumer contracts. We compare the main methodological strands in the field, identify the limitations they leave unresolved, and explain how the present study contributes to both the legal and technical development of the area. Existing work distinguishes between two related tasks: detecting potentially problematic clauses and classifying the specific type of potential abuse. These tasks are closely connected but differ in difficulty, especially where legal categories vary in doctrinal specificity and interpretive openness. The first comprehensive study on automated unfair-clause detection is CLAUDETTE by Lippi et al. (Lippi et al. 2019). The authors introduce the Unfair Terms of Service dataset, consisting of 50 English-language European consumer contracts annotated with eight categories of potentially abusive clauses. Their system 28

proposes a two-stage pipeline: (1) detection of unfair clauses and (2) multi-class classification into abuse types. For detection, they compare classical Machine Learning methods, i.e., Support Vector Machines, Hidden Markov Models (HMM), and Tree Kernels, with deep learning models such as Convolutional Neural Networks (CNN) and Long Short-Term Memory networks (LSTM). For classification, SVMs are used exclusively. Classical models rely primarily on Bag-of-Words (BoW) representations and syntactic tree structures encoding grammatical relations. CLAUDETTE established the feasibility of clause-level unfairness detection and provided an important benchmark dataset. However, its methodology reflects limitations characteristic of early supervised NLP systems. First, BoW-based SVMs are prone to overfitting and lack robustness to paraphrasing or semantically equivalent reformulations (Löffler et al. 2025). Because representations are largely lexical, generalization to novel formulations is limited. Second, the system does not provide calibrated uncertainty estimates. In legal applications, probabilistic outputs are critical. A prediction with marginal confidence should trigger human review. The absence of predictive uncertainty constrains practical deployment. To address the need for interpretability, Ruggeri et al. (Ruggeri et al. 2022) propose incorporating legal rationales into clause classification using Memory-Augmented Neural Networks (MANNs) (Santoro et al. 2016). Their model jointly processes (i) annotated clauses and (ii) a memory bank of expert-written legal rationales, that are textual justifications explaining why a clause is unfair. The model learns to associate rationales with clauses and controls the inclusion of rationales from the memory banks into the classification process of clauses. The authors report that model makes substantial use of the memory bank. During inference, the network retrieves relevant rationales to support its predictions, thereby improving performance and providing explanation-like outputs. For example, a clause categorized as unilateral termination may retrieve a rationale stating that termination grounds are unspecified, such as “The clause mentions the contract or access may be terminated but does not state the grounds for termination”. This approach represents a significant shift toward explanation-aware modeling. However, it introduces two structural limitations. First, legal rationales must be manually defined in addition to the annotations, substantially increasing the cost of dataset construction. Rationales effectively become extended supervision signals, exacerbating data scarcity. Second, the model’s explanatory capacity is bounded by the predefined rationale set. It cannot generate novel explanations or generalize beyond the manually curated reasoning patterns. Our RAG-based approach retrieves examples that may serve as implicit rationales. We leave the automatic mining of robust rationales via methods such as Graph-RAG (Edge et al. 2025; Ongris et al. 2025) to future work. Rather than treating classification as a parametric prediction task, Dadas et al. (Dadas et al. 2024) formulate it as a ranking problem. Furthermore, they release a Polish-language dataset of 24,000 clauses labeled as safe or abusive, significantly increasing available training data. Clauses are embedded into a semantic vector space and stored alongside their annotations. Given a query clause, the system retrieves the k most similar clauses using cosine similarity and infers the label via majority vote

using k-Nearest Neighbors. This retrieval-based approach has several attractive properties. It is modular, interpretable at the example level, and avoids explicit parametric training for classification. Conceptually, it clusters semantically similar clauses and propagates labels locally within the embedding space. Nevertheless, the method exhibits fundamental limitations. First, it performs similarity matching rather than utilizing in-context reasoning. Retrieved clauses may resemble the query lexically or distributionally, but the system does not analyze why they may be abusive. Second, performance depends heavily on the density and coverage of annotated examples. Sparse regions in embedding space lead to unreliable predictions. Löffler, Martínez, and Rey (Löffler et al. 2025) advance the field by systematically comparing fine-tuned Transformer models and prompted LLMs for detecting and classifying abusive clauses in Chilean ToS. They introduce a dataset of 50 annotated contracts structured according to Chilean legal doctrine (de la Maza Gazmuri 2003; López Díaz 2022; Barrientos Camus 2019; Morales 2018; Pizarro Wilson 2007; SERNAC 2021), comprising four principal groups and 20 clause types. Their pipeline separates detection and classification. For detection, they recommend finetuned Transformer models. For classification, they compare fine-tuning with few-shot prompting of LLMs such as OpenAI GPT-4o (OpenAI 2024) and Meta Llama3 (Dubey et al. 2024). Their results show that with abundant annotated data, fine-tuning outperforms prompting. Conversely, under limited supervision or higher ambiguity, LLMs leverage their broad pretraining to achieve superior performance by injecting up to 10 examples per type of clause into the prompt. Performance improves substantially by up to 16% when prompts include multiple annotated examples per class. Despite these advances, several limitations remain. First, few-shot examples are sampled based on clause length rather than semantic similarity, potentially leading to suboptimal contextual support. Second, while larger prompts constructed via higher shot-counts of few-shot learning increase model performance, the context length grows as well, thus increasing inference cost. Third, the prompting strategy does not fully exploit autoregressive reasoning capabilities that may enhance prediction quality. In our contribution, we compute dense and sparse semantic embeddings for all annotated clauses and store them in a vector database to enable efficient similarity search. At inference time, a query clause is embedded into the same spaces, and the most semantically similar annotated clauses are retrieved and reranked. Unlike pure ranking-based approaches, these examples are not used for majority voting (Dadas et al. 2024). Instead, they are inserted into a structured prompt that guides an LLM to perform classification through contextualized reasoning. This design offers several advantages. Semantic retrieval ensures that injected examples are meaningfully related to the query, replacing heuristic sampling strategies (Löffler et al. 2025). The LLM then performs the final decision step, enabling analysis beyond surface similarity (Löffler et al. 2025; Dadas et al. 2024) and allowing consideration of legal context and structure. Retrieval further grounds the model’s reasoning in domain-specific evidence rather than relying solely on parametric knowledge (Lippi et al. 2019).

The resulting framework integrates embedding-based retrieval with generative language modeling. Retrieval supplies relevant contextual evidence and generation performs classification and legal reasoning over that evidence. The proposed approach addresses the core limitations of prior work: weak semantic grounding in few-shot prompting (Löffler et al. 2025), lack of reasoning in similarity-based systems (Dadas et al. 2024), requirement of expertly-designed legal rationales as part of training data (Ruggeri et al. 2022), and limited robustness of feature-engineered models (Lippi et al. 2019). The goal is improved generalization, greater robustness to paraphrasing, and more transparent automated analysis of Terms of Service. More broadly, our results illustrate a recurring theme in AI and law: predictive performance is not uniform across legal categories because legal concepts themselves differ in determinacy (Barrientos Camus 2019; Löffler et al. 2025). Clauses that correspond to explicit statutory prohibitions are comparatively amenable to annotation and automated detection, whereas clauses governed by open-ended standards such as good faith are harder to classify because disagreement is not merely a matter of data scarcity but also of legal interpretation (Morales 2018; Barrientos Camus 2019). This helps explain why retrieval-augmented prompting is particularly valuable in the present setting. Rather than asking the model to rely exclusively on parametric knowledge, retrieval grounds the prediction in analogous annotated clauses and thereby supplies context that is closer to the structure of legal reasoning by example. This pattern was especially visible in clauses drawn from foreign-origin or globally standardized ToS, some of which contained provisions on foreign jurisdiction, mandatory arbitration, or provider-controlled internal dispute procedures. In the Chilean setting, such clauses are often easier to identify as legally problematic because they can be evaluated against relatively explicit statutory or procedural constraints. By contrast, clauses that do not directly contradict a specific rule, but instead create subtler forms of imbalance or contractual discretion, remain much harder to classify automatically. This contrast helps explain both the comparatively stronger performance on rule-like categories and the continuing difficulty of gray clauses grounded in open-textured standards such as good faith (Barrientos Camus 2019; Morales 2018; Löffler et al. 2025). At the same time, the system should be understood as a decision-support tool rather than a substitute for legal adjudication. The annotation labels reflect expert legal assessment for research purposes, but the legal invalidity or abusiveness of a clause in an individual case remains a matter for judicial or administrative determination (Barrientos Camus 2019; Löffler et al. 2025). This limitation is especially important for gray clauses, where the relevant standard is intentionally open-textured. Accordingly, the main practical value of the framework lies in improving consumer information and supporting preliminary review, not in producing definitive legal judgments.

8 Conclusion We presented a retrieval-augmented framework for the detection and multi-label classification of potentially abusive clauses in Chilean Terms of Service. The framework

combines efficient clause filtering, hybrid retrieval, reranking, and prompt augmentation to support local language models in a legally grounded analysis task. In addition, we introduced the Chilean Abusive ToS Extended corpus and a revised 24-category annotation scheme that more closely reflects the structure and interpretive difficulty of Chilean consumer law. The results show that retrieval-augmented prompting can substantially improve automated contract analysis while reducing dependence on large cloud-based models. This is especially relevant for privacy-preserving and locally deployable legal AI tools. At the same time, the study highlights that automated performance varies with legal determinacy: categories tied to explicit rules are easier to operationalize than those governed by open-ended standards such as good faith. Our contribution is therefore not only technical but jurisprudentially informed. By linking retrieval-based language-model prompting to a doctrinally structured annotation scheme, we provide a practical and extensible approach to AI-assisted consumer contract review. In practical terms, the framework is best understood as supporting the consumer’s right to information and more informed contractual choice in digital markets, rather than as replacing legal advice or judicial determination. Future work should examine cross-jurisdictional transfer, user-facing explanation quality, and the interaction between automated predictions, legal uncertainty, and consumer decision-making. Acknowledgements. • Author contribution Andrea Martínez Freile and Christoffer Löffler contributed to the study conception and design. Material preparation, data collection, and analysis were performed by Tomás Rey Pizarro, Andrea Martínez Freile, Christoffer Löffler, and Daniel Miranda. The legal methodology was contributed by Andrea Martínez Freile and Tomás Rey Pizarro. The Machine Learning methodology was contributed by Christoffer Löffler and Daniel Miranda. The first draft of the manuscript was written by Christoffer Löffler and all authors extended and commented on previous versions of the manuscript. All authors read and approved the final manuscript. • Funding Partial financial support was received from ANID/FONDECYT Iniciación (No. 11250673). • Data availability The annotated dataset will be made publicly available in a repository upon publication. • Code availability The source code and experimental scripts will be made publicly available in a repository upon publication. • Materials availability Not applicable.

Declarations • Competing interests The authors declare no competing interests. • Ethics approval and consent to participate Not applicable. • Consent for publication Not applicable.

Appendix A

Annotated Contracts

The extended dataset contains 50 newly annotated contracts by the following companies: Andes gear, Aramco, Buba, Burger King, Cabify, Wondershare, ChatGPT, CinePlanet, Cinemark, Cookidoo, Cruz Verde, Disney+, Domino’s Pizza, Dunkin Donuts, EFE App, Evercrisp, Diario Financiero, Ansaldo, Infanti, Isdin, JustBurger, Jumbo Prime, KFC App, Knop, Kliper, Lego, Under Armour, Líder, McDonald’s, Melt, Micocola Chile, Money Gram, Movistar App, Niu Sushi, Papa Johns, Philips, Saba, Salcobrand, Sony Music, SOPROLE, Starken, Super Zoo, Trotter, Tommy.cl, Unimarc, Unsplash, La Vinoteca, Yapo.cl, Zara, PG. Previously annotated contracts (Löffler et al. 2025) were published by the following companies: Academia.edu, Airbnb, Amazon, App Copec, Apple, Badoo, Bluexpress, Booking, Box, Canva, Despegar, Dropbox, Ebay, Edreams, Evernote, Facebook, Fitbit, Google, Headspace, Instagram, LinkedIn, MercadoLibre, Microsoft, MyHeritage, Nespresso, Netflix, Nike, Nintendo, Paris, Pokémon GO, Pullman Bus, Rappi, Ripley, Rovio, Skype, Skyscanner, Snapchat, Spotify, Starbucks, Tenpo, Tinder, TripAdvisor, Uber, Vimeo, WhatsApp, Wild Foods, World of Warcraft, X (Twitter), Yahoo, YouTube.

Appendix B

Glossary of Annotation Abbreviations

Table B1 explains the abbreviations used in the annotation scheme for the Chilean Abusive Terms of Service Extended dataset. The glossary is based on the legal definitions developed for the present study under Chilean consumer law, related procedural rules, and connected provisions of the Civil Code. As discussed in the main text, the labels are grouped into three broad families: illegal clauses, which directly contradict an explicit legal norm; dark clauses, which are manifestly abusive and presumptively unfair; and gray clauses, whose abusiveness depends more heavily on contextual interpretation, especially under Article 16(g) LPC and the requirement of good faith. Table B1: Glossary of annotation abbreviations used in the Chilean annotation scheme for potentially abusive Terms of Service. Abbreviation Meaning / legal explanation ILG Illegal clause. Prefix added when a clause, rather than being merely potentially abusive, directly contradicts an express legal rule. Dark Manifestly abusive clause. A provision that creates an evident contractual imbalance and is presumptively unfair under the LPC. Gray Potentially abusive clause whose assessment depends more heavily on interpretation, context, and the open-textured standard of good faith under Article 16(g) LPC. Liq Clause contrary to insolvency or liquidation proceedings (procedimiento concursal ). Used for provisions that conflict with the regime of Law No. 20.720 or with the statutory order of priority among creditors. Continued on next page 33

Abbreviation LPC

ng ret

acp

LPC pro

LPC int

CPC

COT

ter

Table B1 – continued from previous page Meaning / legal explanation Clause contrary to Law No. 19.496 on the Protection of Consumer Rights. Residual category for clauses that violate the LPC without fitting a more specific category. Unjustified refusal to sell (negativa injustificada de venta). Clause contrary to Article 13 LPC. Withdrawal right (retracto). Refers to clauses affecting the consumer’s right to terminate the contract within the legal cooling-off period under Article 3 bis LPC. Tacit acceptance / unlawful acceptance procedure. Refers to clauses that attempt to impose tacit acceptance, for example by mere use or website visit, contrary to Article 12 A LPC. Prohibition on altering judicial competence. Refers to clauses that unlawfully modify the competence of courts or impose a forum contrary to Articles 50-A and 50-H LPC. De-intermediation / unlawful intermediation clause. Refers to clauses by which a provider attempts to evade responsibility for services or goods offered through intermediated or multi-provider arrangements, contrary to Article 43 LPC. Clause contrary to the Chilean Code of Civil Procedure (Código de Procedimiento Civil ). Used for provisions that interfere with judicial stages, deadlines, notifications, procedural rights, appeals, or access to competent justice. Clause contrary to the Chilean Code of Courts (Código Orgánico de Tribunales). Used especially for provisions affecting absolute or relative jurisdiction. Clause contrary to the civil liability regime of the Civil Code. Used where a clause attempts to alter the contractual liability regime, especially under Articles 1489, 1545, 1556 and related provisions of the Civil Code. Clause contrary to the Civil Code more generally. Residual category for Civil Code violations not covered by the specific civil-liability category RC. Unilateral and arbitrary modification of the Terms of Service. Clause allowing the provider to alter the contract, wholly or partially, without giving the consumer a meaningful possibility to object. Linked to Article 16(a) LPC. Unilateral termination. Clause allowing the provider to terminate the contract at its sole discretion, without justified cause or adequate prior specification. Linked to Article 16(a) LPC. Unilateral price modification. Clause allowing the provider to increase the tariff or price of the service unilaterally and without proper justification. Linked to Article 16(b) LPC. Continued on next page

Abbreviation er

ltd

blc

nod

bfe

des reser

des det

des lic

des uni

des unila

des us

des def

Table B1 – continued from previous page Meaning / legal explanation Consumer charged for provider errors. Clause making the consumer bear the consequences of deficiencies, omissions, or administrative/internal errors of the provider. Linked to Article 16(c) LPC. Inversion of the burden of proof (onus probandi ). Clause shifting onto the consumer the burden of proving issues that, under the law, should not be imposed in that way. Linked to Article 16(d) LPC. Limitation of liability. Clause by which the provider seeks, in advance, to exclude or excessively limit liability for non-performance or defective performance. Linked to Article 16(e) LPC. Blank spaces. Clause or contract format leaving blank spaces to be completed later at the provider’s discretion. Linked to Article 16(f) LPC. Limitation on the exercise of consumer rights. Clause that does not eliminate rights entirely but imposes obstacles, delays, or burdensome conditions on their exercise. Linked to Article 16(h) LPC. Clause contrary to good faith (buena fe). Residual gray category for provisions that, in light of the purpose of the contract, generate consumer detriment contrary to Article 16(g) LPC. Right to modify the contract. Clause allowing the provider to eliminate, limit, or suppress elements of the contract or service at its sole discretion, without a meaningful possibility of opposition by the consumer. Reference to internal dispute process. Clause allowing the provider to channel conflicts or complaints into its own internal procedure, potentially affecting consumer rights, deadlines, or access to justice. Unlimited or excessive powers. Clause conferring very broad, perpetual, or irrevocable powers on the provider, often extending beyond the duration of the contract. Change of terms without notice. Clause allowing the provider to modify terms without prior notice or justification, leaving the consumer only the option of ceasing to use the service. Change of terms with notice. Clause allowing the provider to modify terms while notifying the consumer afterwards or by some predefined mechanism. Consumer bears risks from interactions with other users. Clause by which the provider disclaims responsibility for fraud, abuse, or harmful interactions occurring through user-to-user features or spaces under the provider’s control. Consumer indemnifies provider. Clause obliging the consumer to defend, indemnify, hold harmless, or otherwise support the provider in litigation or liability situations, thereby creating an imbalance in contractual burdens. Continued on next page

Table C2: Average Macro and Micro F1 scores (mean ± std) for UnfairTOS Classification. Method Qwen3 14b (Dense) Qwen3 14b (Hybrid) Gemma3 27b (Dense) Gemma3 27b (Hybrid) Majority-Vote (Dense) Majority-Vote (Hybrid) Best CLAUDETTE model

Macro-F1

Micro-F1

0.948 ± 0.003 0.933 ± 0.000 0.931 ± 0.002 0.923 ± 0.004 0.917 ± 0.000 0.893 ± 0.000 0.879 ± -

0.932 ± 0.004 0.918 ± 0.000 0.915 ± 0.003 0.906 ± 0.006 0.904 ± 0.000 0.873 ± 0.000 -

Illegal Method

M-F1

Dark µ-F1

Gray

M-F1

µ-F1

M-F1

µ-F1

0.48±0.04 0.60±0.03 0.50±0.06 0.52±0.06

0.69±0.01 0.73±0.02 0.80±0.01 0.78±0.02

0.55±0.00 0.66±0.00 0.66±0.00 0.56±0.04 0.47±0.31

0.51±0.00 0.61±0.00 0.63±0.01 0.55±0.03 0.52±0.17

0.65±0.00 0.64±0.02

0.76±0.00 0.77±0.01

0.64±0.00 0.62±0.00 0.67±0.00 0.65±0.00

Previous results SVM (TF-IDF) 0.47±0.01 0.63±0.01 GPT-4o-mini (5-shot) 0.61±0.01 0.61±0.01 GPT-4o (10-shot) 0.68±0.01 0.62±0.00 BETO cased 0.63±0.04 0.74±0.01 XLM-RoBERTa-large 0.70±0.01 0.78±0.01 Our proposed method Qwen3 14b (RAG) Qwen3 14b (Hybrid)

0.64±0.01 0.66±0.01

0.66±0.00 0.69±0.00

Table C3: Comparing classification scores for the smaller Chilean Abusive Terms of Service (Löffler et al. 2025) dataset with our proposed method as macro-F1 (M-F1) and micro-F1 (µ-F1).

Abbreviation des risk

des inf

Appendix C

Table B1 – continued from previous page Meaning / legal explanation Consumer assumes risks. Clause shifting to the consumer risks or costs arising from the contract without corresponding provider responsibility, including risks linked to external events, interruptions, or force majeure. Information shared with third parties. Clause allowing the provider to transfer user information to third parties unrelated to the service, potentially affecting privacy or informational self-determination.

Additional Results

Table C3 shows our method’s classification results as M-F1 and µ-F1 for the smaller Chilean Abusive Terms of Service (Löffler et al. 2025) and compares them with the previous state-of-the-art. In the category of potentially "Illegal" clauses, the fine-tuned XLM-RoBERTalarge is the dominant model with 0.70 M-F1 and 0.78 µ-F1). These clauses are often defined by rigid language or even specific keywords in legislation. Thus, fine-tuned 36

models like BERT perform exceptionally well in this more strict textual pattern recognition problem. While the Qwen3 Hybrid RAG approach performs respectably with 0.66 M-F1, it does not beat the specialized encoder. However, we argue that the smaller dataset allows for overfitting to the problems, as our analysis with 100 contracts shows. Our detailed breakdown in Section 6.3.2 suggests that LLMs with RAG excel with more diverse data distributions, as real-world problems are not as clear-cut, black-and-white legal violations. The "Dark" category of the older dataset exhibits diverging performance between fine-tuend models like BETO and XLM-RoBERTa leading µ-F1 and LLMs like Qwen3 (Dense) leading M-F1, significantly outperforming fine-tuned models. This result confirms overfitting during fine-tuning, in the form of overfitting to the majority class, leading to higher micro than macro f1 for BETO with 0.80 vs 0.50. In contrast, the LLMs maintain much smaller gap between the metrics with 0.77 vs 0.64, showing they far better detect rarer classes. The "Gray" category involves ambiguous language and is thus harder to classify. Here, our proposed Hybrid RAG method helps Qwen3 achive the highest score in both metrics with 0.67 M-F1 and 0.65 µ-F1. Fine-tuned models struggle, indicating that LLM’s reasoning capabilities are leveraging the RAG context. On these three smaller datasets, the hybrid RAG approach consistently outperforms or matches the standard RAG approach in almost every metric. The improvement is most visible in the Illegal (µ-F1 +3 points) and Gray (M-F1 +3 points) categories. This justifies the added complexity of the Hybrid architecture, as it appears to retrieve better context or integrate it more effectively for classification.

References Bajaj P, Campos D, Craswell N, et al (2018) MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. https://doi.org/10.48550/arXiv.1611. 09268, arXiv:1611.09268 Baraona González J (2014) La regulación contenida en la ley 19.496 sobre protección de los derechos de los consumidores y las reglas del código civil y comercial sobre contratos: un marco comparativo. Revista Chilena de Derecho 41(2):381–408. URL https://revistachilenadederecho.uc.cl/index.php/Rchd/article/view/71355 Barrientos Camus F (2019) Lecciones de derecho del consumidor, primera edición edn. Thomson Reuters, Santiago, Chile Beltagy I, Peters ME, Cohan A (2020) Longformer: The Long-Document Transformer. arXiv:2004.05150 Borenstein M, Hedges LV, Higgins JP, et al (2010) A basic introduction to fixedeffect and random-effects models for meta-analysis. Research Synthesis Methods 1(2):97–111. https://doi.org/10.1002/jrsm.12

Cañete J, Chaperon G, Fuentes R, et al (2023) Spanish Pre-trained BERT Model and Evaluation Data. arXiv:2308.02976 Chalkidis I, Fergadiotis M, Malakasiotis P, et al (2020) LEGAL-BERT: The Muppets straight out of Law School. arXiv:2010.02559 Conneau A, Khandelwal K, Goyal N, et al (2020) Unsupervised Cross-lingual Representation Learning at Scale. arXiv:1911.02116 Cortes C, Vapnik V (1995) Support-vector networks. Machine Learning 20(3):273–297. https://doi.org/10.1007/BF00994018 Dadas S, Kozłowski M, Poświata R, et al (2024) A support system for the detection of abusive clauses in B2C contracts. Artificial Intelligence and Law https://doi.org/ 10.1007/s10506-024-09408-8 de la Maza Gazmuri I (2003) Contratos por adhesión y cláusulas abusivas ¿Por qué el Estado y no solamente el mercado? Revista chilena de derecho privado (1):109–148 DeepSeek-AI, Liu A, Mei A, et al (2025) DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models. https://doi.org/10.48550/arXiv.2512.02556, arXiv:2512.02556 Demsar J (2006) Statistical Comparisons of Classifiers over Multiple Data Sets. Journal of Machine Learning Research (7):1–30 DerSimonian R, Laird N (1986) Meta-analysis in clinical trials. Controlled Clinical Trials 7(3):177–188. https://doi.org/10.1016/0197-2456(86)90046-2 Dettmers T, Zettlemoyer L (2023) The case for 4-bit precision: K-bit Inference Scaling Laws. In: Proceedings of the 40 Th International Conference on Machine Learning, Honolulu, Hawaii, USA Dettmers T, Lewis M, Shleifer S, et al (2022) 8-bit Optimizers via Block-wise Quantization. https://doi.org/10.48550/arXiv.2110.02861, arXiv:2110.02861 Devlin J, Chang MW, Lee K, et al (2019) BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, Douze M, Guzhva A, Deng C, et al (2025) THE FAISS LIBRARY. IEEE Transactions on Big Data pp 1–17. https://doi.org/10.1109/TBDATA.2025.3618474 Dubey A, Jauhri A, Pandey A, et al (2024) The Llama 3 Herd of Models. arXiv:2407.21783

Edge D, Trinh H, Cheng N, et al (2025) From Local to Global: A Graph RAG Approach to Query-Focused Summarization. https://doi.org/10.48550/arXiv.2404. 16130, arXiv:2404.16130 Frantar E, Ashkboos S, Hoefler T, et al (2023) GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers. https://doi.org/10.48550/arXiv. 2210.17323, arXiv:2210.17323 Gao Y, Xiong Y, Gao X, et al (2024) Retrieval-Augmented Generation for Large Language Models: A Survey. https://doi.org/10.48550/arXiv.2312.10997, arXiv:2312.10997 Glass M, Rossiello G, Chowdhury MFM, et al (2022) Re2G: Retrieve, Rerank, Generate. https://doi.org/10.48550/arXiv.2207.06300, arXiv:2207.06300 Greco CM, Tagarelli A (2024) Bringing order into the realm of Transformer-based language models for artificial intelligence and law. Artificial Intelligence and Law 32(4):863–1010. https://doi.org/10.1007/s10506-023-09374-7 Gutiérrez-Fandiño A, Armengol-Estapé J, Gonzalez-Agirre A, et al (2021) Spanish Legalese Language Model and Corpora. arXiv:2110.12201 Jain A, Aggarwal P, Saladi A (2025) AutoChunker: Structured Text Chunking and its Evaluation. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track), Vienna, Austria Jaiswal S, Bisht P, Kansara K, et al (2025) Comparison of chunking techniques across diverse document types in NLP retrieval tasks. In: 2025 International Conference on Responsible, Generative and Explainable AI (ResGenXAI). IEEE, Bhubaneswar, India, pp 1–6 Jara Amigo R (1999) Ámbito de aplicación de la ley chilena de protección al consumidor: inclusiones y exclusiones. In: Corral Talciani H (ed) Derecho del consumo y protección al consumidor: estudios sobre la Ley N° 19.496 y las principales tendencias extranjeras. Universidad de los Andes, Santiago Kamath A, Ferret J, Pathak S, et al (2025) Gemma 3 Technical Report. https://doi. org/10.48550/arXiv.2503.19786, arXiv:2503.19786 Kingma DP, Ba∗ JL (2015) ADAM: A METHOD FOR STOCHASTIC OPTIMIZATION. In: International Conference on Learning Representations, San Diego, CA Lewis P, Perez E, Piktus A, et al (2021) Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. https://doi.org/10.48550/arXiv.2005.11401, arXiv:2005.11401

Li S, Ning X, Wang L, et al (2024) Evaluating Quantized Large Language Models. In: Proceedings of the 41st International Conference on Machine Learning, Vienna, Austria Lin J, Tang J, Tang H, et al (2025) AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration. GetMobile: Mobile Computing and Communications 28(4):12–17. https://doi.org/10.1145/3714983.3714987 Lippi M, Palka P, Contissa G, et al (2019) CLAUDETTE: An Automated Detector of Potentially Unfair Clauses in Online Terms of Service. Artificial Intelligence and Law 27(2):117–139. https://doi.org/10.1007/s10506-019-09243-2, 1805.01217 Liu Y, Ott M, Goyal N, et al (2019) RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv:1907.11692 Löffler C, Martínez Freile A, Rey Pizarro T (2025) Predicting potentially abusive clauses in Chilean terms of services with natural language processing. Artificial Intelligence and Law https://doi.org/10.1007/s10506-025-09462-w López Díaz PV (2022) El consumidor hipervulnerable como débil jurídico en el derecho chileno: Una taxonomía y alcance de la tutela aplicable. Latin american legal studies 10(2):340–415. https://doi.org/10.15691/0719-9112vol10n2a7 Lu M, Chen C, Eickhoff C (2025) Pathway to Relevance: How Cross-Encoders Implement a Semantic Variant of BM25. In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Suzhou, China, pp 25525–25547 Morales ME (2018) Algunos problemas de la extensión del Derecho del consumo a contratos entre empresarios en el ordenamiento jurídico chileno. In: Estudios de Derecho Del Consumidor II, 2021st edn. No. VII in Jornadas Nacionales de Derecho Del Consumo, Rubicón, Santiago, Chile, p 176 pp. Ochal M, Patacchiola M, Storkey A, et al (2021) Few-Shot Learning with Class Imbalance. https://doi.org/10.48550/arXiv.2101.02523, arXiv:2101.02523 Ongris JG, Darari F, Tobing BCL, et al (2025) Benchmarking KG-based RAG Systems: A Case Study of Legal Documents. In: RAGE-KG 2025: The Second International Workshop on Retrieval-Augmented Generation Enabled by Knowledge Graphs, Co-Located with ISWC 2025, Nara, Japan OpenAI (2024) GPT-4o System Card OpenAI (2024) New embedding models and API updates OpenAI, Agarwal S, Ahmad L, et al (2025) Gpt-oss-120b & gpt-oss-20b Model Card. https://doi.org/10.48550/arXiv.2508.10925, arXiv:2508.10925

Pires T, Schlinger E, Garrette D (2019) How Multilingual is Multilingual BERT? In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Florence, Italy, pp 4996– 5001, https://doi.org/10.18653/v1/P19-1493 Pizarro Wilson C (2007) El fracaso de un sistema: Análisis empírico y dogmático del control de cláusulas abusivas en contratos por adhesión. Revista de derecho (Valdivia) 20(2). https://doi.org/10.4067/S0718-09502007000200002 Qdrant (2026) Qdrant: High-performance, massive-scale vector database and vector search engine. https://github.com/qdrant/qdrant Ram O, Levine Y, Dalmedigos I, et al (2023) In-Context Retrieval-Augmented Language Models. https://doi.org/10.48550/arXiv.2302.00083, arXiv:2302.00083 Reimers N, Gurevych I (2019) Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Association for Computational Linguistics, Hong Kong, China, p 3982–3992, https://doi.org/10.18653/v1/D19-1410 Robertson S, Zaragoza H (2009) The Probabilistic Relevance Framework: BM25 and Beyond. Foundations and Trends® in Information Retrieval 3(4):333–389. https: //doi.org/10.1561/1500000019 Ruggeri F, Lagioia F, Lippi M, et al (2022) Detecting and explaining unfairness in consumer contracts through memory networks. Artificial Intelligence and Law 30(1):59–92. https://doi.org/10.1007/s10506-021-09288-2 Santoro A, Bartunov S, Botvinick M, et al (2016) Meta-learning with memoryaugmented neural networks. In: Balcan MF, Weinberger KQ (eds) Proceedings of The 33rd International Conference on Machine Learning, Proceedings of Machine Learning Research, vol 48. PMLR, New York, New York, USA, pp 1842–1850, URL https://proceedings.mlr.press/v48/santoro16.html Sechidis K, Tsoumakas G, Vlahavas I (2011) On the Stratification of Multi-label Data. In: Gunopulos D, Hofmann T, Malerba D, et al (eds) Machine Learning and Knowledge Discovery in Databases, vol 6913. Springer Berlin Heidelberg, Berlin, Heidelberg, p 145–158, https://doi.org/10.1007/978-3-642-23808-6_10 SERNAC (2021) Resolución Exenta N◦ 931. Aprueba circular interpretativa sobre criterios de equidad en las estipulaciones contenidas en contratos de adhesión de consumo. Shi F, Chen X, Misra K, et al (2023) Large language models can be easily distracted by irrelevant context. In: Krause A, Brunskill E, Cho K, et al (eds) Proceedings of the 40th International Conference on Machine Learning, Proceedings of Machine 41

Learning Research, vol 202. PMLR, pp 31210–31227, URL https://proceedings.mlr. press/v202/shi23a.html Singh A, Fry A, Perelman A, et al (2025) OpenAI GPT-5 System Card. https://doi. org/10.48550/arXiv.2601.03267, arXiv:2601.03267 Wang F, Li Y, Xiao H (2025) Jina-reranker-v3: Last but Not Late Interaction for Listwise Document Reranking. https://doi.org/10.48550/arXiv.2509.25085, arXiv:2509.25085 Wang L, Yang N, Huang X, et al (2024a) Text Embeddings by WeaklySupervised Contrastive Pre-training. https://doi.org/10.48550/arXiv.2212.03533, arXiv:2212.03533 Wang L, Yang N, Huang X, et al (2024b) Multilingual E5 text embeddings: A technical report. Tech. Rep. MSR-TR-2024-45, Microsoft Wu K, Wu E, Zou J (2024) ClashEval: Quantifying the tug-of-war between an LLM’s internal prior and external evidence. In: 38th Conference on Neural Information Processing Systems (NeurIPS 2024) Track on Datasets and Benchmarks Wu X, Zhou B, Gao Z, et al (2025) KLLM: Fast LLM Inference with K-Means Quantization. https://doi.org/10.48550/arXiv.2507.23035, arXiv:2507.23035 Yang A, Li A, Yang B, et al (2025) Qwen3 Technical Report. https://doi.org/10. 48550/arXiv.2505.09388, arXiv:2505.09388 Yao R, Zhang Y, Song S, et al (2025) ParetoRAG: Leveraging Sentence-Context Attention for Robust and Efficient Retrieval-Augmented Generation. Findings of the Association for Computational Linguistics: EMNLP 2025 Zarrinkia Y, Srinivasan V, Thomo A (2026) The reasoning bottleneck in graph-rag: Structured prompting and context compression for multi-hop qa. URL https:// arxiv.org/abs/2603.14045, arXiv:2603.14045

Record · ID 224512 · SHA-256 010d25b1ac980784
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.