Show Me How You Reason and I’ll Tell You Who You Are: Reasoning Graphs for Robust LLM Authorship Attribution Zlata Kikteva1 , Artur Romazanov1 , Annette Hautli-Janisz1 , and Ramon Ruiz-Dolz2 1 University of Passau, Germany 2 University of Dundee, United Kingdom
arXiv:2607.14905v1 [cs.CL] 16 Jul 2026
Abstract
searchers report on increased proliferation of LLMauthored material on social media and other online Given the current trend to employ large lanspheres (Sun et al., 2025; Dolezal et al., 2026). As guage models (LLMs) in almost any imaginsuch, there is no denying the fact of a significant able context, LLM-generated text detection and rise in machine authorship in the near past. authorship attribution have become a pressing issue. Prior work has primarily focused on LLM-authored text detection task has generated surface-level linguistic features, an approach considerable interest within the NLP community, shown to be susceptible to paraphrasing and making it one of the prominent research questions other obfuscation techniques. In this paper, in recent years (Wu et al., 2025). This is further we go beyond the linguistic surface, extractevidenced by the significant number of machineing and analysing reasoning structures in LLMgenerated text detection benchmarks and datasets generated texts with the goal of capturing more covering multiple languages, domains, and gencomplex signals of LLM authorship. We proerative models (Macko et al., 2023; La Cava and pose a graph neural network approach that leverages reasoning graphs extracted by an Tagarelli, 2025). With numerous studies reportargument mining pipeline, demonstrating iming on the high performance of detection models proved robustness and generalisation over a (Wang et al., 2024b; Mitchell et al., 2023; Hans traditional Longformer baseline. Our approach et al., 2024), the task might appear trivial. Howoutperforms the baseline by up to 27 percentever, most of the benchmarks focus on specific age points under the obfuscation attacks such model versions and rarely consider generalisation as paraphrasing and backtranslation, and 19 capabilities across versions, a scenario which repercentage points when evaluated on the texts generated by the unseen model versions, simuflects the challenges of a fast-paced technological lating real-world conditions in which new LLM landscape. versions are continuously released. Furthermore, various authorship obfuscation techniques, such as word-level perturbations or text 1 Introduction paraphrasing, have been shown to noticeably deEver since the popularisation of large language crease the detection models’ performance (Macko models (LLMs) and their growing ubiquity in ed- et al., 2024; Zhou et al., 2024b; Xing et al., 2024). ucation, scientific writing, and peer review, they With the proliferation of tools for ‘humanising’ AIhave been misused. This is showcased by seem- authored text online, it is necessary to include obingly high-quality plagiarised work in education fuscated text variants in the evaluation of detection (Kasneci et al., 2023) or ICLR-26 suspecting 21% models if the aim is to assess performance in a of fully AI-generated paper reviews.1 Simultane- setting that reflects real-world use of AI detection ously, there are growing concerns among the gen- systems, e.g., educators verifying authorship in stueral public over the proportion of LLM-generated dents’ writing. to human-authored content online,2 raising quesIn other scenarios, however, it is necessary to tions about the future of the internet.3 Indeed, re- go deeper than simple detection. Earlier work sug1 gests that models from different providers exhibit https://www.nature.com/articles/ d41586-025-03506-6 different behavioural tendencies in terms of their 2 https://graphite.io/five-percent/ biases (Bang et al., 2024), ideologies (Buyl et al., ai-now-writes-as-many-online-articles-as-humans-do 3 2026), or capabilities for disinformation generation https://www.forbes.com.au/news/innovation/ is-ai-quietly-killing-itself-and-the-internet (Vykopal et al., 2024). Combined with the risk 1
No obfuscation
Nodes: Sent. emb. Edges: Arg. relations
Paraphrasing
Train GNN model
Backtranslation Generate LLM-OWL-AE essay corpus (4 LLM familes x 2 versions each)
Apply authorship obfuscation strategies
Construct reasoning graphs (oAMF argument mining pipeline)
Attribute authorship to LLMs
Figure 1: Reasoning graph-based approach using GNN architecture for LLM authorship attribution.
of misuse of the LLM technology, for example, for the automation of online disinformation campaigns (Guo, 2024), it is increasingly important to be able to identify the model that authored a text. Some work has begun to move beyond the problem of detecting LLM-generated text (La Cava and Tagarelli, 2025), focusing on authorship attribution, i.e., backtracking text’s origin to a specific model provider (Yang et al., 2023). In this paper, we address the task of attributing authorship of LLM-generated texts, with the challenges that arise in real-world application scenarios in mind. To this end, we focus on two key challenges: (1) Robustness in conditions when the LLM-authored texts are obfuscated, and (2) generalisation across different versions of LLMs belonging to the same family.
et al., 2021; Zhang et al., 2023), matching argument extraction (Mao et al., 2024), and winning stance prediction (Ruiz-Dolz et al., 2023), we go significantly beyond linguistic and semantic representations for authorship attribution and build on a meta-linguistic, pragmatic representation of the underlying textual structure. To evaluate the robustness and generalisation of the reasoning graphbased approach, we create a corpus of 2,240 LLMgenerated argumentative essays spanning eight different model versions belonging to four different families (Gemma, Qwen, Llama, and Phi). We further augment this with three obfuscated variants per essay via paraphrasing and backtranslation (through French and Turkish), resulting in a corpus of 8,960 essays in total. Our approach to authorship attribution with reasoning graphs is visualised in Figure 1. Results show that our reasoning graph-based fingerprinting fares significantly better (up to +27% F1-score) than the text-only baseline in terms of both robustness (i.e., under the obfuscation conditions) and generalisation, when we train and test on different model versions of the same family (e.g., train on Llama 3.3 and test on Llama 4). Our contributions are as follows: (1) We create and publicly release a new dataset of LLM-generated argumentative essays across different model versions and families for generalisation assessment and under different perturbation strategies for robustness assessment. (2) We propose a novel reasoning graphbased approach to authorship attribution, combining language and reasoning-based features, which demonstrates superior performance in terms of robustness and generalisation when compared to a text-only approach.
Prior work has shown that with only surfacelevel lexical features as an input, it is often difficult (or impossible) to ascertain the specific model version that was used to generate a text (Huang et al., 2025). In addition to that, Li et al. (2025) demonstrate that a syntax-graph-based approach exhibits a much more robust performance under perturbation conditions than current transformer-based approaches. Given these insights, we propose a novel approach to authorship attribution with reasoning graphs as input to train a classification model, using generalisation and robustness as the two main evaluation criteria. By learning from structured graph-based representations of the underlying reasoning, constructed using an argument mining pipeline, our model predicts the model that was used to generate a text based on structural reasoning features in the form of arguments and the relations between them. By way of graph neural networks (GNNs), models that have been applied to the argumentation-related tasks, including argument acceptability prediction (Kuhlmann and Thimm, 2019; Malmqvist et al., 2020; Cibier and Mailly, 2024; Gehlot et al., 2026), argumentative component identification (Ruggeri
2
Related Work
Detection Liu et al. (2023) investigate the task of LLM-generated text detection for argumentative essays authored by different GPT models and report high performance results on the task. Such 2
findings are not uncommon in Transformer-based approaches to LLM-generated text detection under in-domain evaluation scenarios (Wang et al., 2024b). In contrast, Abassy et al. (2024) report a noticeable drop in performance when evaluating on the unseen domains and generators, with Dugan et al. (2024) making similar observations. Furthermore, the detector model accuracies vary when evaluated on the different model versions from the same family (Yu et al., 2025). This highlights a critical challenge: 90%+ accuracy has limited practical significance if the performance does not generalise well, especially given the fast-paced landscape where new models and model versions are released regularly.
tures, especially given how fast generative models change and improve (Bevendorff et al., 2025). The approach adopted in this paper goes beyond the surface-based line of earlier work by taking into account the underlying reasoning structure of essays in order to attribute individual texts to specific LLMs and their versions.
3
Data
We create a new corpus of LLM-generated argumentative essays based on the Purdue Online Writing Lab guidelines,4 a widely used academic writing resource. There are two main reasons for creating our own dataset: First, it is necessary to ensure the argumentative essay length is sufficient for constructing informative reasoning graphs. This condition rules out a significant amount of publicly available corpora, because these consist of short machine-generated text sequences. Secondly, to evaluate the generalisation of our approach, data generated by LLMs belonging to different families with at least two model versions per family is required – a condition that none of the earlier datasets fulfills. In the following sections, we describe the process to generate LLM-OWL-AE.5
Obfuscation At the same time, some research focuses on the robustness of the detection models when faced with the intentionally modified versions of the LLM-generated texts, aiming to investigate the extent to which the obfuscation hinders the detection and attribution of generative models (Uchendu et al., 2023; Chakraborty et al., 2023; Macko et al., 2024). Some of the most widely used approaches rely on rewriting or paraphrasing, either by way of LLMs (Fang et al., 2025) or dedicated paraphrasing tools (Krishna et al., 2023; Sadasivan et al., 2023), as well as backtranslation (Altakrori et al., 2022; Macko et al., 2024; Ayoobi et al., 2025), while word-level perturbations are frequently employed in the adversarial attacks targeting the detector architecture (Shi et al., 2024; Wang et al., 2024a; Zhou et al., 2024a).
3.1
Data Generation
The generation process covers eight LLMs from four different families (two versions per family): Gemma3 (27B), Gemma4 (31B), Qwen3 (32B), Qwen3.5 (35B), Llama3.3 (70B), Llama4 (109B), Phi3 (14B), and Phi4 (14B). We refer to the essays generated by the earlier model versions (Gemma3 released in Mar. 2025, Qwen3 in Apr. 2025, Llama3.3 and Phi3 in Apr. 2024) as LLM-OWLAE-I. The essays generated with the later model versions are called LLM-OWL-AE-II (Gemma4 released in Mar. 2026, Qwen3.5 in Feb. 2026, Llama4 in Apr. 2025, Phi4 in Dec. 2024). We focus on open LLMs because, as pointed out in previous work, these models make the task more challenging due to the increased diversity in their architectures and training data (La Cava and Tagarelli, 2025). All models are run with their respective 4-
Attribution The significant increase in model providers, each of them exhibiting different generative styles, has sparked interest in a particular sub-task: LLM authorship attribution (Yang et al., 2023). Beyond simply identifying whether a text is human-written or LLM-generated (or a mixture of both), authorship attribution aims at identifying the specific model that generated a given text (Li et al., 2023). One of the first studies in this area conducts a stylometric analysis based on lexical, syntactic, and structural features of texts generated by six different models from the GPT and Llama families (Kumarage and Liu, 2023). Their findings reveal clear differences between closed and openweight model families, evidencing the relevance of analysing models not only individually but also considering their families. A recent analysis highlights the challenges of LLM authorship attribution when relying only on surface-level language fea-
4
https://owl.purdue.edu/owl/general_writing/ academic_writing/essay_writing/argumentative_ essays.html 5 The LLM-OWL-AE corpus will be released under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International licence (CC BY-NC-SA 4.0) and will be made publicly available in a GitHub repository together with the code upon acceptance of the paper.
3
bit quantised versions in Ollama6 with temperature set to 1. They were prompted to generate essays of about 500 words (max. 600, min. 400) following the Purdue OWL argumentative essay writing instructions, covering 140 different topics, such as animal rights or climate change, and two stances (one in favour and one against) for each topic. Topics are adopted from Ruiz-Dolz et al. (2024). The process results in a collection of 2,240 essays (280 essays per model version; 560 essays per model family). 3.2
with the paraphrasing setup, backtranslation is also performed at the paragraph level. We conduct a manual sanity check on a sample of the data to ensure that the obfuscated texts remain semantically coherent and preserve the meaning of the original texts. 3.3
In terms of the linguistic surface of the generated essays, we find several differences across different model families, model versions, as well as different perturbation strategies, indicating a varied linguistic surface learned during the training of a text-based model. The detailed dataset statistics are presented in Appendix A.
Obfuscated Data Variants
To investigate the robustness of the models’ performance on the task of authorship attribution, we adopt two widely used authorship obfuscation techniques, namely paraphrasing and backtranslation (Macko et al., 2024; Zhou et al., 2024b). These methods introduce lexical and syntactic variation while preserving semantic meaning, thus potentially reducing authorship signals. We do not employ word-level substitution strategies or characterlevel perturbations. The former would only have a limited impact on the reasoning graph, while the latter might negatively impact the performance of the argument identification pipeline.
Essay length Depending on the model, the average length of an essay ranges from 466 to 543 words, with later model versions generating shorter essays. When it comes to the perturbation, backtranslation produces shorter essays compared to the original ones, with an average reduction of 140 words for French and 80 words for Turkish, while paraphrasing tends to maintain the original essay length better. Lexical diversity The lexical diversity of the essays is measured with the MTLD score (McCarthy and Jarvis, 2010) as an approximation of the diversity of the used vocabulary. We find that both model versions of the Llama family generate essays with the least diverse vocabulary, indicated by the scores of 103 and 87, while other models have scores ranging from 129 to 196 (higher scores indicate a higher degree of lexical diversity). Unlike the word count, the MTLD scores appear to be maintained better between the model versions. Paraphraser has a negative effect on the richness of the vocabulary (scores ranging from 72 to 96), which results in models of the Qwen and Phi families dropping about 90 or more points when compared to the original essays. The backtranslation strategy has less of a negative impact on the lexical diversity, with the scores dropping by no more than 50 points.
Paraphrasing For paraphrasing, we utilise DIPPER paraphrasing tool (Krishna et al., 2023), a detection-model-agnostic method (as opposed to the adversarial paraphrasers) that performs lexical rephrasing as well as sentence reordering. As DIPPER is trained to rewrite text at the paragraph level, paraphrasing is applied to each paragraph individually, and then the paraphrased paragraphs are recombined in their original order to reconstruct the full essay. For lexical and order diversity parameters, the settings are adopted as reported by the authors of the paper to achieve optimal performance (L60, O60). Backtranslation Open-source machinetranslation models by OPUS-MT (Tiedemann and Thottingal, 2020; Tiedemann, 2020) are used for backtranslation via French and Turkish languages. We choose French as a lexically and syntactically close language to English. Turkish is chosen due to its minimal lexical overlap with English and its distinct morphological properties as an agglutinative language. This allows us to test different degrees of obfuscation. For consistency 6
Essay Linguistic Surface
Syntactic complexity The syntactic complexity of the essays is approximated in terms of the mean number of conjuncts, clausal modifiers of nouns, adverbial clause modifiers, clausal complements, clausal subjects and parataxis per sentence. Models of the Qwen family, especially the later version, tend to generate simpler sentences, while earlier
https://ollama.com/
4
Table 1: Train/test splits across dataset partitions. I refers to data partition LLM-OWL-AE-I; II to LLMOWL-AE-II. The S AME -V ERSION setting includes an extended test set with 80 additional essays per model, indicated by a * next to the data partition name.
Llama and Gemma use more complex structures. Under the paraphrasing strategy and backtranslation via Turkish, there is a minor drop in complexity, with sentences after French backtranslation being more complex than the original ones.
4
Config
Method
Our approach to authorship attribution is as follows: We first apply an existing argument mining pipeline to extract argument relations within each essay, representing each essay as a graph with text segments as nodes and the argument relations as edges. The resulting graphs are used to train and evaluate a GNN, which classifies each essay into model families and versions.
Train set
Test set
S AME -V ERSION
I II
I* II*
C ROSS -V ERSION
I II
II I
into a one-hot-encoded vector by assigning a value of 1 to the class with the highest probability and 0 to all others Wargmax = (winf , wconf , wrephr ). The resulting vector contains three elements, each corresponding to one of the relation classes: inference, conflict, and rephrase. With the probabilities strategy, the four-element vector contains raw probabilities from the ARIR module for the three relation classes (inference, conflict, rephrase) and the no relation probability Wprobs = (winf , wconf , wrephr , wno_rel ). Under each of the edge processing strategies, a separate homogeneous directed graph is constructed for each essay.
Argument relation extraction oAMF (Open Argument Mining Framework) (Gemechu et al., 2025) is an open-source modular end-to-end argument mining pipeline. The TARGER module (Chernodub et al., 2019) segments the individual essays into argumentative discourse units (ADUs), which roughly correspond to sentences. We then use the ARIR module (Ruiz-Dolz et al., 2021) to predict the probability of inference (relation of support), conflict (relation of attack), rephrase (relation between two ADUs when one is used to reformulate another), and no relation between two ADUs.
Graph Classification The resulting graphs are being classified by a GNN : G → Y, which aggregates the node representations of a graph to predict a label for a model ŷ = GNN(G) that produced an original essay.
Graph construction As the first step in the pipeline, we create graph representation G = (V, E, W ), in which nodes V = {vi } are ordered according to their appearance in the text, and directed edges E = {(vi , vj ) ∈ V × V | i < j} represent relations W : E P → [0, 1]4 , W (e) = (winf , wconf , wrephr , wno_rel ), k wk (e) = 1 between nodes as retrieved from the ARIR module. The existence of an edge is verified with predefined thresholds T = (tinf , tconf , trephr ) for relations of inference, conflict, and rephrase. If any probability of the relation is above the threshold for the respective relation, we retain the edge E ′ = {e ∈ E ∃k ∈ {inf, conf, rephr} : wk (e) ≥ Tk }.
5
Experiments
5.1
Experimental Design
Train/test splits We consider two train/test split configurations of the LLM-OWL-AE corpus: S AME -V ERSION is a standard setup, in which both training and test data are generated by the same model version; C ROSS -V ERSION evaluates generalisation to the data produced by unseen model version. As illustrated in Table 1, in S AME -V ERSION, the two dataset partitions, LLM-OWL-AE-I (essays by earlier model versions, i.e., Gemma3, Qwen3, Llama3.3, Phi3) and LLM-OWL-AE-II (essays by later model versions, i.e., Gemma4, Qwen3.5, Llama4, Phi4), are treated independently, i.e., for each partition, training, development and test data are drawn from the same partition, e.g., models trained on LLM-OWL-AE-I are evaluated on LLMOWL-AE-I. We use 200 essays per model for training and development, and an additional 80 essays
Node encoding The sentence-transformer model all-mpnet-base-v2 (Reimers and Gurevych, 2019) generates the sentence embeddings for each graph node vi . Edge encoding There are two interchangeable edge processing strategies. Under the argmax strategy, the raw probability distribution over the relation classes from the ARIR module is converted 5
per model for testing, covering 100 topics in the training set and 40 topics in the test. In C ROSS -V ERSION, the models are trained and evaluated across the partition: The models trained on the LLM-OWL-AE-I are evaluated on the LLM-OWL-AE-II, and the models trained on LLM-OWL-AE-II are evaluated on LLM-OWLAE-I. We use 200 essays per model for training from one partition and 200 essays per model for testing from the other partition.
we report the average F1-score across the runs in Tables 2 and 3 (Text-only baseline). 5.3
Reasoning graphs are used to train and evaluate four distinct GNN layer types: Graph Convolutional Networks (GCN) (Kipf and Welling, 2017), Graph Attention Networks (GAT) (Veličković et al., 2017), Graph Transformer (Shi et al., 2021), and General Powerful Scalable (GPS) networks (Rampášek et al., 2022). The architectures differ primarily in their neighbourhood aggregation mechanisms, with GCN as a baseline using averaged neighbour aggregation, GAT learning attention weights for the neighbours, Graph Transformer capturing global attention, and GPS combining local message-passing with transformer-style global attention. Based on the average path length of 7.5 for the essay graphs, we evaluate configurations with up to 7 GNN layers as an empirically plausible range (Li et al., 2018). Node representations from the GNN layers of a model are aggregated into a graph-level embedding using global attention pooling. The resulting representation is passed to a classification head consisting of a LayerNorm, a 256-dimensional feed-forward layer with GELU activation, dropout with rate 0.2, and a final feed-forward projection to the label space of the 4 families: Gemma, Qwen, Llama, or Phi GNN : G → Y, ŷ = GNN(G), Y = {Gemma, Qwen, Llama, Phi}. All models are trained for 30 epochs with early stopping (patience = 5). Each experiment is repeated with 3 random seeds, and we report the average F1-score across the runs in Tables 2 and 3 (Text + Reasoning Structure).
Obfuscation strategies To investigate robustness, we evaluate the models’ performance on the original essays (i.e., non-obfuscated essays) and essays obfuscated under paraphrasing, backtranslation via French, and backtranslation via Turkish strategies. Edge processing strategies in graphs We apply thresholds to the relation probabilities from the ARIR module to filter the edges with low label certainty. To evaluate how different decisions boundary for the argument relation prediction affect GNN models’ performance, we define the (1) (1) (1) three sets of thresholds T1 = (tinf , tconf , trephr ) = (0.5, 0.5, 0.5), T2
=
(2)
(2)
(2)
(tinf , tconf , trephr )
=
(3) (3) (3) (tinf , tconf , trephr )
(0.7, 0.7, 0.7), and T3 = = (0.9, 0.7, 0.7). This means that, for instance, when threshold T1 is applied, all edges with relation probability for inference, rephrase, and conflict below 0.5 are dropped. Each threshold configuration results in a separate set of graphs. Additionally, we create a set of graphs with the thresholds set at 0, thus retaining all of the edges. Edge processing strategies, argmax and probabilities, are then applied to the generated graphs, transforming their edges, to produce the final graph structures that are ready for classification. This process results in eight graphs for each essay: two edge processing strategies (argmax and probabilities) x four threshold strategies (none, T1 , T2 , T3 ). 5.2
Graph Neural Networks
6
Properties of the Constructed Graphs
The resulting graphs based on the original generated essays and their paraphrased variants contain between 8 and 10 edges per node in T1 , while Turkish backtranslation results in a range between 9 and 6 edges per node, and French in even fewer, 6 edges per node. With the higher thresholds, the number of edges per node drops consistently. The detailed dataset statistics are presented in Appendix B. Furthermore, in the original essays, there are very few isolated components: The proportion of graphs containing isolated components ranges between 0% and 5%, depending on the model family and version as well as the threshold. That propor-
Baseline
We consider a text-only baseline that models authorship attribution as a text sequence classification problem. For that purpose, we make use of the Longformer architecture (Beltagy et al., 2020), which extracts semantically rich features from long text input documents. The model is trained for 5 epochs on the four-class classification task, with a learning rate of 1e-5 and a 0.01 weight decay. Each experiment is repeated with 3 random seeds, and 6
Table 2: Macro F1-score results in the S AME -V ERSION setup. None in Threshold column indicates a complete graph; best refers to the best-performing threshold among T1 , T2 , T3 , which are defined in Section 5. Argmax and probs refer to the edge processing strategies defined in Section 4. ‘Orig.’ refers to the non-obfuscated essays, ‘paraphr.’ to the paraphrased, ‘BT-FR’ and ‘BT-TR’ to the backtranslated essays via French and Turkish, respectively. LLM-OWL-AE-I Model
Edge
Thresh.
Orig.
Paraphr.
BT-FR
BT-TR
Orig.
Paraphr.
BT-FR
BT-TR
Text-only baseline Longformer
–
–
0.97
0.63
0.84
0.88
0.96
0.45
0.62
0.69
none best none best
0.66 0.69 0.66 0.67
0.56 0.59 0.56 0.59
0.57 0.58 0.57 0.58
0.61 0.66 0.61 0.66
0.68 0.72 0.68 0.72
0.59 0.63 0.59 0.63
0.61 0.62 0.61 0.62
0.64 0.65 0.64 0.65
none best none best
0.73 0.71 0.74 0.69
0.60 0.59 0.62 0.63
0.67 0.65 0.66 0.65
0.66 0.68 0.67 0.66
0.75 0.75 0.77 0.76
0.61 0.64 0.63 0.61
0.68 0.67 0.67 0.65
0.70 0.70 0.72 0.70
none best none best
0.73 0.72 0.74 0.71
0.62 0.59 0.63 0.60
0.63 0.65 0.67 0.64
0.67 0.66 0.69 0.65
0.78 0.77 0.81 0.80
0.69 0.71 0.72 0.68
0.69 0.68 0.72 0.68
0.72 0.71 0.70 0.71
none best none best
0.74 0.77 0.74 0.77
0.60 0.62 0.64 0.60
0.60 0.67 0.61 0.63
0.63 0.67 0.66 0.67
0.78 0.76 0.80 0.78
0.71 0.68 0.69 0.69
0.60 0.66 0.60 0.68
0.65 0.71 0.66 0.70
Text + Reasoning Structure GCN argmax probs GAT
argmax probs
Graph Transformer
argmax probs
GPS
argmax probs
tion goes up to 8% with paraphrasing and French backtranslation, while Gemma3 and Qwen3 essays backtranslated via Turkish contain the highest proportion of graphs with isolated components (15% and 12% respectively under T3 threshold). The low proportion of graphs with isolated components is likely related to the fact that most of the graphs form a single source-sink path, meaning that these graphs have one starting node, one ending node and all of the intermediary nodes are connected in between – these graphs make up over 90% of all of the graphs for the original essays. Under the obfuscation strategies, this proportion decreases, however, it does not go below 70%. Taken together, these observations indicate that the obtained reasoning graphs are dense and highly interconnected, with paraphrasing and backtranslation affecting the interconnectivity to varying degrees and suggesting that obfuscation strategies do not always perfectly preserve the reasoning structures of the original essays.
7
LLM-OWL-AE-II
in the S AME -V ERSION evaluation setup on the non-obfuscated essays as reported in Table 2, the performance drops around or below 50% in the C ROSS -V ERSION setting (see Table 3). In comparison, a GNN model markedly outperforms Longformer by 19 percentage points when trained on the earlier model versions and evaluated on the later versions, with a more moderate improvement of 4 points when evaluated on the earlier model versions instead. This might be explained by the fact that the later model versions incorporate and expand on the lexical representations generated by the earlier model versions, which would allow the text-only baseline to better generalise to the earlier model essays. However, reasoning features are more consistent across model versions, with less variation, allowing our reasoning graph-based approach to significantly outperform the baseline and show more stable results across the board. Reasoning graph-based approach exhibits more robust performance in most experimental configurations when compared to the text-only baseline. In the S AME -V ERSION evaluation setup, reported in Table 2, obfuscation attacks have a noticeable negative impact on the Longformer performance, with drops ranging 9 to 52 percentage points depending on the obfuscation strategy (with
Results
Reasoning graph-based approach generalises better than the text-only baseline when evaluated on the unseen model versions. While Longformer achieves F1-scores exceeding 90% 7
Table 3: Macro F1-score results in the C ROSS -V ERSION setup. None in Threshold column indicates a complete graph; best refers to the best-performing threshold among T1 , T2 , T3 , which are defined in Section 5. Argmax and probs refer to the edge processing strategies defined in Section 4. ‘Orig.’ refers to the non-obfuscated essays, ‘paraphr.’ to the paraphrased, ‘BT-FR’ and ‘BT-TR’ to the backtranslated essays via French and Turkish, respectively. LLM-OWL-AE-I train / LLM-OWL-AE-II test Model Text-only baseline Longformer
Edge
Thresh.
Orig.
Paraphr.
BT-FR
BT-TR
Orig.
Paraphr.
BT-FR
BT-TR
–
–
0.47
0.30
0.56
0.50
0.49
0.36
0.35
0.37
none best none best
0.57 0.60 0.57 0.60
0.46 0.48 0.46 0.48
0.56 0.56 0.55 0.57
0.55 0.57 0.55 0.58
0.50 0.49 0.50 0.49
0.42 0.41 0.42 0.41
0.43 0.42 0.43 0.42
0.49 0.46 0.49 0.47
none best none best
0.62 0.64 0.63 0.63
0.50 0.53 0.50 0.52
0.61 0.61 0.60 0.60
0.61 0.61 0.59 0.59
0.51 0.53 0.51 0.53
0.43 0.44 0.43 0.42
0.42 0.45 0.43 0.44
0.50 0.50 0.49 0.50
none best none best
0.65 0.66 0.59 0.66
0.54 0.52 0.46 0.53
0.64 0.60 0.59 0.61
0.63 0.62 0.54 0.63
0.52 0.51 0.51 0.53
0.45 0.45 0.47 0.42
0.44 0.44 0.46 0.44
0.49 0.49 0.53 0.49
none best none best
0.59 0.64 0.59 0.64
0.49 0.53 0.49 0.52
0.56 0.62 0.55 0.60
0.57 0.61 0.58 0.62
0.53 0.50 0.53 0.53
0.43 0.43 0.46 0.43
0.46 0.48 0.46 0.47
0.53 0.55 0.57 0.56
Text + Reasoning Structure GCN argmax probs GAT
argmax probs
Graph Transformer
argmax probs
GPS
LLM-OWL-AE-II train / LLM-OWL-AE-I test
argmax probs
paraphrasing having the strongest effect) and model versions (stronger performance degradation is observed in later models). At the same time, while the GNN models perform worse than the baseline on the non-obfuscated data, they exhibit more consistent performance across obfuscation strategies, with the score drops ranging from 10 to 20 points. Furthermore, in the case of later model versions, they achieve up to 27 percentage point improvement over the baseline. In the C ROSS -V ERSION evaluation setup, reported in Table 3, GNNs consistently outperform Longformer under obfuscation attacks with as much as a 24 percentage point improvement under paraphrasing when evaluated with the later model versions and 20 points under Turkish backtranslation when evaluated with the earlier model versions. In terms of GNN architecture, Graph Transformer layer exhibits better performance across most experimental configurations. Moreover, we find that fewer layers improve performance, with 1-layer configurations accounting for about 90% of the best F1-scores across all GNN architectures. This is mainly due to the high connectivity of the generated graphs. Adding more layers means sharing more information across nodes and edges, which, with the structure of our reasoning graphs,
ends up blurring many of the discriminative features. Finally, we could not observe a consistent optimal configuration in terms of edge thresholds and construction strategies. The best performance varies depending on the experiments and GNN architecture used.
8
Conclusion
With this work, we present the first study leveraging argumentative reasoning graphs for LLMauthorship attribution. We propose to go beyond surface-level lexical cues and instead incorporate reasoning information into authorship attribution. An existing argument mining pipeline allows us to construct reasoning graphs, which we then use to train and evaluate GNN models. Our extensive experimentation considers different authorship obfuscation techniques and evaluates across versions of models belonging to different families. We demonstrate that reasoning graph-based approach is more robust to obfuscation attacks than the standard finetuned Transformer-based approach, achieving up to a 27 percentage point increase, and exhibits better generalisation capabilities, indicated by up to a 19 percentage point increase when evaluated on the unseen model versions. 8
Limitations
Malik Altakrori, Thomas Scialom, Benjamin C. M. Fung, and Jackie Chi Kit Cheung. 2022. A multifaceted framework to evaluate evasion, content preservation, and misattribution in authorship obfuscation techniques. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2391–2406, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
We measure generalisation only in conditions in which the GNN model is trained and tested on the texts generated by the different versions of the LLM from the same family, in such a way approximating it across model versions. However, in future work, generalisation under different conditions, including different text genres and domains, should be considered. Another limitation concerns the reasoning graphs. The tasks of argument component detection and argument relation identification remain an open problem in the field of argument mining. Therefore, there is still room for improvement in terms of the quality of the constructed graphs, which could further strengthen the performance of our GNN-based approach.
Navid Ayoobi, Lily Knab, Wen Cheng, David Pantoja, Hamidreza Alikhani, Sylvain Flamant, Jin Kim, and Arjun Mukherjee. 2025. Esperanto: Evaluating synthesized phrases to enhance robustness in ai detection for text origination. In Proceedings of the 36th ACM Conference on Hypertext and Social Media, pages 1–10. Yejin Bang, Delong Chen, Nayeon Lee, and Pascale Fung. 2024. Measuring political bias in large language models: What is said and how it is said. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11142–11159.
Ethical Considerations
Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150.
Our work focuses on the task of LLM authorship attribution, and no human-authored data was produced or used during the experiments. As a result, there are no direct privacy concerns with respect to human authors. We acknowledge the risk of misuse of the authorship attribution systems for the purposes of surveilling LLM usage. Furthermore, the observations that were made regarding the impact of the obfuscation strategies and cross-model-version evaluation can inform evasion strategies. However, we believe that the benefits of such research outweigh the risks and highlight that our corpus and code are intended for research purposes only. All models used to generate data are open-source and accessed locally via Ollama. To the best of our knowledge, their respective licences place no restrictions on output generation for non-commercial research purposes as used in the present work.
Janek Bevendorff, Matti Wiegmann, Emmelie Richter, Martin Potthast, and Benno Stein. 2025. The two paradigms of llm detection: Authorship attribution vs authorship verification. In Findings of the Association for Computational Linguistics: ACL 2025, pages 3762–3787. Maarten Buyl, Alexander Rogiers, Sander Noels, Guillaume Bied, Iris Dominguez-Catena, Edith Heiter, Iman Johary, Alexandru-Cristian Mara, Raphaël Romero, Jefrey Lijffijt, and 1 others. 2026. Large language models reflect the ideology of their creators. npj Artificial Intelligence, 2(1):7. Megha Chakraborty, S.M Towhidul Islam Tonmoy, S M Mehedi Zaman, Shreya Gautam, Tanay Kumar, Krish Sharma, Niyar Barman, Chandan Gupta, Vinija Jain, Aman Chadha, Amit Sheth, and Amitava Das. 2023. Counter Turing test (CT2): AI-generated text detection is not as easy as you may think - introducing AI detectability index (ADI). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2206–2239, Singapore. Association for Computational Linguistics.
References Mervat Abassy, Kareem Elozeiri, Alexander Aziz, Minh Ngoc Ta, Raj Vardhan Tomar, Bimarsha Adhikari, Saad El Dine Ahmed, Yuxia Wang, Osama Mohammed Afzal, Zhuohan Xie, Jonibek Mansurov, Ekaterina Artemova, Vladislav Mikhailov, Rui Xing, Jiahui Geng, Hasan Iqbal, Zain Muhammad Mujahid, Tarek Mahmoud, Akim Tsvigun, and 5 others. 2024. LLM-DetectAIve: a tool for fine-grained machinegenerated text detection. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 336–343.
Artem Chernodub, Oleksiy Oliynyk, Philipp Heidenreich, Alexander Bondarenko, Matthias Hagen, Chris Biemann, and Alexander Panchenko. 2019. TARGER: Neural argument mining at your fingertips. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 195–200, Florence, Italy. Association for Computational Linguistics. Paul Cibier and Jean-Guy Mailly. 2024. Graph convolutional networks and graph attention networks for
9
approximating arguments acceptability. In 10th International Conference on Computational Models of Argument (COMMA 2024). IOS Press.
on opportunities and challenges of large language models for education. Learning and individual differences, 103:102274.
Jonas Dolezal, Sawood Alam, Mark Graham, and Maty Bohacek. 2026. The impact of ai-generated text on the internet. arXiv preprint arXiv:2604.26965.
Thomas N. Kipf and Max Welling. 2017. Semisupervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net.
Liam Dugan, Alyssa Hwang, Filip Trhlík, Andrew Zhu, Josh Magnus Ludan, Hainiu Xu, Daphne Ippolito, and Chris Callison-Burch. 2024. Raid: A shared benchmark for robust evaluation of machinegenerated text detectors. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12463– 12492.
Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. 2023. Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. Advances in neural information processing systems, 36:27469–27500.
Hao Fang, Jiawei Kong, Tianqu Zhuang, Yixiang Qiu, Kuofeng Gao, Bin Chen, Shu-Tao Xia, Yaowei Wang, and Min Zhang. 2025. Your language model can secretly write like humans: Contrastive paraphrase attacks on llm-generated text detectors. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 8596–8613.
Isabelle Kuhlmann and Matthias Thimm. 2019. Using graph convolutional networks for approximate reasoning with abstract argumentation frameworks: A feasibility study. In International Conference on Scalable Uncertainty Management, pages 24–37. Springer. Tharindu Kumarage and Huan Liu. 2023. Neural authorship attribution: Stylometric analysis on large language models. In 2023 International conference on cyber-enabled distributed computing and knowledge discovery (cyberc), pages 51–54. IEEE.
Preesha Gehlot, Anna Rapberger, Fabrizio Russo, and Francesca Toni. 2026. Heterogeneous graph neural networks for assumption-based argumentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 19117–19125.
Lucio La Cava and Andrea Tagarelli. 2025. Openturingbench: An open-model-based benchmark and framework for machine-generated text detection and attribution. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 26666–26682.
Debela Gemechu, Ramon Ruiz-Dolz, Kamila Górska, Somaye Moslemnejad, Eimear Maguire, Dimitra Zografistou, Yohan Jo, John Lawrence, and Chris Reed. 2025. The open argument mining framework. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 318–328, Vienna, Austria. Association for Computational Linguistics.
Linyang Li, Pengyu Wang, Ke Ren, Tianxiang Sun, and Xipeng Qiu. 2023. Origin tracing and detecting of llms. arXiv preprint arXiv:2304.14072.
Ziyi Guo. 2024. Online disinformation and generative language models: Motivations, challenges, and mitigations. In Companion Proceedings of the ACM Web Conference 2024, pages 1174–1177.
Qimai Li, Zhichao Han, and Xiao-ming Wu. 2018. Deeper insights into graph convolutional networks for semi-supervised learning. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1).
Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. 2024. Spotting llms with binoculars: zero-shot detection of machine-generated text. In Proceedings of the 41st International Conference on Machine Learning, pages 17519–17537.
Xiang Li, Zhiyi Yin, Hexiang Tan, Shaoling Jing, Du Su, Yi Cheng, Huawei Shen, and Fei Sun. 2025. PRDetect: Perturbation-robust LLM-generated text detection based on syntax tree. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 8305–8316, Albuquerque, New Mexico. Association for Computational Linguistics.
Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. 2020. spaCy: Industrialstrength Natural Language Processing in Python.
Yikang Liu, Ziyin Zhang, Wanyang Zhang, Shisen Yue, Xiaojing Zhao, Xinyuan Cheng, Yiwen Zhang, and Hai Hu. 2023. Argugpt: evaluating, understanding and identifying argumentative essays generated by gpt models. arXiv preprint arXiv:2304.07666.
Baixiang Huang, Canyu Chen, and Kai Shu. 2025. Authorship attribution in the era of llms: Problems, methodologies, and challenges. ACM SIGKDD Explorations Newsletter, 26(2):21–43.
Dominik Macko, Robert Moro, Adaku Uchendu, Jason Lucas, Michiharu Yamashita, Matúš Pikuliak, Ivan Srba, Thai Le, Dongwon Lee, Jakub Simko, and 1 others. 2023. Multitude: Large-scale multilingual machine-generated text detection benchmark.
Enkelejda Kasneci, Kathrin Seßler, Stefan Küchemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan Günnemann, Eyke Hüllermeier, and 1 others. 2023. Chatgpt for good?
10
In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9960–9987.
Ramon Ruiz-Dolz, Joaquin Taverner, John Lawrence, and Chris Reed. 2024. Nlas-multi: A multilingual corpus of automatically generated natural language argumentation schemes. Data in Brief, 57:111087.
Dominik Macko, Robert Moro, Adaku Uchendu, Ivan Srba, Jason S Lucas, Michiharu Yamashita, Nafis Irtiza Tripto, Dongwon Lee, Jakub Simko, and Maria Bielikova. 2024. Authorship obfuscation in multilingual machine-generated text detection. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 6348–6368.
Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. 2023. Can ai-generated text be reliably detected? arXiv preprint arXiv:2303.11156. Lucas Shen. 2022. LexicalRichness: A small module to compute textual lexical richness.
Lars Malmqvist, Tommy Yuan, Peter Nightingale, and Suresh Manandhar. 2020. Determining the acceptability of abstract arguments with graph convolutional networks. In SAFA@ COMMA, pages 47–56.
Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjing Wang, and Yu Sun. 2021. Masked label prediction: Unified message passing model for semi-supervised classification. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, pages 1548–1554. International Joint Conferences on Artificial Intelligence Organization.
Tiezheng Mao, Osamu Yoshie, Jialing Fu, and Weixin Mao. 2024. Seeing both sides: context-aware heterogeneous graph matching networks for extractingrelated arguments. Neural Computing and Applications, 36(9):4741–4762.
Zhouxing Shi, Yihan Wang, Fan Yin, Xiangning Chen, Kai-Wei Chang, and Cho-Jui Hsieh. 2024. Red teaming language model detectors with language models. Transactions of the Association for Computational Linguistics, 12:174–189.
Philip M McCarthy and Scott Jarvis. 2010. MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods, 42(2):381–392.
Zhen Sun, Zongmin Zhang, Xinyue Shen, Ziyi Zhang, Yule Liu, Michael Backes, Yang Zhang, and Xinlei He. 2025. Are we in the AI-generated text world already? quantifying and monitoring AIGT on social media. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 22975–23005, Vienna, Austria. Association for Computational Linguistics.
Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. 2023. Detectgpt: Zero-shot machine-generated text detection using probability curvature. In International conference on machine learning, pages 24950–24962. PMLR.
Jörg Tiedemann. 2020. The tatoeba translation challenge – realistic data sets for low resource and multilingual MT. In Proceedings of the Fifth Conference on Machine Translation, pages 1174–1182, Online. Association for Computational Linguistics.
Ladislav Rampášek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Dominique Beaini. 2022. Recipe for a general, powerful, scalable graph transformer. Advances in Neural Information Processing Systems, 35:14501–14515.
Jörg Tiedemann and Santhosh Thottingal. 2020. OPUSMT – building open translation services for the world. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pages 479–480, Lisboa, Portugal. European Association for Machine Translation.
Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. Federico Ruggeri, Marco Lippi, and Paolo Torroni. 2021. Tree-constrained graph neural networks for argument mining. arXiv preprint arXiv:2110.00124.
Adaku Uchendu, Thai Le, and Dongwon Lee. 2023. Attribution and obfuscation of neural text authorship: A data mining perspective. ACM SIGKDD Explorations Newsletter, 25(1):1–18.
Ramon Ruiz-Dolz, Jose Alemany, Stella M Heras Barberá, and Ana García-Fornes. 2021. Transformerbased models for automatic identification of argument relations: A cross-domain evaluation. IEEE Intelligent Systems, 36(6):62–70.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2017. Graph attention networks. 6th International Conference on Learning Representations. Ivan Vykopal, Matúš Pikuliak, Ivan Srba, Robert Moro, Dominik Macko, and Maria Bielikova. 2024. Disinformation capabilities of large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14830–14847, Bangkok, Thailand. Association for Computational Linguistics.
Ramon Ruiz-Dolz, Stella Heras, and Ana Garcia. 2023. Automatic debate evaluation with argumentation semantics and natural language argument graph networks. In Proceedings of the 2023 conference on empirical methods in natural language processing, pages 6030–6040.
11
James Liyuan Wang, Ran Li, Junfeng Yang, and Chengzhi Mao. 2024a. Raft: Realistic attacks to fool text detectors. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 16923–16936. Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Osama Mohammed Afzal, Tarek Mahmoud, Giovanni Puccetti, Thomas Arnold, and 1 others. 2024b. M4gtbench: Evaluation benchmark for black-box machinegenerated text detection. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3964– 3992. Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Lidia Sam Chao, and Derek Fai Wong. 2025. A survey on llm-generated text detection: Necessity, methods, and future directions. Computational Linguistics, 51(1):275–338. Eric Xing, Saranya Venkatraman, Thai Le, and Dongwon Lee. 2024. Alison: Fast and effective stylometric authorship obfuscation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19315–19322. Xianjun Yang, Wei Cheng, Yue Wu, Linda Ruth Petzold, William Yang Wang, and Haifeng Chen. 2023. Dna-gpt: Divergent n-gram analysis for training-free detection of gpt-generated text. In The Twelfth International Conference on Learning Representations. Xiao Yu, Yi Yu, Dongrui Liu, Kejiang Chen, Weiming Zhang, Nenghai Yu, and Jing Shao. 2025. Evobench: Towards real-world llm-generated text detection benchmarking for evolving large language models. In Findings of the Association for Computational Linguistics: ACL 2025, pages 14605–14620. Gechuan Zhang, Paul Nulty, and David Lillis. 2023. Argument mining with graph representation learning. In Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law, ICAIL ’23, page 371–380, New York, NY, USA. Association for Computing Machinery. Ying Zhou, Ben He, and Le Sun. 2024a. Humanizing machine-generated content: evading ai-text detection through adversarial attack. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 8427–8437. Ying Zhou, Ben He, and Le Sun. 2024b. Navigating the shadows: Unveiling effective disturbances for modern ai content detectors. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10847–10861.
12
A
Linguistic Feature Statistics
We use LexicalRichness Python module to calculate essay MTLD scores (Shen, 2022). We use spaCy (model en_core_web_sm) (Honnibal et al., 2020) to extract sentence clauses (acl, conj, advcl, ccomp, csubj, discourse, parataxis). The number of extracted clauses per sentence is then used to approximate syntactic complexity. Table 4: Essays Linguistic Features (number of words, MTLD score and measure of syntactic complexity) in the LLM-OWL-AE-I data partition. We report mean and standard deviation. LLM-OWL-AE-I Obfus.
Feature
Gemma3
Qwen3
Llama3
Phi3
Orig.
Words MTLD Syntactic compl.
543 ± 26 160 ± 28 2.33 ± 0.28
481 ± 40 184 ± 31 2.00 ± 0.28
532 ± 39 103 ± 17 2.44 ± 0.43
504 ± 55 180 ± 25 2.22 ± 0.34
Paraphr.
Words MTLD Syntactic compl.
557 ± 42 85 ± 18 2.17 ± 0.34
508 ± 54 96 ± 20 1.96 ± 0.35
495 ± 46 78 ± 13 2.30 ± 0.41
483 ± 61 94 ± 18 2.06 ± 0.36
BT-FR
Words MTLD Syntactic compl.
367 ± 40 127 ± 26 2.57 ± 0.61
327 ± 40 142 ± 27 2.27 ± 0.52
375 ± 48 92 ± 17 2.93 ± 1.06
391 ± 57 135 ± 22 2.58 ± 0.65
BT-TR
Words MTLD Syntactic compl.
392 ± 31 134 ± 25 2.04 ± 0.31
384 ± 33 147 ± 26 1.85 ± 0.31
437 ± 42 94 ± 16 2.28 ± 0.41
481 ± 56 148 ± 23 2.18 ± 0.38
Table 5: Essay linguistic Features (number of words, MTLD score and measure of syntactic complexity) in the LLM-OWL-AE-II data partition. We report mean and standard deviation. LLM-OWL-AE-II Obfus.
Feature
Gemma4
Qwen3.5
Llama4
Phi4
Orig.
Words MTLD Syntactic compl.
466 ± 20 129 ± 26 1.91 ± 0.28
476 ± 34 185 ± 37 1.85 ± 0.33
477 ± 37 88 ± 15 2.11 ± 0.39
487 ± 39 196 ± 28 2.07 ± 0.30
Paraphr.
Words MTLD Syntactic compl.
465 ± 36 79 ± 17 1.88 ± 0.34
499 ± 46 90 ± 21 1.78 ± 0.33
447 ± 45 72 ± 13 2.04 ± 0.41
494 ± 50 94 ± 19 1.99 ± 0.38
BT-FR
Words MTLD Syntactic compl.
349 ± 37 113 ± 26 2.01 ± 0.46
322 ± 41 146 ± 32 1.95 ± 0.63
358 ± 44 83 ± 15 2.58 ± 0.77
356 ± 41 143 ± 24 2.46 ± 0.63
BT-TR
Words MTLD Syntactic compl.
388 ± 24 120 ± 24 1.74 ± 0.29
345 ± 40 152 ± 33 1.64 ± 0.31
438 ± 35 84 ± 14 2.09 ± 0.42
456 ± 41 155 ± 24 1.98 ± 0.31
13
(a) Mean number of words per essay.
(b) Mean MTLD score per essay.
(c) Mean syntactic complexity score per sentence per essay.
Figure 2: Essay linguistic features. Violin plots on the left are for the LLM-OWL-AE-I data partition; violin plots on the right for LLM-OWL-AE-II data partition.
14
B
Reasoning Graph Statistics Table 6: Average number of nodes and edges per essay across different obfuscation strategies and thresholds. LLM-OWL-AE-I Obfus.
Orig.
Thresh.
Gemma3
Qwen3
Llama3
Phi3
Gemma4
Qwen3.5
Llama4
Phi4
Nodes
– None T1 T2 T3
25.08 304.35 258.24 225.21 171.68
23.63 270.24 242.52 215.02 169.13
21.79 228.98 214.98 188.03 149.37
20.83 210.58 191.92 169.42 133.52
20.4 199.05 176.82 157.85 125.25
23.34 264.68 222.01 199.29 160.58
21.6 225.58 210.7 180.89 142.74
22.84 252.33 229.61 204.88 163.39
– None T1 T2 T3
23.75 273.37 216.78 187.85 139.93
22.1 236.9 198.88 173.89 132.86
20.75 208.19 184.1 158.82 122.98
19.59 185.9 159.57 138.28 106.32
19.49 182.26 152.13 133.52 102.45
22.1 237.18 186.31 164.78 128.40
20.36 200.88 176.91 149.84 115.26
21.42 222.79 189.02 165.07 127.41
– None T1 T2 T3
15.24 111.35 92.14 79.86 59.81
14.18 96.52 84.33 73.79 55.50
13.62 89.80 83.89 72.21 54.44
13.72 90.88 82.85 72.37 55.33
14.68 102.80 87.74 77.05 58.71
14.65 104.42 83.83 74.16 57.62
14.09 95.40 88.7 74.11 56.01
14.23 97.25 87.84 77.45 59.25
– None T1 T2 T3
19.16 176.72 137.17 115.01 83.64
19.31 179.60 151.09 129.34 97.67
19.17 177.28 161.11 137.5 105.50
19.96 193.29 170.1 148.35 115.92
18.24 158.64 130.36 112.3 84.73
17.83 155.13 118.93 103.14 78.70
20.31 199.43 181.84 154.22 120.31
21.56 224.96 195.52 170.63 131.58
Edges
Nodes Paraphr.
Edges
Nodes BT-FR
Edges
Nodes BT-TR
LLM-OWL-AE-II
Prop.
Edges
(a) Original essays.
(b) Paraphrased essays.
(c) Backtranslated essays (French).
(d) Backtranslated essays (Turkish)
Figure 3: Ratio of the average number of edges per node across different thresholds.
15
(a) Original essays.
(b) Paraphrased essays.
(c) Backtranslated essays (French).
(d) Backtranslated essays (Turkish)
Figure 4: Proportion of isolated components in graphs.
(a) Original essays.
(b) Paraphrased essays.
(c) Backtranslated essays (French).
(d) Backtranslated essays (Turkish)
Figure 5: Proportion of the graphs forming a single source-sink path.
16
C
Extended Experimental Results
Extended experimental results with all threshold configurations for GNN architectures are included. While we evaluate 1- to 7-layer GNN architectures, the best results were consistently achieved with 1-3 layers, so we do not report results for models with more than three layers. Table 7: Macro F1-score results in the S AME -V ERSION setup, ∗ is used to indicate when the best result is achieved with a 2-layer GNN, † is used for a 3-layer GNN, and the rest of the results are reported for a 1-layer GNN. None in Threshold column indicates a complete graph. Argmax and probs refer to the edge processing strategies defined in Section 4. ‘Orig.’ refers to the non-obfuscated essays, ‘paraphr.’ to the paraphrased, ‘BT-FR’ and ‘BT-TR’ to the backtranslated essays via French and Turkish, respectively. LLM-OWL-AE-I
LLM-OWL-AE-II
Model
Edge
Thresh.
Orig.
Paraphr.
BT-FR
BT-TR
Orig.
Paraphr.
BT-FR
BT-TR
Text-only baseline Longformer
–
–
0.97
0.63
0.84
0.88
0.96
0.45
0.62
0.69
None T1 T2 T3 None T1 T2 T3
0.66 0.67 0.69 0.67 0.66 0.67 0.67 0.67
0.56 0.52 0.51 0.59 0.56 0.52 0.50 0.59
0.57 0.55 0.58 0.58 0.57 0.55 0.55 0.58
0.61 0.62 0.62 0.66 0.61 0.62 0.59 0.66
0.68 0.69 0.70 0.72 0.68 0.69 0.70 0.72
0.59 0.61 0.60 0.63 0.59 0.61 0.60 0.63
0.61 0.62 0.61 0.59 0.61 0.62 0.61 0.59
0.64 0.64 0.65 0.64 0.64 0.65∗ 0.65 0.64
None T1 T2 T3 None T1 T2 T3
0.73 0.71 0.70 0.70 0.74 0.69 0.69∗ 0.67
0.60 0.59 0.58 0.59 0.62 0.58 0.58 0.63
0.67 0.64 0.65 0.64 0.66 0.64 0.63 0.65
0.66 0.66 0.66 0.68 0.67 0.65 0.64 0.66
0.75 0.72 0.75 0.73 0.77 0.72 0.76 0.73
0.61 0.63 0.64 0.61 0.63 0.60 0.60 0.61
0.68 0.66 0.67 0.65 0.67 0.65 0.65 0.63
0.70 0.68 0.70 0.68 0.72 0.69 0.70 0.69
None T1 T2 T3 None T1 T2 T3
0.73 0.70 0.72∗ 0.69 0.74 0.71 0.70 0.69
0.62 0.58 0.59 0.55 0.63 0.60∗ 0.58 0.56
0.63 0.65∗ 0.62∗ 0.57∗ 0.67 0.64∗ 0.60 0.59
0.67 0.66∗ 0.64∗ 0.61∗ 0.69 0.65 0.64 0.62
0.78 0.77 0.77 0.77 0.81 0.80 0.78 0.78
0.69 0.71 0.69 0.69 0.72 0.68∗ 0.63 0.68
0.69 0.68 0.67 0.66 0.72 0.68 0.68 0.65
0.72 0.70 0.71 0.70 0.70 0.71 0.71 0.69
None T1 T2 T3 None T1 T2 T3
0.74∗ 0.75∗ 0.77 0.75 0.74† 0.77 0.76 0.73
0.60 0.59∗ 0.59 0.62 0.64∗ 0.59 0.60 0.60∗
0.60 0.59 0.61 0.67 0.61 0.62 0.61 0.63
0.63 0.61 0.67 0.67 0.66 0.67 0.67 0.65
0.78∗ 0.76 0.76∗ 0.76 0.80∗ 0.76 0.78† 0.77∗
0.71∗ 0.68∗ 0.67† 0.66 0.69† 0.67∗ 0.66 0.69†
0.60 0.59 0.66 0.65 0.60 0.68 0.65 0.65
0.65 0.66 0.67 0.71 0.66 0.70 0.68 0.67
Text + Reasoning Structure GCN argmax
probs
GAT
argmax
probs
Graph Transformer
argmax
probs
GPS
argmax
probs
17
Table 8: Macro F1-score results in the C ROSS -V ERSION setup for 1-layer GNN configurations, ∗ is used to indicate when the best result is achieved with a 2-layer GNN, † is used for a 3-layer GNN, and the rest of the results are reported for a 1-layer GNN. None in Threshold column indicates a complete graph. Argmax and probs refer to the edge processing strategies defined in Section 4. ‘Orig.’ refers to the non-obfuscated essays, ‘paraphr.’ to the paraphrased, ‘BT-FR’ and ‘BT-TR’ to the backtranslated essays via French and Turkish, respectively. LLM-OWL-AE-I train / LLM-OWL-AE-II test Model Text-only baseline Longformer
Edge
Thresh.
Orig.
Paraphr.
BT-FR
BT-TR
Orig.
Paraphr.
BT-FR
BT-TR
–
–
0.47
0.30
0.56
0.50
0.49
0.36
0.35
0.37
None T1 T2 T3 None T1 T2 T3
0.57 0.60 0.59 0.58 0.57 0.60 0.60 0.58
0.46 0.46 0.46 0.48 0.46 0.46 0.46 0.48
0.56 0.56 0.56 0.55 0.55 0.56 0.57 0.55
0.55 0.56 0.57 0.55 0.55 0.56 0.58 0.55
0.50 0.49 0.46 0.48 0.50 0.49 0.46 0.48
0.42 0.41 0.40∗ 0.40 0.42 0.41 0.40∗ 0.40
0.43 0.42 0.40∗ 0.39 0.43 0.42 0.40∗ 0.39
0.49 0.46 0.45∗ 0.46 0.49 0.46 0.45∗ 0.47
None T1 T2 T3 None T1 T2 T3
0.62 0.64 0.60 0.61 0.63 0.62 0.63 0.59
0.50 0.53 0.51 0.53 0.50 0.49 0.52 0.51
0.61 0.61 0.58 0.58 0.60 0.59 0.60 0.60
0.61 0.61 0.58 0.59 0.59 0.58 0.59 0.57
0.51 0.53 0.50 0.47 0.51 0.47 0.50 0.53
0.43 0.44 0.41 0.38∗ 0.43∗ 0.39 0.40 0.42
0.42 0.45 0.42 0.39† 0.43∗ 0.40† 0.42 0.44
0.50 0.50 0.48 0.43 0.49 0.46 0.48 0.50
None T1 T2 T3 None T1 T2 T3
0.65 0.63 0.63 0.66 0.59 0.64 0.66 0.63
0.54 0.52 0.52 0.51 0.46 0.53 0.52 0.49
0.64 0.60 0.60 0.59 0.59 0.61 0.61 0.60
0.63 0.60 0.61 0.62 0.54 0.61 0.63 0.60
0.52 0.51 0.49 0.51 0.51 0.53 0.50 0.50
0.45 0.45 0.42 0.42 0.47 0.42 0.41∗ 0.42
0.44 0.44 0.42 0.43 0.46 0.44 0.42 0.43
0.49 0.48 0.48 0.49 0.53 0.49 0.48 0.48
None T1 T2 T3 None T1 T2 T3
0.59 0.64 0.64 0.63 0.59 0.64 0.63 0.63
0.49 0.51 0.53 0.52 0.49 0.51 0.51 0.52
0.56 0.61 0.60 0.62 0.55 0.60 0.59 0.60
0.57 0.61 0.61 0.60 0.58 0.62 0.59 0.60
0.53 0.50 0.48 0.50∗ 0.53 0.53 0.51 0.52
0.43 0.41 0.41 0.43 0.46 0.43 0.41∗ 0.43
0.46 0.43 0.48 0.47 0.46 0.47 0.45 0.45
0.53 0.55∗ 0.55∗ 0.55∗ 0.57 0.56∗ 0.55 0.53∗
Text + Reasoning Structure GCN argmax
probs
GAT
argmax
probs
Graph Transformer
argmax
probs
GPS
LLM-OWL-AE-II train / LLM-OWL-AE-I test
argmax
probs
18
D
Computational Resources
Backtranslated via French In addition to ethical concerns, animal tests are also unreliable and often produce false or misleading results.The animals have physiological and biological systems different from those of humans, which means that the results of animal tests may not apply to humans. For example, a drug may be safe in animals but toxic in humans, or vice versa. This has led to many cases where drugs have been approved on the basis of animal tests, only to be removed later from the market due to adverse effects in humans.
For the data generation, we used the following GPU resources: NVIDIA GeForce RTX 4090 24 GB and NVIDIA GeForce RTX 5090 32 GB. The RTX 4090 was additionally used for the Longformer experiments. RTX 5090 was used for the oAMF argument mining pipeline (module ARIR) and GNN experiments. The other modules of the pipeline, the linguistic features, and the graph statistics were computed without GPU acceleration.
E
Backtranslated via Turkish In addition to ethical concerns, animal testing is unreliable and often produces inaccurate or misleading results. Animals have different physiological and biological systems than humans, which means that the results of animal testing may not apply to humans. For example, a drug may be safe in animals, but it may be toxic in humans or vice versa. This has led to many cases where drugs are more accurate simulations than animal testing and subsequently withdrawn from the market due to adverse effects on human cells.
Examples of Original and Obfuscated Essays
We include an example of a paragraph from an essay generated by Llama3.3 on the topic of Animal Rights (stance: against) as well as its obfuscated versions. The texts are copied verbatim from the dataset. The examples demonstrate that the obfuscated paragraphs closely follow the original content in meaning and retain fluency, while exhibiting surface-level lexical and syntactic differences. Original essay In addition to the ethical concerns, animal testing is also unreliable and often produces false or misleading results. Animals have different physiological and biological systems than humans, which means that the results of animal tests may not be applicable to humans. For instance, a drug may be safe in animals but toxic in humans, or vice versa. This has led to many cases where drugs have been approved based on animal testing, only to be later withdrawn from the market due to adverse effects in humans. Moreover, alternative methods such as computer simulations, cell cultures, and human clinical trials are now available, which can provide more accurate and reliable results than animal testing. Paraphrased Animals have different physiological and biological systems from humans, which means that the results of animal experiments may not be applicable to humans. A drug can be safe for animals but toxic for humans, or vice versa. In addition to being unreliable, animal experiments often produce misleading results. Besides, the methods of the computer simulation, the cell culture, and the clinical trial have all been developed. They are more reliable and accurate than animal testing. It is not uncommon for medicines approved based on animal experiments to later be withdrawn from the market because of side effects in humans. 19