arXiv CSopen access

Large-Language-Models-as-a-Judge in Theory-Agnostic Adaptive Metric-Alignment for Prototypical Networks in Personality Recognition

1

Large-Language-Models-as-a-Judge in Theory-Agnostic Adaptive Metric-Alignment for Prototypical Networks in Personality Recognition

arXiv:2607.08374v1 [cs.CL] 9 Jul 2026

Jing Jie Tan∗ , Ban-Hoe Kwan∗ , Danny Wee-Kiat Ng∗ , Yan-Chai Hum∗ , Shih-Yu Lo † , Po-An Chen ‡ , Noriyuki Kawarazaki§ , Kosuke Takano§ , Anissa Mokraoui¶ ∗ Department of Mechatronics and Biomedical Engineering, Lee Kong Chian Faculty of Engineering and Science, Universiti Tunku Abdul Rahman, Malaysia † Institute of Communication Studies, National Yang Ming Chiao Tung University, Taiwan ‡ Institute of Information Management, National Yang Ming Chiao Tung University, Taiwan § Faculty of Information Technology, Kanagawa Institute of Technology, Japan ¶ Laboratoire de Traitement et Transport de l’Information, Université Sorbonne Paris Nord, France Email: [email protected], {kwanbh, ngwk, humyc}@utar.edu.my,{shihyulo, poanchen}@nycu.edu.tw,{kawara@rm, takano@ic}.kanagawa-it.ac.jp, [email protected]

Abstract—Personality recognition has traditionally been constrained by theory-dependent formulations, where models are trained to fit predefined psychological taxonomies rather than uncovering shared underlying behavioral structure. This limits generalization, as personality itself is better understood as theory-invariant, emerging from stable psychological patterns that should manifest consistently across different frameworks, while existing annotations reflect only partial and sometimes inconsistent views of the same latent traits. In this work, we introduce JAM ((J)udge for (A)daptive (M)etric-Alignment), a theory-agnostic framework that shifts learning from adapting to predefined personality theories toward discovering unified latent “pseudo-facets” that capture shared psychological structure. Rather than constraining the model to any personality taxonomy during training or inference, the framework learns generalizable psychological representations and can infer an individual’s latent psychological profile directly from the textual samples, without requiring theory-specific labels. JAM achieves this through an Attention-Pooled Graph Prototypical Network that learns structured representations via clustering in embedding space, together with a Cross-Theory Harmonization (CTH) approach that integrates (i) Human-Guided Linkage and (ii) MachineInduced Consensus to unify heterogeneous datasets without relying on predefined labels. To further improve robustness and data quality, we incorporate an LLM-as-a-Judge mechanism operating in two configurations, (i) LLM-before-the-loop and (ii) LLM-in-the-loop which identifies ambiguous, mislabeled, and boundary samples to guide adaptive metric learning. Experiments on Essays and Kaggle personality datasets show that JAM improves cross-framework generalization and performance, establishing a strong step toward theory-agnostic personality inference and supporting low-resource personality theories. The related code repository, model weights, and artifacts are available at https://research.jingjietan.com/JAM. Index Terms—Personality Classification, Large Language Models (LLMs), N-Shot Prompting, Prototypical Networks, FineTuning, Natural Language Understanding

I. I NTRODUCTION Personality recognition has become increasingly important, especially in recommendation systems [1]. By understanding

user personalities, these systems can provide personalized suggestions, enhancing user satisfaction and trust. Understanding user personality is crucial for delivering a superior user experience, making this an important area of study [2]. By tailoring interactions and recommendations to individual personality traits, AI systems and robots can achieve higher levels of personalization, leading to increased user satisfaction and trust [3]. Furthermore, incorporating personality traits into recommendation algorithms can help address issues such as the cold start and data sparsity problems, resulting in more accurate and user-centric recommendations [4], [5]. Additionally, this approach can contribute to explainable AI by providing clearer explanations for the recommendations made. Despite these advancements, current personality-recognition models remain constrained by several fundamental limitations. Most approaches are built around specific psychological theories, such as the Big-5 or MBTI, limiting their ability to generalize across datasets and cultural contexts. This dependence is further exacerbated by the scarcity of large-scale annotated data. The widely used myPersonality dataset [6], which originally contained data from millions of users, was discontinued due to privacy concerns, leaving only relatively small public datasets. Moreover, existing methods treat personality recognition as a static classification task, without leveraging psychological insights, limiting their generalizability. These limitations motivate a theory-agnostic framework that eliminates dependence on predefined personality taxonomies during both training and inference. Instead of learning theory-specific representations, the model discovers latent pseudo-facets that capture underlying psychological structure across heterogeneous data sources. At inference time, the model directly infers an individual’s latent psychological profile from behavioral or textual samples without requiring theory-specific labels or adherence to a particular personality framework. This design enables the integration of heterogeneous datasets while improving generalization across existing

2

personality theories. A. Contributions In this study, we make the following key contributions: 1) We incorporate a psychology-based methodology for integrating datasets constructed under different personality theories and generalize the model through human-guided linkage, supporting low-resource personality theory. 2) We enhance the prototypical network by incorporating machine-induced consensus for pseudo-facet construction, improving cross-theory harmonization and mitigating class imbalance, leading to improved performance. 3) We investigate the LLM-as-a-Judge mechanism, assessing its comparative effectiveness when applied during the prelearning stage (LLM-before-the-loop) versus within the learning process (LLM-in-the-loop) for filtering ambiguous or noisy personality-related text samples. II. L ITERATURE R EVIEW A. Personality Theory Recent studies have demonstrated that humans express their personality through language use, providing a rich source of data for personality recognition [7], [8], [9], [10]. The rise of social media platforms has provided a rich source of data to understand how particular individuals often leave behind a personality footprint through their online activities [7]. This creates a significant opportunity to extract personality information from such indirect background data, which is particularly beneficial for initializing personality-aware functionalities in new robots or applications. There are two primary types of assessments to measure, quantify, and classify personality traits [11], [12]: • Self-report tests: These tests require test-takers to understand the provided statements and evaluate how well they describe themselves. The results can be quantitatively standardized, ensuring high reliability and validity [13]. • Projective tests: These tests involve asking test-takers to provide their interpretations of scenes, scenarios, or objects [14]. This type of test considers various aspects, such as tone, message, or body language, making it more capable of addressing potential issues like misinterpretation of questions or dishonesty from the test-taker [15]. Various personality theories have been proposed to model personality using distinct sets of dimensions [16], [17]. For instance, the Myers–Briggs Type Indicator (MBTI), sometimes described in relation to 4 dichotomies, categorizes personality across paired preferences [18], [19]. The Big-5 framework (OCEAN) defines five broad traits: Openness to Experience, Conscientiousness, Extraversion, Agreeableness, and Neuroticism [20], [21]. The HEXACO model extends this structure by adding Honesty–Humility to Emotionality, Extraversion, Agreeableness, Conscientiousness, and Openness [22], [23]. Table I summarizes the correspondence among these three major frameworks, offering a general comparison of their dimensions. Basically, facets are commonly regarded as the lower-level constructs or descriptions that collectively define

a broader personality dimension or trait. They provide a more granular representation of personality by capturing specific behavioral and psychological tendencies within each dimension. The terminology can be summarized as in Fig. 1. Theory/ Model The model is built based on a personality theory which analyses a person through an assessment which is a test, questionnaire, or survey. Each of the personality types in the model is contributed by all dimensions of the model. The number of personality types is based on the number of classes in the model, commonly formulated as:

Dimension/ Factor/ Class The number of dimensions is determined by the respective personality theory/model used.

Traits

Facets/ Description

Each dimension has either: Single trait

evaluate using a level, score or scale

Paired trait

evaluate by categorising into 2 polar opposites

The facet is the description of respective traits

Fig. 1. Overview of the relationship, terminology, and theoretical structure of personality assessment models, illustrating how theories define dimensions, traits, facets, and personality types.

These mappings should be interpreted cautiously: they reflect partial conceptual overlap rather than equivalence. For example, Big-5 Openness includes facets such as curiosity and preference for novelty, whereas MBTI’s Sensing–Intuition dimension distinguishes concrete, detail-oriented perception from abstract, future-oriented thinking. Similarly, related constructs are distributed differently across frameworks; emotional reactivity in Big-5 Neuroticism only loosely corresponds to the MBTI Thinking–Feeling dimension, which primarily reflects decision-making style rather than affective stability. This highlights a limitation: personality models are human-constructed abstractions rather than directly observable ground truth categories. Consequently, their use in machine learning can introduce systematic noise and inconsistency, especially at the facet level where definitions differ across studies. In practice, available datasets seldom provide sufficiently fine-grained and consistently annotated signals to support reliable bottom-up learning of stable psychological components, not only due to annotation cost but also because personality is inherently composite and context-dependent. To our knowledge, no prior work has directly addressed this mismatch between psychologically defined trait hierarchies and their operationalization in data-driven settings, or proposed effective strategies to mitigate the resulting label ambiguity. B. Related Algorithms Personality recognition has evolved alongside advances in natural language processing and representation learning [24]. Early research primarily focused on developing task-specific models for predicting personality labels defined by a particular framework, such as the Big-5 or MBTI. As language models became increasingly capable of capturing semantic and contextual information, researchers began adopting pretrained Transformer architectures as general-purpose representations for personality-related tasks. More recently, large language models have enabled prompting-based inference, reducing reliance on framework-specific training procedures and expanding the possibility of transferring personality knowledge across different theoretical formulations.

3

TABLE I M APPING THE DIMENSIONS OF THREE PROMINENT PERSONALITY FRAMEWORKS : B IG -5, MBTI, AND HEXACO.

Symbol O C E A N H

Big-5 Openness to Experience (O) Conscientiousness (C) Extraversion (E) Agreeableness (A) Neuroticism (N) -

MBTI Sensors (S)-Intuitive (N) Perceivers (P)-Judgers (J) Introverts (I)-Extroverts (E) Thinkers (T)-Feelers (F) -

HEXACO Openness (O) Conscientiousness (C) Extraversion (X) Agreeableness (A) Emotionality (E) Honesty-Humility (H)

General Description Creativity, curiosity, novelty Organise, responsibility, discipline Sociability, energy, assertiveness Empathy, kindness, cooperativeness Sensitivity, anxiety, hostility Sincerity, fairness, modesty

Note: The correlations presented are intended as reference points, aligning these theories to the widely recognized Big-5 framework. This alignment supports evaluating model performance across diverse personality theories, fostering comparative analysis and deeper insights into their unique characteristics.

To contextualize these developments, this section first reviews the language-model foundations underlying modern personality recognition, including (i) sentence transformer language models (encoder-only models) and (ii) autoregressive transformer large language models (decoder-only models). It then surveys (iii) prior personality recognition methods built upon these representations, followed by (iv) recent generative approaches that increasingly support flexible personality inference. Together, these developments provide the foundation for investigating personality recognition beyond a single predefined personality theory. 1) Sentence Transformer Language Model (Encoder-only Model): These models are trained on large corpora of text data, enabling them to capture complex linguistic patterns and relationships [25], [26], [27], [28], [29]. The produced embeddings represent the semantic meaning of the input text, facilitating downstream tasks such as text classification, clustering, and similarity analysis. MiniLM [25] compresses large Transformer models through deep self-attention distillation, where a small student model mimics the self-attention of the teacher’s last layer. It retains over 99% accuracy on SQuAD 2.0 and GLUE benchmarks while reducing Transformer parameters and computation by 50%. Moreover, MPNet [26] integrates permuted language modeling (PLM) and auxiliary position information, improving dependency modeling and reducing position discrepancy. It surpasses BERT, XLNet, and RoBERTa across GLUE and SQuAD. Furthermore, SentenceT5 [27] explores sentence embeddings from T5 models, introducing an encoder-only approach that outperforms SentenceBERT and SimCSE in STS tasks. Scaling T5 to billions of parameters further enhances performance, setting new state-ofthe-art results for sentence embeddings. Given the efficiency and strong semantic representation capabilities of these models, we select a relatively lightweight yet effective encoderonly model tailored for the personality domain, ensuring optimal performance while balancing computational efficiency. 2) Autoregressive Transformer Large Language Model (Decoder-only Model): Generative Pretrained Transformer 3 (GPT-3) was the first to demonstrate effectiveness in Natural Language Understanding (NLU) through its few-shot learning capability, facilitated by techniques such as Chainof-Thought (CoT) prompting. This family of decoder-only models enhances reasoning, coherence, and adaptability in tasks such as emotion detection, sentiment analysis, and personality recognition [30], [31], [32]. The open-source LLaMA

model introduced a family of models with varying parameter sizes, achieving competitive performance while maintaining efficiency in training and inference [33]. Following this, Mistral AI released the Mistral 7B model, which, despite its relatively smaller size, demonstrated remarkable capabilities in code generation, mathematics, and reasoning tasks, benefiting from an extended context window of 128k tokens [34]. Qwen models have been proposed as strong open-weight alternatives, exhibiting robust performance across reasoning, multilingual understanding tasks [35]. OpenAI’s GPT-4 further pushed the boundaries by introducing multimodal capabilities, allowing the model to process both text and images, thereby enhancing its applicability across diverse domains [36]. Later, reasoning models further extended their capabilities in scientific and mathematical contributions. OpenAI’s o1 model introduced a novel approach by generating extended chains of thought before arriving at a final answer, thereby improving the model’s reasoning depth and accuracy [37]. DeepSeek’s R1 model emerged as a notable open-source contribution, achieving performance comparable to OpenAI’s o1 model across mathematics, coding, and reasoning tasks [38] with a relatively low computational complexity. The R1 model employs a unique training methodology that emphasizes reinforcement learning to enhance its reasoning capabilities. These developments underscore a growing trend toward integrating advanced reasoning processes within large language models, aiming to improve their problem-solving abilities and decisionmaking processes. 3) Prior Works for Personality Recognition: Prior research has consistently highlighted the importance of large, highquality datasets for training machine learning models that generalize effectively [39], [40]. However, in privacy-sensitive domains, the development and utilization of such datasets are often constrained by regulatory and ethical considerations. In contrast, other research areas are better positioned to achieve scale by aggregating or integrating existing public datasets. A frequently cited example is the myPersonality Project [6], once a prominent open dataset for computational personality research, which was discontinued in 2018 due to increasing challenges related to regulatory compliance, controlled data access, and governance obligations. In personality recognition, we are limited by personality theory; hence, few researchers are tackling algorithmic improvements, including architecture, feature exploration, graph neural networks, loss reweighted contributions, etc.

4

Building on the importance of data and representation, prior work has explored a range of modeling approaches for personality classification from text. Early advances include BiLSTM-based models [41], which leverage bidirectional context to outperform traditional machine learning methods such as SVM, RF, and DT, highlighting the role of sequential modeling and hyperparameter tuning. To further enhance representation learning, [42] proposes Personality2vec, which integrates semantic, linguistic, and structural information from social networks through biased random walks and skip-gram modeling, demonstrating robustness particularly in data-scarce settings. After that, [40] further explored the dataset splitting and proposed a stratification feature, as well as focal loss for reweighting the contribution in the neural network, and analyzed the evaluation metrices. Subsequent studies have focused on enriching deep contextual representations with complementary linguistic and structural information. For example, a hybrid Transformer–BLSTM framework [43] integrates psycholinguistic features with attention mechanisms to improve both predictive performance and interpretability. Graph-based methods have also emerged as an effective direction. KGrAt-Net [44] leverages knowledge graph attention over DBpedia entities to model richer semantic relationships, while TranSentGAT [45] combines BERT with sentiment knowledge and graph attention to enhance contextual representations and improve personality prediction. Nonetheless, these advances in modeling and feature integration do not resolve the core bottleneck of limited and fragmented data. Performance in personality recognition remains constrained by dataset scale, diversity, and ecological validity under strict privacy and governance requirements. From this perspective, expanding datasets in a cost-efficient and flexible manner is essential to overcoming current theoretical and empirical limits. However, collecting such data is often difficult, particularly for low-resource settings, due to time, cost, and regulatory constraints, while cross-institutional approaches such as federated learning remain limited in practice (to the best of our knowledge), partly because of heterogeneous theoretical frameworks and incompatible data assumptions. These challenges motivate a shift toward more theory-agnostic approaches. 4) Generative Approaches to Personality Recognition: Recent work on generative AI for personality inference has explored both data-centric and architecture-centric improvements. Early studies focus on data augmentation and heterogeneous graph-based models for dialogue understanding. For instance, Wu et al. [46] introduce personality trait interpolation to generate synthetic training data and propose HC-GNN to capture both inter- and intra-speaker dependencies, improving conversational personality recognition. Similarly, SemiPerGCN [47] adopts a semi-supervised framework with multiview graph augmentation and heterogeneous graph construction, addressing limited labeled data and enhancing robustness in low-resource settings. Collectively, these approaches reflect a transition from sequential modeling toward graph-based and hybrid architectures, with increasing emphasis on representation richness and data efficiency. More recent studies have investigated large language mod-

els as direct inference engines for personality recognition. ChatGPT has been shown to exhibit strong zero-shot capability and partial interpretability in personality prediction tasks [48]. Related work integrates emotional knowledge with structured prompting strategies for trait inference [32], while PICEPR [49] not only proposes embedding-based knowledge elicitation, but it also demonstrates that modular prompting can improve classification performance. Compared with traditional approaches that require taskspecific feature engineering or model training, these methods leverage in-context learning, enabling models to infer personality labels directly from instructions, label descriptions, or a small number of exemplars. Consequently, prompting-based approaches move toward a more theory-agnostic paradigm, as the same pretrained model can be adapted to different personality frameworks by modifying the prompt and label definitions rather than redesigning or retraining the prediction architecture. While the provided labels may still originate from a specific theory, the underlying inference mechanism is not inherently bound to a specific personality theory. Through in-context learning, the same model can perform personality inference in few-shot settings using a small number of labeled examples or in zero-shot settings using only trait descriptions and task instructions, without requiring model retraining. However, these approaches also introduce important limitations. First, decoder-only large language models rely on inference-time reasoning, which is computationally expensive and often impractical for large-scale deployment. Second, they do not produce explicit intermediate representations of psychological states, limiting their usefulness for downstream modeling of structured cognitive or behavioral processes. More critically, since these models are trained on internet-scale corpora, there is a non-trivial risk of implicit exposure to similar evaluation data, weakening the assumption of strict zero-shot generalization and introducing potential dataset leakage concerns. These issues suggest that encoder-based models remain necessary for stable and controllable representation learning in personality recognition systems. III. M ETHODOLOGIES Fig. 2 illustrates the proposed JAM architecture, which targets tailoring generalization across datasets annotated under different psychological theories, thus achieving a theoryagnostic model. A. Datasets We utilized Tan’s train-validation-test split algorithm for the standard Essays and Kaggle datasets [40]. This stratified algorithm ensures an even distribution of personality traits across all dimensions within each split, promoting fairness in comparison and validating the effectiveness of our model. Table II summarizes the datasets used in this study. Note that, due to the nature of the JAM’s prototypical learning, the training sets are resampled into support and query sets for each episode in training; while evaluating, the existing validation set will serve as the support set, and the existing test set will be the query set.

5

Dataset

B. LLM-in-the-loop

Truth Label

O=1 C=0 E=1 A=0 N=1

Decision

Human-Guided Linkage

Essays Dataset

O

S/N

C

P/J

E

↔

A N

Kaggle Dataset

Large Language Model

B.3

I can't wait until friday because I am...

Possible/ Impossible/ Ambiguous?

Continue No

Potential Problematic Dataset

Collect

I/E T/F

-

remove/ remain

sent to

Training

Model Architecture

B.2

Language Embeddings Backbone

B.1

Encoder Layer

Big-5 (OCEAN)

I can't wait until friday because I am...

A. LLM-before-the-loop A.2

Large Language Model

1 5

2|5

5

5

2

2

e

c|d c

Embedding

4

3 6

Contrastive Loss

Meta-Training

Cross Entropy Loss

FC Adapter Layer

Embedding

...

d 3

3 b

Cross Entropy Loss

FC

Testing

Attention Pooling

O=1 C=0 E=1 A=0 N=0

Possible/ Impossible/ Ambiguous?

Embedding

Embedding

Embedding

Integrated Dataset

Big-4 (MBTI)

1 2

Decision

A.3

Shared

1

a

Fully Connected Layer

Query set

Encoder Layer

Processed & Support & Format Query set Ready

Encoder Layer

Support set

Encoder Layer

Machine-Induced Consensus

A.1 sent to

Yes

Few-shot Learning

Personality

Support set

Meta-Testing

Essays Dataset

Kaggle Dataset

Proposed Cross-Theory Harmonization

Attention-Pooled Graph Neural Network

Fig. 2. Overview of the proposed JAM architecture for theory-agnostic personality recognition. The framework integrates a language embedding backbone with an attention-pooled graph prototypical network to learn representations that capture latent pseudo-facets through clustering in the embedding space. A CrossTheory Harmonization module is introduced to bridge heterogeneous psychological annotations and consists of Human-Guided Linkage and Machine-Induced Consensus for constructing consistent pseudo-facet structures. Under this harmonization process, Essays and Kaggle datasets are organized into support–query pairs for prototypical learning. Two training pipelines are explored: (A) LLM-before-the-loop, where an LLM filters and refines data prior to training, and (B) LLM-in-the-loop, where the LLM dynamically assesses sample quality during training to identify ambiguous, mislabeled, or boundary cases. The resulting embeddings are attention-pooled and passed through a projection layer. The model is optimized using a meta-learning objective with contrastive loss, followed by a classification head trained with cross-entropy, enabling the formation of machine-induced pseudo-facets.

As in Eq. 1, we treated the dataset, Dstandard , as a multi-label (or multi-task) classification problem instead of a multi-class classification problem, where: n is the total number of samples in the dataset, x represents the sample text, and y represents the labels (the number of labels depends on the dimensions of the personality theory, d = {d1 , d2 , . . . , d|d| }) associated with x. This is because personality theory treats each dimension as independent from the others [52], [53]. However, certain research argues that there are interdependencies or correlations between personality dimensions, suggesting that traits might not be entirely independent but could interact in complex ways [16], [53]. Nevertheless, we adopt a multilabel classification approach to ensure that the model outputs a probability distribution over the dimensions. This approach avoids framing the task as a binary classification problem, instead allowing the intermediate layers of the neural network to automatically capture potential correlations between the labels via gradient backpropagation. In addition to the labels y, we include a

vector of sample-specific confidence z, where each zi adjusts the influence of the corresponding label yi during training.

Dstandard =

        

x = sample text (x, y, z)n

B. Proposed Algorithms

y = (y0 , y1 , . . . , y|d|−1

    ), 

z = (z0 , z1 , . . . , z|d|−1 ),     yi ∈ {0, 1}, zi ∈ R ∀i

(1)

As illustrated in Fig. 2, the JAM framework comprises three major components designed to achieve the overall objectives. We design an Attention-Pooled Graph Prototypical Network to learn representations that capture underlying pseudo-facets through clustering in the embedding space. JAM incorporates a Cross-Theory Harmonization module, which includes Human-Guided Linkage and Machine-Induced Consensus, enabling the model to derive pseudo-facets from

6

TABLE II DATASETS AND THEIR DESCRIPTIONS WITH THE NUMBER OF SAMPLES IN TRAIN , VALIDATION , AND TEST SPLITS Dataset Essays (Public) [50]

Theory Big-5

Splitting Distribution 1578, 395, 494 (Train, Validation, Test)

Kaggle (Public) [51]

MBTI

5552, 1388, 1735 (Train, Validation, Test)

Description This dataset was collected in a controlled environment where volunteers were instructed to write down whatever came to mind over a 20-minute period. It includes both self-reported ratings (via questionnaire) and projective ratings (by 18 observers). This dataset comprises data crawled, labeled, and filtered from PersonalityCafe, an online community forum where users share self-reported personality test results.

learned representations rather than relying solely on predefined human annotations, as discussed in Section II-A. We further investigate an LLM-as-a-Judge mechanism operating under two configurations: LLM-before-the-loop and LLM-inthe-loop. These configurations differ in the stage and manner in which the LLM influences training. The mechanism is incorporated to assess training sample quality and identify hard examples, including ambiguous, mislabeled, or boundaryadjacent instances, thereby guiding more focused learning. 1) Attention-Pooled Graph Prototypical Network: We adopt the Longformer model as the language embedding backbone due to its ability to efficiently process long textual sequences, which is particularly important for personality recognition tasks involving extensive user-generated content. Longformer employs a combination of local and global attention mechanisms, enabling scalable encoding of long documents while maintaining computational efficiency [54]. Given an input text x, we first tokenize it and feed it into the Longformer encoder to obtain contextualized hidden representations. The encoder produces layer-wise hidden states across all transformer layers. From these, we derive a set of L pooled representations, where each node corresponds to a layer-wise pooled representation of the input in Rv obtained via pooling over token embeddings within that layer. The resulting node feature matrix is defined in Eq. 2. H = Longformer(x),

H ∈ RL×v

(2)

Later, the output H is used as input to the graph neural network, which refines the representations by modeling interactions among embedding vectors derived from the language encoder. We construct a normalized adjacency matrix à from an adjacency matrix A that defines the relationships between nodes. In this work, we adopt a fully connected weighted graph, where each node is connected to all other nodes, to enable unrestricted and symmetric information exchange across all representation nodes. The number of nodes in the graph neural network corresponds to the number of layers (L) in the selected encoder, and the adjacency matrix remains shared across all layers. The node aggregation process is illustrated in Eq. 3. X à = D−1/2 AD−1/2 , Dii = Aij , j (3) L×L Aij = 1, ∀i ̸= j, Aii = 0, à ∈ R . This design treats all layer-wise representations as mutually interacting components without imposing predefined hierarchical or locality constraints, thereby allowing the model to learn how information should be integrated across different

abstraction levels in a data-driven manner. While this choice provides a simple and uniform mechanism for cross-layer fusion, it does not explicitly encode heterogeneous or sparse inter-layer dependencies. Investigating adaptive or learned graph structures that could more finely capture layerspecific relationships is therefore left as a promising direction for future work. Through matrix multiplication with the normalized adjacency matrix Ã, information is aggregated from neighboring nodes to update node embeddings. This propagation is performed iteratively across GNN layers, where H(k) denotes the node representations at the k-th layer. Eq. 4 illustrates the node update process. Here, σ(·) denotes the LeakyReLU activation function with negative slope coefficient α = 0.01, and W(k) is the trainable weight matrix at the k-th layer.   H(k) = σ ÃH(k−1) W(k) ,

σ(x) = max(αx,x), W(k) ∈ Rv×v , H(k) ∈ RL×v .

(4)

Lastly, we apply attention-based pooling to obtain the final graph representation, denoted as hgraph , as shown in Eq. 5. The attention mechanism computes a scalar importance score for each node based on its interaction with a trainable query vector q. These scores are normalized via a softmax function to obtain attention weights αi , which determine the contribution of each node to the final representation. This operation is fully differentiable, allowing gradients to propagate not only to the query vector q, but also to each dimension of the node (k) embeddings Hi . This enables the model to jointly learn which nodes are important and how the inferred embeddings should be adjusted to optimize personality trait prediction.   (k) exp Hi q (k)  , q ∈ Rv (5)  hgraph = αi Hi , αi = P (k) L exp H q i=1 j j=1 L X

2) Cross-Theory Harmonization (CTH): In order to achieve theory-agnostic modeling, we aim to ensure that the model captures the core personality features embedded within the text. Given the presence of personality theories across different datasets, this setting provides an opportunity for the model to learn not only surface-level patterns but also to structure the representation space in a more disentangled manner. Fig. 3 illustrates the overall idea behind this motivation. Under a standard cross-entropy (CE) formulation, different personality theories are treated as independent labels. As a result, the model tends to prioritize dominant and easily separable signals during optimization, while underutilizing or completely ignoring subtler but potentially shared representational

7

Big-5 (OCEAN)

Shared

Big-4 (MBTI)

Big-5 (OCEAN)

Shared

5

3|2

a

3|2

a c

c

b|4

e

4

e

b|4

5

d

d

4

6

6

(a) Regular CE

(b) Weighted CE

exp −∥fϕ (q) − si ∥2 LPF = −w · log P , 2 j exp (−∥fϕ (q) − sj ∥ ) P (S) w · fϕ (x) (x,w)∈Di P si = (S) w (x,w)∈D (Q)

5

2|1

2|1

5

Big-4 (MBTI)

1|3

1|3

(6)

i

Big-5 (OCEAN)

Shared

Big-4 (MBTI)

Big-5 (OCEAN)

Shared

1

2|5

Big-4 (MBTI)

1|3

1

1 5

c

5

e

a

3

5

2

3

4

d

6

b

4

(c) Prototypical Finetuning (PF) Big-5 (OCEAN)

Shared

Big-4 (MBTI)

6

(d) PF + HGL Big-5 (OCEAN)

Shared

1

1

1

1

e

5

a

5

c|d

3 4

6

(e) PF + MIC

2

2

e

c|d

d 3

b

1 5

2|5

5 c

Big-4 (MBTI)

1 2

5

2 a

2

c|d

c|d

4

e

3

3

b|4

3 b

5

2

2|1

a

1

3|2

2

c

d 3

2 b

3

3 4

6

(f) PF + HGL + MIC (Full CTH)

Fig. 3. The conceptual schematic visualization comparing the effects of the baseline (prior) algorithm, the proposed CTH algorithm, and its ablation variants. Each particle represents a pseudo-facet that belongs to the respective cluster of the personality theory: red and blue correspond to the respective personality theories used during inference. The numbers inside each particle represent shared features that can potentially be learned and aligned. Identical numbers (e.g., 1, 2, 3, . . . ) indicate the same feature across theories, while the numerical values themselves are only used as indices and do not carry intrinsic meaning. Alphabetic labels denote features unique to each personality theory; letters (a, b, c, . . . ) are also meaningless sequential identifiers. Purple particles indicate shared features that have been successfully learned and aligned, while grey particles represent incorrectly aligned features. The values are presented ·|· where the left side corresponds to the Big-5 representation and the by ⃝, right side corresponds to the MBTI representation.

facets across theories. Although weighted CE (eg., focal loss) can rebalance gradient contributions and improve sensitivity to harder samples, these approaches primarily amplify alreadylearned discriminative cues rather than fundamentally restructuring the representation space. Consequently, the underlying bias toward dominant features remains largely unchanged. To address this limitation, we adopt prototypical finetuning (PF). The class prototypes are computed from the support set D(S) , where each prototype si is defined as the (weighted) centroid of embeddings belonging to a given class. As shown in Eq. 6, each prototype is obtained as a weighted mean of support embeddings, where sample weights are denoted by z (set to z = 1 in our current setting). The encoder fϕ (x) maps each input x into an embedding space parameterized by ϕ, and classification is performed by comparing query embeddings against class prototypes in this space. In particular, the model minimizes the distance between a query embedding and the prototype of its ground-truth class while maximizing its distance to all other prototypes. A softmax over negative Euclidean distances is then used to define the probability of assigning a query q to class di . The training objective is formulated as a weighted negative loglikelihood over the query set D(Q) , where each query is also associated with a sample-specific weight.

Through sampling combinations of instances and prototypes, PF naturally alleviates data imbalance and reduces representational bias. This induces a structured clustering behavior in which distinct facets are progressively aligned with their corresponding theory-specific prototypes, while latent shared structures begin to emerge. However, at this stage, certain facets remain under-trained or ambiguously represented, which may still lead to misclassification. Overall, PF encourages the representation space to organize around theoryaware prototypes, as illustrated in Fig. 3(c). We extend this with Human-Guided Linkage (HGL), where external human-defined supervision is introduced to explicitly align shared facets across personality theories, encouraging the emergence of a more unified shared representation space. As in Fig. 3(d), while this strategy improves crosstheory alignment, it may also introduce noise due to imperfect or overly rigid mappings that do not always reflect true semantic correspondence. Nevertheless, these guided linkages are particularly valuable for handling edge cases: even when the provided alignments are partially inaccurate, they still gently steer the representations toward shared regions, facilitating the re-discovery and re-association of related facets. In parallel, we explore Machine-Induced Consensus (MIC). We employ a cross-entropy objective (Eq. 7) to guide cross-dataset adaptation via a lightweight adaptation layer that projects embeddings into a shared space. This layer is not treated as part of the core model architecture; instead, it serves as a training-time mechanism to facilitate convergence toward representations that remain discriminative across different personality theories. (D∈{Essays,Kaggle}) (y, ŷ) = − LMIC

" y log ŷ

+ (1 − y) log(1 − ŷ)

#

(7)

As illustrated in Fig. 3(e), consensus is learned through joint optimization over paired tasks, enabling the model to infer shared structure from agreement signals rather than explicit manual supervision. This process helps refine and further “clean” the shared space established by prototypical fine-tuning (PF), while also revealing latent pseudo-facets that are consistently aligned across theories. However, MIC exhibits a conservative alignment tendency, prioritizing highconfidence correspondences and potentially down-weighting weaker but still informative relationships. Consequently, it is most effective when applied after Human-Guided Linkage (HGL), where it acts as a stabilizing mechanism that regularizes and consolidates the previously introduced human-guided alignments. Lastly, we integrate PF, HGL, and MIC, termed CrossTheory Harmonization (CTH). This hybrid design leverages the complementary strengths of each component: PF provides

8

stable prototypical anchors for organizing the representation space, HGL introduces external guidance for soft alignment of potentially shared facets across theories, and MIC further refines these relations through data-driven consensus. Together, these mechanisms progressively reshape the representation space from one dominated by isolated theory-specific signals into a coherent shared manifold, where both distinct and overlapping pseudo-facets are more faithfully encoded, and previously under-represented or unlearned features are systematically recovered and integrated. 3) LLM-as-a-Judge (LAJ): While CTH aims to align representations across personality theories, residual noise may still hinder effective learning and lead to incorrect connections, as shown in Fig. 3(c), despite MIC efforts, which cannot fully address noise arising from mislabeled data. Moreover, while LLM-based augmentation has been shown to improve performance [49], it introduces potential risks of data leakage. We therefore argue that LLMs are better positioned as auxiliary evaluators rather than primary predictors. In particular, encoder-based architectures remain the backbone for representation learning and classification, while LLMs are employed as reasoning-based judges for data quality assessment rather than direct personality inference. They evaluate label consistency, contradictions, and noisy or implausible samples, improving dataset integrity and reducing leakage into core predictions. This preserves the theoretical grounding of encoder-based personality modeling while leveraging LLMs’ world knowledge and reasoning capabilities. In this work, we adopt LAJ to evaluate data correctness, specifically using (i) OpenAI’s ChatGPT model (gpt, gpt-4o-2024-08-06), (ii) Alibaba Qwen model (qwen, Qwen3.6-35B-A3B), (iii) Meta Llama model (llama, Llama-3.1-8B-Instruct), and (iv) DeepSeek model (ds, DeepSeek-V4-Flash). We employ Chain-of-Thought (CoT) prompting to guide the LLMs in analyzing and determining whether a labeled sample is Possible, Impossible, or Ambiguous with respect to its associated personality label, as in Fig. 4. 1

messages = [

2 3

{ "role": "system",

4 5

"content": ( "Evaluate whether the given written content is consistent with the specified personality traits using the Big Five (OCEAN) framework.\n\n"

6 7

"Your task is to assess if someone with the provided personality labels could plausibly have written the content.\n" "Use psychological reasoning and textual analysis to determine whether the content reflects each trait.\n\n"

8 9 10

"Input:\n" " 'user_personality': ['High Openness', 'Low Conscientiousness', ...],\n" " 'user_content': '...' # The written content to analyze\n"

11 12

"Output format (JSON):\n" "{\n"

13 14

" "

'openness': {'analysis': '', 'judgment': ''},\n" 'conscientiousness': {'analysis': '', 'judgment': ''},\n"

15 16 17

" " "

'extraversion': {'analysis': '', 'judgment': ''},\n" 'agreeableness': {'analysis': '', 'judgment': ''},\n" 'neuroticism': {'analysis': '', 'judgment': ''}\n"

18 19

"}\n\n" "Use: 'possible', 'impossible', or 'ambiguous' for each judgment."

20 21 22

) }, {

23 24 25 26

"role": "user", "content": processed_text } ]

Fig. 4. The CoT System Prompt generates output in a structured JSON schema. It performs an analysis of text samples and provides recommendations on whether to ‘maintain’, ‘mute,’ or ‘lower’ contributions for training.

We further conduct QLoRA fine-tuning experiments on open-source LLMs. Training is performed exclusively on the training split using the reasoning-data synthesis method proposed by [49], where ground-truth labels are leveraged to generate reasoning traces that supervise the model’s decisionmaking process. However, the original dataset consists only

of prototypical binary personality labels and lacks ambiguous cases that require reasoning over mixed or conflicting personality signals. To address this limitation, we augment the training data by constructing ambiguous samples through the combination of two distinct personality profiles, randomly selecting sentences from each profile to form a single input. This augmentation exposes the model to more realistic borderline cases and enables us to investigate whether LLM-based judges exhibit systematic failures or introduce new prediction biases when personality evidence is ambiguous. The augmentation is applied exclusively to the training data, while the test set remains untouched, ensuring that no data leakage occurs. As shown in Fig. 2, we study two pipelines. In the LLMbefore-the-loop (LBL) approach (Algorithm 1), the LLM is employed prior to training to assess and refine training samples based on label plausibility. In contrast, the LLMin-the-loop (LIL) strategy (Algorithm 2) integrates the LLM during training, where it dynamically evaluates samples in real time based on their loss values. Algorithm 1 LLM-before-the-loop (LBL) Require: Dstandard = {(x, y, z)}, where zi = 1 by default, system_prompt, JSON_output_schema Ensure: Updated Dstandard with modified z 1: for all (x, y, z) in Dstandard do 2: for i = 0 to |d| − 1 do 3: Build x and yi content and format as the "user" role, and Merge with system_prompt 4: repeat 5: Send to LLM using JSON_output_schema 6: Attempt to parse LLM output 7: until valid JSON is returned 8: Extract LLM judgment and update the corresponding z. 9: end for 10: end for 11: return Dstandard

Algorithm 2 LLM-in-the-loop (LIL) Require: Dstandard = {(x, y, z)}, where zi = 1 by default, model fϕ , system prompt, JSON_output_schema, threshold τ Ensure: Updated model parameters and possibly revised labels and weights 1: for each training episode do 2: Sample support and query sets (D(S) , D(Q) ) from Dstandard , 3: Compute prototypes {sj } from D(S) 4: for all (x, y, w(Q) ) in D(Q) do 5: Compute query embedding fϕ (q) for LJAM(LIL) 6: if LJAM(LIL) > τ then 7: Build x and yi content and format as the "user" role, and Merge with system_prompt 8: repeat 9: Send message to LLM with JSON_output_schema 10: Receive and attempt to parse response 11: until valid JSON is returned 12: Extract LLM judgment and update the corresponding z. 13: end if 14: end for 15: Aggregate and minimize total JAM(LIL) loss over query set 16: Update model parameters ϕ using gradient descent 17: end for

Regardless of the pipeline, the outcomes, depending on the LLM’s judgment, update z in Dstandard . This updated z

9

thereby influences each example’s contribution during prototype computation and training. The resulting examples are later sampled to form Dprototypical in each episode and are organized into Support (D(S) ) and Query (D(Q) ) sets for training. We implemented an N -way K-shot classification task. In our setting, D(S) = K ∗ N and D(Q) = q ∗ N , where N = |d| and q depends on the GPU RAM capacity. These were designed to facilitate representational learning, enabling the model to generalize to new personality theories. Eq. 8 illustrates the structure of the dataset for prototypical model training, while Eq. 9 shows the weighting in this experiment (modified later for sensitivity testing). We also used the weighted mean of the N embeddings as the support set for few-shot learning. Dprototype   N n o    [   (S) (S) (S) D(S) =   j = 1, . . . , K , xij , yij , zij     i=1 = N n o    [   (Q) (Q) (Q)    D(Q) = j = 1, . . . , q x , y , z   ij ij ij   i=1

(8)

  1.0 if ζ ∈ {Possible}, the LLM judg    ment is Possible (valid); maintain con    tributions.   0.2 if ζ ∈ {Ambiguous}, the LLM judgz= ment is Ambiguous; lower the contribu- (9)    tion.     0 if ζ ∈ {Impossible}, the LLM judg    ment is Impossible (invalid); mute the  contribution. Finally, we incorporate the LLM-as-a-Judge mechanism to weight the contributions of each loss term. We then summarize the overall formulation in Eq. 10, where the joint loss combines multiple components from different datasets and tasks.   (Essays) (Kaggle) (Essays ⇔ Kaggle) LJAM = ϕ LPF + LPF + ψ LHGL + ρ LMIC (10) C. Experiment Design We train the model individually on the Essays-only dataset and the Kaggle-only dataset to establish the baseline reference for the ablation study. Next, we train on the combined dataset to evaluate whether the JAM can generalize to new personality theories. Subsequently, we study the JAM algorithm to determine whether it can improve dataset quality and model performance. The experiments use a batch size of 32, a learning rate of 1 × 10−5 , a maximum of 30,000 training episodes with early stopping, a random seed of 42, and 4 NVIDIA A100 GPUs. Table III presents the acronyms used and the corresponding experiment configurations. D. Evaluation To evaluate the performance, we adopted the following metrics: Regular Accuracy (RA) (Eq. 11) to determine the overall

TABLE III OVERVIEW OF EXPERIMENTAL CONFIGURATIONS AND NOTATIONS Notation

Description

CE†

Regular cross-entropy serves as a standard classification baseline. Off-the-shelf model as the few-shot prototypical baseline. (No training) Fine-tuning the model using regular meta-learning for the few-shot prototypical baseline.(ϕ = 1; ψ = 0; ρ = 0; Eq. 6|z=1 )

PO† PF†

HGL‡ MIC‡ CTH‡

PF with HGL. (ϕ = 1; ψ = 1; ρ = 0; Eq. 6|z=1 ) PF with MIC. (ϕ = 1; ψ = 0; ρ = 1; Eq. 6|z=1 ) PF with (HGL + MIC). (ϕ = 1; ψe+1 ≤ ψe , ∀e ≥ 1; ρ = 1; Eq. 6|z=1 ).

JAM(LBL) <model> JAM(LIL) <model> [Essays] [Kaggle] [Both]

Proposed JAM approach with the LBL pipeline on CTH. Proposed JAM approach with the LIL pipeline on CTH. Training using only the Essays Dataset. Training using only the Kaggle Dataset. Training using both Essays Dataset and Kaggle Dataset.

Notes: †

Baseline performance without any CTH modules. Ablation setting experiments for CTH modules of the JAM approach. Experiments conducted without any weighting (no LAJ involved). <model> Indicates which LLM was used as judge in the JAM experiments. [·] Indicates the training dataset, attached after the notation. ‡

† ‡

accuracy of the model, Balanced Accuracy (BA) (Eq. 12) to assess performance on imbalanced datasets, and the F1 Score (Eq. 13) to measure the model’s bias. Here, T P , F P , T N , and F N represent true positives, false positives, true negatives, and false negatives, respectively. TP + TN . TP + TN + FP + FN   1 TN TP . BA = + 2 TP + FN TN + FP RA =

F1 =

2 · TP . 2 · TP + FP + FN

(11) (12) (13)

IV. R ESULTS A. Baseline Acquisition First, we establish a baseline for comparison by training the model separately on the Essays dataset, the Kaggle dataset, or both. Fig. 5 illustrates the balanced accuracy of each approach under different dataset settings: Cross-Entropy (CE), Off-theshelf Prototypical (PO), and Fine-tuned Prototypical (PF). In the CE experiment, there is no doubt that when the model is trained and evaluated on the same personality theory dataset, it achieves relatively good performance. However, when the model is trained on one dataset and evaluated on another, there is a significant drop in performance, indicating that the model struggles to generalize across different personality theories. We also observe that a combination of datasets does not improve performance; rather, it degrades it. This is likely because the model is confused by the conflicting signals from the two different personality theories. On the other hand, in the PO experiment, we observe that the Essays dataset shows very poor performance, while the Kaggle dataset is still able to capture relationships. This can

10

TABLE IV P ERFORMANCE COMPARISON ON THE E SSAYS DATASET, INCLUDING PRIOR WORK , THE BASELINE , ABLATION , AND LAJ MECHANISMS . Experiment Psycholinguistic MLP [55] BERT MLP [55] CoT with Emotion [32]

BA

O - Openness F1 RA

-

0.6093

0.6460 0.6040 0.6102

Baseline Representative [Essays]

0.6030

0.6278

0.6032

HGL [Both] MIC [Both] CTH [Both]

0.5144 0.6388 0.6734

0.5482 0.6564 0.6874

JAM(LBL) [Both] gpt JAM(LIL) [Both] gpt

0.7163 0.6752

0.7255 0.6911

C - Conscientiousness BA F1 RA

E - Extraversion BA F1 RA

-

0.6864

0.5920 0.5730 0.6800

0.5346

0.5287

0.5344

0.5162 0.6397 0.6741

0.5003 0.5426 0.6118

0.4907 0.5407 0.6000

0.7166 0.6761

0.6422 0.6036

0.6305 0.5934

A - Agreeableness BA F1 RA

-

0.6302

0.6000 0.5690 0.6201

0.6036

0.6117

0.6032

0.5000 0.5425 0.6113

0.5424 0.5843 0.6278

0.5794 0.6019 0.6320

0.6417 0.6032

0.6423 0.6162

0.6424 0.6138

N - Neuroticism BA F1 RA

-

0.6501

0.5880 0.5700 0.6498

-

0.5600

0.6050 0.5980 0.5600

0.5760

0.6190

0.5749

0.5536

0.5445 0.5850 0.6275

0.5575 0.5928 0.6389

0.5680 0.5976 0.6341

0.6258

0.5573

0.5567 0.5911 0.6356

0.5061 0.6032 0.6640

0.5396 0.6157 0.6693

0.5061 0.6032 0.6640

0.6417 0.6154

0.6511 0.6192

0.6463 0.6107

0.6478 0.6154

0.6842 0.6781

0.6917 0.6762

0.6842 0.6781

TABLE V P ERFORMANCE COMPARISON ON THE K AGGLE DATASET, INCLUDING PRIOR WORK , THE BASELINE , ABLATION , AND LAJ MECHANISMS . Experiment BERT MLP [55] TrigNet [56] TAE [57] DGCN [58]

BA

O - Openness F1 RA

C - Conscientiousness BA F1 RA

BA

E - Extraversion F1 RA

A - Agreeableness BA F1 RA

-

0.6717 0.8117 0.6719

0.6840 -

-

0.6769 0.7020 0.6816

0.6440 -

-

0.6954 0.7090 0.6952

0.7830 -

-

0.7906 0.6621 0.8053

0.7440 -

Baseline Representative [Kaggle]

0.8163

0.9397

0.8974

0.7837

0.7392

0.7914

0.8182

0.7218

0.8720

0.8412

0.8552

0.8427

HGL [Both] MIC [Both] CTH [Both]

0.5901 0.7695 0.8144

0.6717 0.9039 0.9260

0.5499 0.8409 0.8761

0.6409 0.6902 0.7634

0.5869 0.6390 0.7177

0.6398 0.6893 0.7660

0.6232 0.6825 0.7739

0.4333 0.5010 0.6480

0.6231 0.7049 0.8334

0.7583 0.8058 0.8408

0.7626 0.8177 0.8559

0.7556 0.8058 0.8427

JAM(LBL) [Both] gpt JAM(LIL) [Both] gpt

0.8133 0.7941

0.9287 0.9194

0.8801 0.8651

0.7721 0.7538

0.7280 0.7072

0.7735 0.7556

0.7872 0.7356

0.6659 0.5818

0.8403 0.7879

0.8498 0.8171

0.8648 0.8372

0.8519 0.8202

be attributed to the nature of how the datasets are collected. The Essays dataset comes from a constrained environment, whereas the Kaggle dataset is sourced from social media, which is more informal and diverse. This provides confidence that few-shot learning can be effective in personality recognition tasks. One interesting observation is that it naturally solves the class imbalance problem, as the algorithm focuses on learning the class prototypes rather than being biased towards the majority class. When the model is fine-tuned using PF, we see a significant improvement in performance within the same dataset (training and evaluation). This is especially evident for the Kaggle dataset, giving confidence that the model can learn the underlying personality patterns in the data. An interesting observation is that a model trained on the Kaggle dataset can generalize to the Essays dataset, but not vice versa, further demonstrating the quality of the Kaggle dataset. Additionally, the results show that the model trained on the Kaggle dataset achieves similar results to the CE approach trained on the Essays dataset, further justifying that a prototypical network has potential in capturing personality features from text. B. Performance 1) Cross-Theory Harmonization (CTH) Performance: Table IV and Table V tabulate the performance of each CTH module under different dataset settings with its ablation study. To provide a reference baseline, we include the row corresponding to the highest performance (BA) from the aforementioned experiments (as shown in Fig. 5) as representative baseline. We also include prior work; however, it is not fully comparable since they focus on training on a single dataset,

whereas we train 1 model for 2 personalities theories. Nevertheless, our performance remains competitive and significant in most cases, highlighting the advantage of our approach for generalization. By observing the results on the Essays dataset, we find that the model improves on average by more than 9% in balanced accuracy across all traits compared to the best-performing baseline. We further analyze the contribution of each component. Across both datasets, when only HGL is included, performance actually degrades regardless of the dataset. This supports the aforementioned hypothesis that human knowledge is limited; in the Essays dataset, due to its constrained data collection environment, such limitations may hinder the model and expose it to suboptimal supervision, obscuring the ultimate learning objective through self-consensus. Considering MIC alone, it only outperforms the baseline on the Essays dataset but not on Kaggle, further strengthening the observation that the combination of Essays data introduces additional noise. Nevertheless, the combination of both components (CTH) is able to restore performance to a level comparable with the baseline on Kaggle and further improve results on the Essays dataset, suggesting that integrating HGL and MIC allows the model to better merge complementary knowledge sources, achieving theory-agnostic modeling. 2) LLM-as-a-Judge (LAJ) Performance: Next, we study the impact of the LAJ mechanism. Generally, the JAM(LBL) gpt [Both] approach consistently outperforms the JAM(LIL) [Both] gpt approach across all personality traits in both datasets. This suggests that pre-evaluating and refining the dataset before training is more effective than dynamically assessing samples during training. The JAM(LBL) [Both] model converges gpt

Balanced Accuracy (BA)

11

0.68 0.66 0.64 0.62 0.60 0.58 0.56 0.54 0.52 0.50 0.48 0.46 0.44 0.42 0.40

CE [Essays] CE [Kaggle] CE [Both] PO PF [Essays] PF [Kaggle] PF [Both]

Balanced Accuracy (BA)

O_1 O_1 O_1 O_1 C_1 C_1

0.569 0.548 0.546 0.510 0.508 0.499 0.470

C_1 C_1

E_1 E_1 A_1 A_1

possible possible (85.7%) (85.7%)

N_1 N_1

C_0 C_0 E_0 E_0

O

C

E

A

N

CE [Essays] CE [Kaggle] CE [Both] PO PF [Essays] PF [Kaggle] PF [Both]

0.815

0.727 0.709 0.656 0.645 0.573 0.568

O (S/N)

C (J/P)

E (E/I)

A (S/N)

(b) Kaggle Dataset Fig. 5. Visualization of balanced accuracy for regular classification using cross-entropy (CE), prototypical few-shot learning using off-the-shelf ready model (PO), and fine-tuned prototypical few-shot learning (PF) on the Essays and Kaggle datasets. Each line in the legend corresponds to an individual experiment, with the dataset used for training indicated in the respective brackets, while the dotted lines with numbers represent the average value across dimensions of the respective experiment.

within approximately 3000 episodes, whereas JAM(LIL) [Both] requires around 7000 episodes to converge. This difference arises because the JAM(LIL) [Both] approach only gradually gpt obtains a cleaner dataset over multiple iterative episodes, and not all samples are evaluated by the LLM due to the thresholding mechanism applied during training. As a result, an initially noisy dataset may mislead the model toward suboptimal local minima, which helps explain why the JAM(LIL) [Both] gpt approach is less effective than JAM(LBL) [Both]. This trend is gpt also reflected in the Essays dataset, where the performance of JAM(LBL) [Both] is comparable to CTH [Both]. gpt Fig. 6 illustrates the distribution of Possible and Impossible judgments across personality traits for both datasets. The Essays dataset contains a higher proportion of Possible judgments (85.7%) compared to the Kaggle dataset (70.8%). Given that only a limited proportion of the dataset is estimated to be noisy (approximately > 8%), the observed 2% improvement is within a reasonable range. This indicates that the filtering process effectively reduces the influence of noisy samples, while also suggesting that the achievable performance gain

possible possible (70.8%) (70.8%) A_1 A_1 E_1 E_1 O_0 O_0

O_0 O_0

C_0 C_0 impossible impossible (5.5%) (5.5%) ambiguous ambiguous (8.8%) (8.8%)

E_0 E_0

impossible impossible (15.7%) (15.7%)

ambiguous ambiguous (13.4%) (13.4%)

A_0 A_0 A_0 A_0

N_0 N_0

(a) Essays Dataset

(a) Essays Dataset 0.98 0.96 0.94 0.92 0.90 0.88 0.86 0.84 0.82 0.80 0.78 0.76 0.74 0.72 0.70 0.68 0.66 0.64 0.62 0.60 0.58 0.56 0.54 0.52 0.50

is naturally bounded by the proportion of removable noise in the dataset.

(b) Kaggle Dataset

Fig. 6. Distribution of LLMgpt judgments on personality trait implications across datasets. The figure illustrates the proportion of LLM-judged outcomes—Possible, Impossible, and Ambiguous—based on the binary labels for each of the personality traits. Labels in the form d_y represent trait d with label y, where y = 1 denotes a positive class (presence of the trait) and y = 0 denotes a negative class. The divergence in distributions highlights the influence of annotation conditions and data origin on interpretability judgments made by language models.

Fig. 7 shows the performance of different LLMs under varying hyperparameter settings. Overall, gpt performs the best. On the Kaggle dataset, LLMs generally do not improve over the baseline, which is consistent with the CTH stage (without LAJ). In contrast, the Essays dataset shows a different trend: with the involvement of LAJ, performance generally improves over CTH alone. This supports the generalisability claims for the low-resource theory (Essays dataset) compared with the Kaggle dataset, showing that in most cases, model selection does not degrade performance. We observed that setting zζ∈{Impossible} = 0 (muting the Impossible samples) often leads to better performance in the Essays dataset, supporting the generalisation claim. However, this effect is less consistent in the Kaggle dataset and depends more on the choice of LLM. From the results of the Kaggle dataset, zζ∈{Impossible} = 0 generally has relatively low performance compared to other hyperparameters, suggesting that the model is not only incapable of filtering data but also introduces more noise by removing some important contributions. In addition, we conduct an experiment in which 80% of the minor (relatively underrepresented) labels/classes are removed (in gpt) using the setting zζ∈{Ambiguous} = 0.2 and zζ∈{Impossible} = 0, to simulate a worse model scenario. The results show a significant drop across all dimensions, despite this being the best observed hyperparameter setting for gpt. This again supports the claim that a worse LLM can negatively affect training, and shows that zζ∈{Impossible} is sensitive. Across different models, zζ∈{Ambiguous} does not significantly affect performance (evaluated under settings zζ∈{Ambiguous} = 0.2 and zζ∈{Impossible} = 0.8, as well as zζ∈{Impossible} = 0). This suggests that it primarily acts as a tunable non-sensitive hyperparameter rather than a structural factor. On the other hand, in terms of their fine-tuned model, the results show that although there is improvement in certain dimensions for some LLMs on the Kaggle dataset, the gains are not statistically significant. In some cases, performance even drops on the Essays dataset, likely due to imbalance, as Kaggle is more dominant and relatively easier to learn. These findings suggest that well-trained LLMs are already

Balanced Accuracy (BA)

12

0.80 0.78 0.76 0.74 0.72 0.70 0.68 0.66 0.64 0.62 0.60 0.58 0.56 0.54 0.52 0.50 0.48 0.46

CTH (No LAJ) z = 1

z ζ∈

(

Ambigous Imposible

)

= 0

z ζ∈

(

Ambigous Imposible

)

zζ∈{Ambigous} = 0.8 zζ∈{Imposible} = 0.5

= 0.5

zζ∈{Ambigous} = 0.8 zζ∈{Imposible} = 0

zζ∈{Ambigous} = 0.2 zζ∈{Imposible} = 0

Modified model zζ∈{Ambigous} = 0.2 zζ∈{Imposible} = 0

G: G: 1 1 G: 1 G: G: 1 1

G: G: 0 0 G: 0 G: G: 0 0

PF: PF: TP TP PF: TP PF: PF: TP TP

PF: PF: FN FN PF: FN PF: PF: FN FN

MIC: MIC: TP TP MIC: TP MIC: MIC: TP TP

MIC: MIC: FN FN MIC: FN MIC: MIC: FN FN

PF: PF: TN TN PF: TN PF: PF: TN TN

G: G: 1 1 G: 1 G: G: 1 1

G: G: 0 0 G: 0 G: G: 0 0

PF: PF: FP FP PF: FP PF: PF: FP FP

MIC: MIC: TP TP MIC: TP MIC: MIC: TP TP

MIC: MIC: FN FN MIC: FN MIC: MIC: FN FN

MIC: MIC: FP FP MIC: FP MIC: MIC: FP FP

CTH: CTH: TP TP CTH: TP CTH: CTH: TP TP

CTH: CTH: FN FN CTH: FN CTH: CTH: FN FN

p-value= 1.969e-06

(a)

0.6118

0.6278

0.6389

0.6640

0.6985 0.6897 0.6840 0.6866 ± 0.0145 ± 0.0163 ± 0.0052 ± 0.0129

0.6351 0.6172 0.6262 0.6095 ± 0.0082 ± 0.0068 ± 0.0128 ± 0.0054

0.6371 0.6238 0.6241 0.6169 ± 0.0062 ± 0.0095 ± 0.0057 ± 0.0068

0.6443 0.6292 0.6278 0.6291 ± 0.0055 ± 0.0068 ± 0.0078 ± 0.0088

0.6765 0.6700 0.6664 0.6660 ± 0.0058 ± 0.0067 ± 0.0044 ± 0.0067

O

C

E

A

N

gpt qwen llama ds

gpt qwen llama ds

gpt qwen llama ds

gpt qwen llama ds

z ζ∈

(

Ambigous Imposible

)

= 0

z ζ∈

(

Ambigous Imposible

)

= 0.5

zζ∈{Ambigous} = 0.8 zζ∈{Imposible} = 0.5

(b)

G: G: 0 G: 0 G: G: 0 0 0

zζ∈{Ambigous} = 0.8 zζ∈{Imposible} = 0

G: G: 1 G: 1 G: G: 1 1 1

PF: PF: TP TP PF: TP PF: PF: TP TP

zζ∈{Ambigous} = 0.2 zζ∈{Imposible} = 0

Modified model zζ∈{Ambigous} = 0.2 zζ∈{Imposible} = 0

MIC: MIC: TP TP MIC: TP MIC: MIC: TP TP

(d)

Balanced Accuracy (BA)

0.85

JAM: JAM: TP TP JAM: TP JAM: JAM: TP TP

JAM: JAM: FN FN JAM: FN JAM: JAM: FN FN

CTH: CTH: TN TN CTH: TN CTH: CTH: TN TN

CTH: CTH: FP FP CTH: FP CTH: CTH: FP FP

(c)

JAM: JAM: TN TN JAM: TN JAM: JAM: TN TN

JAM: JAM: FP FP JAM: FP JAM: JAM: FP FP

Essays Dataset (LBL) (CTH→ JAMgpt )[Both]

G: G: 1 G: 1 G: G: 1 1 1

G: G: 0 G: 0 G: G: 0 0 0

PF: PF: FN FN PF: FN PF: PF: FN FN

PF: PF: TN TN PF: TN PF: PF: TN TN

PF: PF: FP FP PF: FP PF: PF: FP FP

MIC: MIC: TP TP MIC: TP MIC: MIC: TP TP

MIC: MIC: FN FN MIC: FN MIC: MIC: FN FN

MIC: MIC: FN FN MIC: FN MIC: MIC: FN FN

MIC: MIC: TN TN MIC: TN MIC: MIC: TN TN

MIC: MIC: TN TN MIC: TN MIC: MIC: TN TN

MIC: MIC: FP FP MIC: FP MIC: MIC: FP FP

CTH: CTH: TP TP CTH: TP CTH: CTH: TP TP

p-value= 8.748e-268

MIC: MIC: FP FP MIC: FP MIC: MIC: FP FP

Kaggle Dataset (PF→MIC)[Both]

CTH: CTH: TP TP CTH: TP CTH: CTH: TP TP

(e)

CTH: CTH: FN FNCTH: CTH: TN TN CTH: FN CTH: TN CTH: CTH: FN FN CTH: CTH: TN TN

Kaggle Dataset (MIC→CTH)[Both]

CTH: CTH: FN FNCTH: CTH: TN TN CTH: FN CTH: TN CTH: CTH: FN FN CTH: CTH: TN TN

CTH: CTH: FP FP CTH: FP CTH: CTH: FP FP

p-value= 0.1011

CTH: CTH: FP FP CTH: FP CTH: CTH: FP FP

JAM: JAM: TP TP JAM: TP JAM: JAM: TP TP

(f)

JAM: JAM: FN FNJAM: JAM: TN TN JAM: FN JAM: TN JAM: JAM: FN FN JAM: JAM: TN TN

JAM: JAM: FP FP JAM: FP JAM: JAM: FP FP

Kaggle Dataset (LBL) (CTH→ JAMgpt )[Both]

Fig. 8. The Sankey diagram illustrates the transitions between 4 approaches: from the PF [Both] to the MIC-involved method (first MIC [Both], and then to CTH [Both] method, and to the proposed full approach JAM(LBL) [Both]). gpt Note that the HGL-only is excluded from this test since it functions as early generalization guidance. It encapsulates noise and performs worse when functioning alone. Each method’s outcomes are compared against the original ground truth (G). Statistical significance is determined using the McNemar pvalue test. Due to the fact that personality encompasses multiple dimensions, we flatten and concatenate these dimensions to facilitate clearer visualization.

0.80 0.75 0.70 0.65 0.60

0.50

CTH: CTH: FN FN CTH: FN CTH: CTH: FN FN

p-value= 0.008246

CTH: CTH: FP FP CTH: FP CTH: CTH: FP FP

G: G: 0 G: 0 G: G: 0 0 0

0.90

0.55

CTH: CTH: TN TN CTH: TN CTH: CTH: TN TN

Essays Dataset (MIC→CTH)[Both]

p-value= 1.074e-128

1.00 CTH (No LAJ) z = 1

CTH: CTH: TP TP CTH: TP CTH: CTH: TP TP

gpt qwen llama ds

(a) Essays Dataset

0.95

MIC: MIC: FP FP MIC: FP MIC: MIC: FP FP

G: G: 0 0 G: 0 G: G: 0 0

p-value= 3.364e-22

Essays Dataset (PF→MIC)[Both]

G: G: 1 G: 1 G: G: 1 1 1

0.6734

MIC: MIC: TN TN MIC: TN MIC: MIC: TN TN

MIC: MIC: TN TN MIC: TN MIC: MIC: TN TN

G: G: 1 1 G: 1 G: G: 1 1

0.8144 0.8131 ± 0.0073

gpt

0.7925 ± 0.0145

0.7927 ± 0.0113

qwen llama

0.7634 0.7893 ± 0.0089

ds

0.7653 ± 0.0105

gpt

O

0.7364 ± 0.0134

0.7402 ± 0.0201

qwen llama

0.7739 0.7443 ± 0.0125

ds

0.7659 ± 0.0212

gpt

0.7448 ± 0.0190

0.7487 ± 0.0246

qwen llama

C

E

0.8408 0.7362 ± 0.0279

ds

0.8371 ± 0.0025

gpt

0.8089 ± 0.0167

0.8110 ± 0.0170

qwen llama

0.8126 ± 0.0190

ds

A

(b) Kaggle Dataset Fig. 7. Visualization of different large language models (gpt, qwen, llama, ds) and their performance under respective hyperparameter settings on ζ to z values in JAM(LBL) [Both], including a reduced contribution of the gpt ambiguous and impossible flags by down-weighting them at certain levels. We further incorporate fine-tuning for open-source models using QLoRA, while for gpt we conduct an experiment in which 80% of the minor (relatively underrepresented) labels/classes are removed to address class imbalance, under the assumption that the LLM significantly underperforms on the task, in order to study the potential effects of such imbalance induction.

sufficiently capable, and additional fine-tuning is unnecessary and induces higher costs for the judging task. C. Statistical Findings We conducted the McNemar test to support our findings, as illustrated in Fig. 8. The results indicate its potential utility in constrained environments, while the CTH method addresses the challenges of merging datasets, suggesting a pathway toward a theory-agnostic model. Furthermore, this demonstrates that our proposed approaches are able to reduce noise that appeared in the native collection of the dataset, where it statistically further improves through LAJ (an average of 2% improvement in the Essays dataset). To visualize the effectiveness of the proposed algorithm, we visualized the personality embeddings using t-SNE in Fig. 9. Compared to the off-the-shelf PO approach, the embeddings produced by JAM(LBL) [Both] are noticeably more structured gpt and exhibit clearer separation between clusters corresponding to distinct personality trait combinations. This improved separation indicates that the proposed method captures and preserves subtle personality cues from textual data more

effectively. In particular, the cluster centers in the JAM(LBL) gpt [Both] visualizations are more distinct and less overlapping, suggesting that the model can better differentiate between similar personality profiles. This pattern is consistent across both datasets, with the effect being particularly pronounced in the Kaggle dataset, where clusters are clearer due to a larger number of samples and relatively higher classification accuracy, further highlighting the robustness and generalizability of JAM(LBL) [Both]. gpt D. Computational Analysis Fig. 10 compares the proposed JAM with previous approaches in terms of Floating Point Operations (FLOPs) and the corresponding computational cost. This comparison applies to both LLM-before-the-loop (LBL) and LLM-in-the-loop (LIL) settings. Technically, the former achieves lower computational cost because the filtering mechanism depends only on the loss values; therefore, not all samples in the dataset require full inference. Overall, the proposed method achieves the lowest inference time compared to the PICEPR(Embeddings) method [49]. Although a slight additional overhead is introduced due to prototype retrieval, this overhead is negligible because it only involves retrieving and averaging 4–5 embedding vectors. Despite this, the method still achieves approximately 8× lower training FLOPs. Furthermore, the proposed approach demonstrates the feasibility of incorporating LLMs to improve performance, while simultaneously reducing the risk of data leakage, which is difficult to achieve with purely decoderonly model-driven approaches such as PICEPR(Contents) [49]. Overall, JAM maintains very low inference latency, which is particularly important for test-time deployment and real-world

13

00000 (0) 00001 (1) 00010 (2) 00011 (3) 00100 (4) 00101 (5) 00110 (6) 00111 (7) 01000 (8) 01001 (9) 01010 (10) 01011 (11) 01100 (12) 01101 (13) 01110 (14) 01111 (15) 10000 (16) 10001 (17) 10010 (18) 10011 (19) 10100 (20) 10101 (21) 10110 (22) 10111 (23) 11000 (24) 11001 (25) 11010 (26) 11011 (27) 11100 (28) 11101 (29) 11110 (30) 11111 (31)

t-SNE Component 2

20 12 15 21931 23 11 28 26 1519 30 25 2927 1614 424 317 7 18 6210 13 2220 0 8

0

20

40 40

20

0 20 t-SNE Component 1

40

40

00000 (0) 00001 (1) 00010 (2) 00011 (3) 00100 (4) 00101 (5) 00110 (6) 00111 (7) 01000 (8) 01001 (9) 01010 (10) 01011 (11) 01100 (12) 01101 (13) 01110 (14) 01111 (15) 10000 (16) 10001 (17) 10010 (18) 10011 (19) 10100 (20) 10101 (21) 10110 (22) 10111 (23) 11000 (24) 11001 (25) 11010 (26) 11011 (27) 11100 (28) 11101 (29) 11110 (30) 11111 (31)

20 t-SNE Component 2

40

1311 24 17 5 31 29 1 2820 19 21 930 15 34 16 228 25 6 27 18 12 7 10

0

23

20

0

26

2

14

40

60

40

Essays Dataset (a) PO [Essays]

20

0 20 t-SNE Component 1

40

60

Essays Dataset (b) JAM(LBL) gpt [Both]

0 Classify's COT (I)/ (O)

PICEPR(Contents)

0

108 0 14 3 6 7 15 11 1

0000 (0) 0001 (1) 0010 (2) 0011 (3) 0100 (4) 0101 (5) 0110 (6) 0111 (7) 1000 (8) 1001 (9) 1010 (10) 1011 (11) 1100 (12) 1101 (13) 1110 (14) 1111 (15)

12

5 13

9 2

20

4

0 2 64

20 t-SNE Component 2

t-SNE Component 2

20

40

40

0

8

14

20

40

20

0 20 t-SNE Component 1

40

60

Kaggle Dataset (c) PO [Kaggle]

0000 (0) 0001 (1) 0010 (2) 0011 (3) 0100 (4) 0101 (5) 0110 (6) 0111 (7) 1000 (8) 1001 (9) 1010 (10) 1011 (11) 1100 (12) 1101 (13) 1110 (14) 1111 (15)

13 9

11 15

40 40

20

0.0019

0 20 t-SNE Component 1

40

60

Kaggle Dataset (d) JAM(LBL) gpt [Both]

Fig. 9. The t-SNE visualization of personality embeddings on both Essays and Kaggle datasets. Subplots 9(a) and 9(c) show embeddings using the offthe-shelf model (PO), while subplots 9(b) and 9(d) show embeddings using the proposed JAM(LBL) [Both] approach. Each color represents a distinct gpt personality trait combination, with annotated labels indicating cluster centers. Compared to the PO approach, JAM(LBL) [Both] produces embeddings that gpt are more structured and better separated, reflecting improved modeling of personality cues obtained from text.

inference scenarios, while also reducing inference FLOPs, thereby lowering energy consumption and carbon footprint and improving the environmental sustainability especially on largescale deployment. E. Limitations and Future Works Although JAM demonstrates consistent improvements across heterogeneous personality datasets, it is currently validated on only two personality theories (MBTI and Big-5), and further evaluation on additional theories and culturally diverse datasets is needed to validate the robustness of the proposed Cross-Theory Harmonization (CTH). In addition, as personality recognition relies on sensitive personal data, future work should prioritize privacy, informed consent, and fairness. One promising direction is to integrate JAM with a federated learning framework, enabling collaboration across institutions or countries while keeping data local and aggregating only model updates, thereby improving generalizability without compromising data privacy. V. C ONCLUSION In this work, we introduced Large-Language-Models-as-aJudge in Theory-Agnostic Adaptive Metric-Alignment for Prototypical Networks in Personality Recognition (JAM), a novel prototypical framework designed to enhance personality prediction and generalizability by leveraging Cross-Theory Harmonization (Human-Guided Linkage and Machine-Induced Consensus) and LLM-as-a-Judge mechanisms, enabling the learning of latent pseudo-facets that capture shared behavioral

0.0028

1.88

3.75 5.81 7.88 23.78T FLOPs $0.0019

Mimic's COT (I)/ (O)

Encoder Model Training (Task-Focus)

EERPD

Classify's COT (I)/ (O)

Sentences Categorisation (I)/ (O)

Retrieval

TAE

Reasoning's COT (I)/ (O)

JAM

Retrieval Inference Reference Library (Single)/ (Whole Training Dataset)

24T

96.02P FLOPs $7.5018

Reference Library (Single)/ (Whole Training Dataset)

Inference Retrieval

12T

0.0037

Psycho's COT (I)/ (O)

Inference time

57 1

12 10 3

Summary's COT (I)/ (O)

Summary's Inference COT (I)/ (O)

PICEPR(Embeddings)

0 40

Cost ($)

0.0009

33.69P FLOPs $2.6322

Reference Library (Single)/ (Whole Training Dataset)

36T Pre-inference

Encoder Model Training (Task-Focus)

16.72P FLOPs $1.3063

Encoder Model Training (Task-Focus)

11.91P FLOPs $0.9301

48T Training time

24P FLOPs

48P 74P 101P

Fig. 10. Visualization of Cost and FLOPs Comparison. In this analysis, we assume that input and output tokens incur identical costs, adopting the estimated FLOPs per token and the corresponding pricing model used in [49] to enable direct comparison. We assume a constant inference cost of 3.2 GFLOPs per token and a pricing rate of $0.25 per 10M tokens. The visualization is based on three components: (i) single inference (grey, with pink representing search time, e.g., retrieval-augmented generation lookup or similarity measurement, quantified according to the respective approach used to achieve the reported performance), (ii) training time (purple, covering all experiments), and (iii) pre-inference (yellow and orange). Pre-inference (e.g., retrieval-augmented generation indexing or prototype preparation) is difficult to standardize because greater involvement can improve performance for certain methods. To address this, we divide the pre-inference stage into yellow and orange segments to represent an estimated ratio of involved classes or prototypes, where this is quantified according to the respective approach used to achieve the reported performance. Note that this visualization excludes the training cost of the pretrained decoder-only model, as it is common across all approaches and assumed to be comparable.

structure across heterogeneous personality theories under an embedding space. Through experiments on the Essays and Kaggle datasets, we demonstrated that JAM consistently outperforms existing baselines and prior work across multiple personality traits, with no class imbalance issues. Specifically, JAM achieves an average BA improvement of 12% on the Essays dataset and 14% on the Kaggle dataset compared to the regular prototypical few-shot learning approach. When combining datasets, it becomes possible to leverage the dataset for generalization in low-resource situations, where the performance of the Kaggle dataset is not affected. Moreover, incorporating the LLM-as-aJudge mechanism further improves performance on the Essays dataset by an additional 2.4%, by reweighting the contribution of each data row during model training. These results indicate that JAM is particularly effective in constrained environments while maintaining robust performance across diverse datasets. VI. ACKNOWLEDGMENTS This research was funded by the Universiti Tunku Abdul Rahman Research Fund (IPSR/RMC/UTARRF/2021-C1/K03). The authors also appreciate the support of Grid5000, France, for providing the computational resources used in this study. The authors thank the Advanced Artificial Intelligence Research Center at the Kanagawa Institute of Technology, Japan, for supporting OpenAI’s model inferences. The first author, Jing Jie Tan, also appreciates National Yang Ming Chiao Tung University for providing the Research Scholarship to establish a professional connection for guidance related to

14

psychology. Additionally, he appreciate the Embassy of France to Malaysia for the Doctoral Research Mobility Grant and the Japan Student Services Organization (JASSO) Scholarships, which facilitated the research collaboration between Universiti Tunku Abdul Rahman, Malaysia, Université Sorbonne Paris Nord, France, and Kanagawa Institute of Technology, Japan. R EFERENCES [1] S. Dhelim, N. Aung, M. A. Bouras, H. Ning, and E. Cambria, “A survey on personality-aware recommendation systems,” Artificial Intelligence Review, vol. 55, no. 3, p. 2409–2454, Sep. 2021. [Online]. Available: http://dx.doi.org/10.1007/s10462-021-10063-7 [2] T. Ait Baha, M. El Hajji, Y. Es-Saady, and H. Fadili, “The power of personalization: A systematic review of personality-adaptive chatbots,” SN Computer Science, vol. 4, no. 5, Aug. 2023. [Online]. Available: http://dx.doi.org/10.1007/s42979-023-02092-6 [3] Z. Li, S. Yang, and S. Wang, “Exploring personality-driven personalization in xai: Enhancing user trust in gameplay,” 2024. [Online]. Available: https://arxiv.org/abs/2408.04778 [4] S. Omidvar and T. Tran, “Tackling cold-start with deep personalized transfer of user preferences for cross-domain recommendation,” International Journal of Data Science and Analytics, Nov. 2023. [Online]. Available: http://dx.doi.org/10.1007/s41060-023-00467-9 [5] S. Dhelim, L. L. Chen, N. Aung, W. Zhang, and H. Ning, “Big-five, mpti, eysenck or hexaco: The ideal personality model for personality-aware recommendation systems,” 2021. [Online]. Available: https://arxiv.org/abs/2106.03060 [6] D. Stillwell and M. Kosinski, “mypersonality project website,” 2015. [7] L. V. Phan and J. F. Rauthmann, “Personality computing: New frontiers in personality assessment,” Social and Personality Psychology Compass, vol. 15, no. 7, Jun. 2021. [Online]. Available: http: //dx.doi.org/10.1111/spc3.12624 [8] M. Dalvi-Esfahani, A. Niknafs, Z. Alaedini, H. Barati Ahmadabadi, D. J. Kuss, and T. Ramayah, “Social media addiction and empathy: Moderating impact of personality traits among high school students,” Telematics and Informatics, vol. 57, p. 101516, Mar. 2021. [Online]. Available: http://dx.doi.org/10.1016/j.tele.2020.101516 [9] G. Lampropoulos, T. Anastasiadis, K. Siakas, and E. Siakas, “The impact of personality traits on social media use and engagement: An overview.” International Journal on Social and Education Sciences, vol. 4, no. 1, pp. 34–51, 2022. [10] E. Ahmed and S. Ahmed, “Social media addiction, personality traits, and disorders: an overview of recent literature,” Current Opinion in Psychiatry, vol. 38, no. 1, p. 72–77, Sep. 2024. [Online]. Available: http://dx.doi.org/10.1097/YCO.0000000000000969 [11] R. M. Spielman, K. Dumper, W. Jenkins, A. Lacombe, M. Lovett, and M. Perlmutter, “Personality assessment,” Introduction to Psychology (A critical approach), 2021. [12] R. W. Robins, J. L. Tracy, and J. W. Sherman, “What kinds of methods do personality psychologists use?: A survey of journal editors and editorial board members,” 2023. [13] M. H. Waugh, C. M. McClain, E. C. Mariotti, A. L. Mulay, E. N. DeVore, K. A. Lenger, A. N. Russell, A. R. Florimbio, K. C. Lewis, J. M. Ridenour, and L. G. Beevers, “Comparative content analysis of self-report scales for level of personality functioning,” Journal of Personality Assessment, vol. 103, no. 2, p. 161–173, Jan. 2020. [Online]. Available: http://dx.doi.org/10.1080/00223891.2019.1705464 [14] I. MUKHTASAR and M. MAVLUDA, “The study of projective methods in psychology,” JournalNX, vol. 7, no. 02, pp. 66–69, 2021. [15] R. Tett and D. Simonet, “Applicant faking on personality tests: Good or bad and why should we care?” Personnel Assessment and Decisions, vol. 7, no. 1, May 2021. [Online]. Available: http://dx.doi.org/10.25035/pad.2021.01.002 [16] B. W. Roberts and H. J. Yoon, “Personality psychology,” Annual Review of Psychology, vol. 73, no. 1, p. 489–516, Jan. 2022. [Online]. Available: http://dx.doi.org/10.1146/annurev-psych-020821-114927 [17] D. Cervone and L. A. Pervin, Personality: Theory and research. John Wiley & Sons, 2022. [18] I. B. Myers, “The myers-briggs type indicator: Manual (1962).” 1962. [19] A. Furnham, “Myers-briggs type indicator (mbti),” Encyclopedia of Personality and Individual Differences, pp. 1–4, 2017. [20] L. R. Goldberg, “The structure of phenotypic personality traits.” American Psychologist, vol. 48, pp. 26–34, 1993.

[21] X. Luo, Y. Ge, and W. Qu, “The association between the big five personality traits and driving behaviors: A systematic review and meta-analysis,” Accident Analysis and Prevention, vol. 183, p. 106968, 2023. [Online]. Available: https://www.sciencedirect.com/ science/article/pii/S0001457523000155 [22] M. C. Ashton, K. Lee, M. Perugini, P. Szarota, R. E. de Vries, L. D. Blas, K. Boies, and B. D. Raad, “A six-factor structure of personalitydescriptive adjectives: Solutions from psycholexical studies in seven languages.” Journal of Personality and Social Psychology, vol. 86, pp. 356–366, 2004. [23] P. William and A. Badholia, “Analysis of personality traits from text based answers using hexaco model,” in 2021 International Conference on Innovative Computing, Intelligent Communication and Smart Electrical Systems (ICSES), 2021, pp. 1–10. [24] J. J. Tan, B.-H. Kwan, D. W.-K. Ng, and Y. C. Hum, “Psychologyinformed natural language understanding: Integrating personality and emotion-aware features for comprehensive sentiment analysis and depression detection,” Pertanika Journal of Science and Technology, vol. 33, no. S4, Jun. 2025. [Online]. Available: http://dx.doi.org/10. 47836/pjst.33.S4.04 [25] W. Wang, F. Wei, L. Dong, H. Bao, N. Yang, and M. Zhou, “Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers,” in Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 5776–5788. [Online]. Available: https://proceedings.neurips.cc/paper_ files/paper/2020/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf [26] K. Song, X. Tan, T. Qin, J. Lu, and T.-Y. Liu, “Mpnet: Masked and permuted pre-training for language understanding,” in Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 16 857–16 867. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/ 2020/file/c3a690be93aa602ee2dc0ccab5b7b67e-Paper.pdf [27] J. Ni, G. H. Ábrego, N. Constant, J. Ma, K. B. Hall, D. Cer, and Y. Yang, “Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models,” 2021. [Online]. Available: https://arxiv.org/abs/2108.08877 [28] D. Bear and P. Cook, “Fine-tuning sentence-RoBERTa to construct word embeddings for low-resource languages from bilingual dictionaries,” in Proceedings of the Workshop on Natural Language Processing for Indigenous Languages of the Americas (AmericasNLP), M. Mager, A. Ebrahimi, A. Oncevay, E. Rice, S. Rijhwani, A. Palmer, and K. Kann, Eds. Toronto, Canada: Association for Computational Linguistics, Jul. 2023, pp. 47–57. [Online]. Available: https://aclanthology.org/2023. americasnlp-1.7/ [29] J. Han and L. Yang, “Sentence embedding generation framework based on kullback–leibler divergence optimization and roberta knowledge distillation,” Mathematics, vol. 12, no. 24, p. 3990, Dec. 2024. [Online]. Available: http://dx.doi.org/10.3390/math12243990 [30] S. G. Tesfagergish, J. Kapočiūtė-Dzikienė, and R. Damaševičius, “Zero-shot emotion detection for semi-supervised sentiment analysis using sentence transformers and ensemble learning,” Applied Sciences, vol. 12, no. 17, 2022. [Online]. Available: https://www.mdpi.com/ 2076-3417/12/17/8662 [31] M. Shafikuzzaman, M. R. Islam, A. C. Rolli, S. Akhter, and N. Seliya, “An empirical evaluation of the zero-shot, few-shot, and traditional fine-tuning based pretrained language models for sentiment analysis in software engineering,” IEEE Access, vol. 12, pp. 109 714–109 734, 2024. [32] Z. Li, D. Zhu, Q. Ma, W. Xiong, and S. Li, “Eerpd: Leveraging emotion and emotion regulation for improving personality detection,” 2024. [Online]. Available: https://arxiv.org/abs/2406.16079 [33] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample, “Llama: Open and efficient foundation language models,” 2023. [Online]. Available: https: //arxiv.org/abs/2302.13971 [34] A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. l. Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M.-A. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed, “Mistral 7b,” 2023. [Online]. Available: https://arxiv.org/abs/2310.06825 [35] J. Bai et al., “Qwen technical report,” 2023. [Online]. Available: https://arxiv.org/abs/2309.16609 [36] OpenAI et al., “Gpt-4 technical report,” 2023. [Online]. Available: https://arxiv.org/abs/2303.08774 [37] T. Zhong et al., “Evaluation of openai o1: Opportunities and challenges of agi,” 2024. [Online]. Available: https://arxiv.org/abs/2409.18486

15

[38] DeepSeek-AI et al., “Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,” 2025. [Online]. Available: https://arxiv.org/abs/2501.12948 [39] J. J. Tan, B.-H. Kwan, D. W.-K. Ng, Y.-C. Hum, N. Kawarazaki, K. Takano, and A. Mokraoui, “Cross-lingual attention distillation with personality-informed generative augmentation for multilingual personality recognition,” IEEE Transactions on Cognitive and Developmental Systems, p. 1–16, 2026. [Online]. Available: http://dx.doi.org/10.1109/TCDS.2026.3682672 [40] J. J. Tan, B.-H. Kwan, D. W.-K. Ng, and Y.-C. Hum, “Adaptive focal loss with personality stratification for stably mitigating hard class imbalance in multi-dimensional personality recognition,” Scientific Reports, vol. 15, no. 1, Nov. 2025. [Online]. Available: http: //dx.doi.org/10.1038/s41598-025-22853-y [41] A. Khattak, N. Jellani, M. Z. Asghar, and U. Asghar, “Personality classification from text using bidirectional long short-term memory model,” Multimedia Tools and Applications, vol. 83, no. 10, p. 28849–28873, Sep. 2023. [Online]. Available: http://dx.doi.org/10.1007/ s11042-023-16661-7 [42] Z. Guan, B. Wu, B. Wang, and H. Liu, “Personality2vec: Network representation learning for personality,” in 2020 IEEE Fifth International Conference on Data Science in Cyberspace (DSC), 2020, pp. 30–37. [43] E. Kerz, Y. Qiao, S. Zanwar, and D. Wiechmann, “Pushing on personality detection from verbal behavior: A transformer meets text contours of psycholinguistic features,” in Proceedings of the 12th Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis, J. Barnes, O. De Clercq, V. Barriere, S. Tafreshi, S. Alqahtani, J. Sedoc, R. Klinger, and A. Balahur, Eds. Dublin, Ireland: Association for Computational Linguistics, May 2022, pp. 182–194. [Online]. Available: https://aclanthology.org/2022.wassa-1.17/ [44] M. Ramezani, M.-R. Feizi-Derakhshi, and M.-A. Balafar, “Text-based automatic personality prediction using kgrat-net: a knowledge graph attention network classifier,” Scientific Reports, vol. 12, no. 1, Dec. 2022. [Online]. Available: http://dx.doi.org/10.1038/s41598-022-25955-z [45] S. S. Bajestani, M. M. Khalilzadeh, M. Azarnoosh, and H. R. Kobravi, “Transentgat: A sentiment-based lexical psycholinguistic graph attention network for personality prediction,” IEEE Access, vol. 12, pp. 59 630– 59 642, 2024. [46] M. Wu, Z. Xu, and L. Zheng, “Heterogeneous graph contrastive learning with adaptive data augmentation for semi-supervised short text classification,” Expert Systems, vol. 42, no. 2, Oct. 2024. [Online]. Available: http://dx.doi.org/10.1111/exsy.13744 [47] H. Zhu, X. Zhang, J. Lu, Y. Wu, Z. Bai, C. Min, L. Yang, B. Xu, D. Zhang, and H. Lin, “Enhancing textual personality detection toward social media: Integrating long-term and short-term perspectives,” 2024. [Online]. Available: https://arxiv.org/abs/2404.15067 [48] Y. Ji, W. Wu, H. Zheng, Y. Hu, X. Chen, and L. He, “Is chatgpt a good personality recognizer? a preliminary study,” 2023. [Online]. Available: https://arxiv.org/abs/2307.03952 [49] J. J. Tan, B.-H. Kwan, D. W.-K. Ng, Y.-C. Hum, A. Mokraoui, and S.-Y. Lo, “Prompting-in-a-series: Psychology-informed contents and embeddings for personality recognition with decoder-only models,” IEEE Transactions on Computational Social Systems, p. 1–15, 2025. [Online]. Available: http://dx.doi.org/10.1109/TCSS.2025.3593323 [50] J. W. Pennebaker and L. A. King, “Linguistic styles: Language use as an individual difference.” Journal of Personality and Social Psychology, vol. 77, pp. 1296–1312, 1999. [51] M. J., “(mbti) myers-briggs personality type dataset,” 2017. [Online]. Available: https://www.kaggle.com/datasets/datasnaek/mbti-type [52] I. Zettler, I. Thielmann, B. E. Hilbig, and M. Moshagen, “The nomological net of the hexaco model of personality: A large-scale meta-analytic investigation,” Perspectives on Psychological Science, vol. 15, no. 3, p. 723–760, Apr. 2020. [Online]. Available: http://dx.doi.org/10.1177/1745691619895036 [53] I. Thielmann, M. Moshagen, B. Hilbig, and I. Zettler, “On the comparability of basic personality models: Metaanalytic correspondence, scope, and orthogonality of the big five and hexaco dimensions,” European Journal of Personality, vol. 36, no. 6, p. 870–900, Jun. 2021. [Online]. Available: http://dx.doi.org/10.1177/08902070211026793 [54] I. Beltagy, M. E. Peters, and A. Cohan, “Longformer: The longdocument transformer,” 2020. [Online]. Available: https://arxiv.org/abs/ 2004.05150 [55] Y. Mehta, S. Fatehi, A. Kazameini, C. Stachl, E. Cambria, and S. Eetemadi, “Bottom-up and top-down: Predicting personality with psycholinguistic and language model features,” in 2020 IEEE International Conference on Data Mining (ICDM), 2020, pp. 1184–1189.

[56] T. Yang, F. Yang, H. Ouyang, and X. Quan, “Psycholinguistic tripartite graph network for personality detection,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), C. Zong, F. Xia, W. Li, and R. Navigli, Eds. Online: Association for Computational Linguistics, Aug. 2021, pp. 4229–4239. [Online]. Available: https: //aclanthology.org/2021.acl-long.326 [57] L. Hu, H. He, D. Wang, Z. Zhao, Y. Shao, and L. Nie, “Llm vs small model? large language model based text augmentation enhanced personality detection model,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 16, pp. 18 234–18 242, Mar. 2024. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/ view/29782 [58] T. Yang, J. Deng, X. Quan, and Q. Wang, “Orders are unwanted: Dynamic deep graph convolutional network for personality detection,” 2022. [Online]. Available: https://arxiv.org/abs/2212.01515

AUTHOR B IOGRAPHIES

Jing Jie Tan received his Bachelor of Computer Science (Hons) and PhD in Engineering, specializing in machine learning, from Universiti Tunku Abdul Rahman (UTAR). He is currently a Research Fellow at the National University of Singapore (NUS). His research interests include affective computing, health intelligence, natural language processing, computer vision, deep learning, large language models, and vision transformers. He is passionate about translating research into practical applications, with the goal of leveraging technology to better serve society.

Ban-Hoe Kwan received the Bachelor of Engineering (Electrical), Master of Engineering Science, and Ph.D. degrees in Engineering from the University of Malaya (UM). He is currently an Associate Professor at Universiti Tunku Abdul Rahman (UTAR). His research interests include image processing, artificial intelligence, medical signal processing, the Internet of Things, and robotics.

Danny Ng Wee Kiat (Senior Member, IEEE) received the Ph.D. degree in Engineering and is a registered Professional Engineer with the Board of Engineers Malaysia. He is currently an Assistant Professor at Universiti Tunku Abdul Rahman (UTAR). His research focuses on robotics and artificial intelligence, particularly AI-driven robotics, generative AI, and autonomous AI agents for industrial and enterprise applications. In addition to his academic role, he serves as the Chief Executive Officer and Chief Technology Officer of Netizen Robotics and as the Technical Director of Netizen Experience, where he leads initiatives in advanced robotics and digital transformation.

16

Yan Chai Hum (Senior Member, IEEE) received the Ph.D. degree in Engineering with specialization in artificial intelligence from Universiti Teknologi Malaysia. He is currently an Associate Professor with the Department of Mechatronics and Biomedical Engineering, Lee Kong Chian Faculty of Engineering and Science, Universiti Tunku Abdul Rahman (UTAR). His research interests include artificial intelligence in healthcare, biomedical imaging, computer vision, Internet of Things systems, and intelligent sensing technologies. His recent work focuses on AI-driven diagnostic systems, multimodal medical data analysis, and autonomous sensing platforms. He is also the founder of Promptiq Enterprise, an AI consultancy specializing in generative AI and intelligent systems.

Shih-Yu Lo is an Associate Professor at the Institute of Communication Studies, National Yang Ming Chiao Tung University, Taiwan. His research integrates cognitive psychology and human–computer interaction, focusing on how emerging technologies such as AI, virtual reality, and social robots shape memory, decision-making, empathy, and social cognition.

Po-An Chen is a Professor and Director at the Institute of Information Management, National Yang Ming Chiao Tung University, Taiwan. He is generally interested in economics and computation, artificial intelligence, and operations research, specifically including algorithmic game theory, theoretical online/reinforcement learning, social networks, and multiagent and distributed systems.

Noriyuki Kawarazaki received the Ph.D. degree in Engineering from Kyushu University. He is currently a Professor and Department Chair of Information Systems at Kanagawa Institute of Technology, Japan. His research interests include human-robot interaction, image processing, and artificial intelligence in robotics.

Kosuke Takano received a B.A. degree in Environment and Information Studies from Keio University, Japan, and his M.A. and Ph.D. degrees in Media and Governance from Keio University. He is currently a Professor in the Department of Information and Computer Sciences at Kanagawa Institute of Technology, Japan. His research interests include emotional AI and multimodal affective computing, data management systems for real-world monitoring, and AI-driven educational systems.

Anissa Mokraoui received the state engineering degree in electrical engineering from national school of telecommunications in 1989 from Algeria, the M.S degree in information technology in 1990 and the Ph.D. degree in 1994 both from University Paris 11, Orsay France. From 1992 to 1994, she worked at the National Institute of Telecommunications (INT, at Evry France) where her research activities were on digital signal processing, fast filtering algorithms and implementation problems on DSP. In 1997, she was appointed as assistant professor and in December 2011 as associate professor at Galilé Institute of University Paris 13, France. Since 2013, she is full professor at Galilée Institute of Université Sorbonne Paris Nord (USPN), France. From 2016 to 2024, she was the director of the L2TI laboratory of USPN. Her current research interests include source coding (image, video, multi-view, stereoscopic); joint source-channel-protocol decoding, robust mobile transmission, MIMO-OFDM channel estimation (massive), computer vision, few-shot object detection, cross-domain. She is co-author of more than 150 contributions to journals and conference proceedings. She served on program committees for conferences. She acts as a reviewer for many IEEE and EURASIP conferences and journals.