Human-Centric Topic Modeling with Goal-Prompted Contrastive Learning and Optimal Transport

Human-Centric Topic Modeling with Goal-Prompted Contrastive Learning and Optimal Transport Rui Wang† , Yi Zheng† , Dongxin Wang† , Haiping Huang†∗ , Yuanzhi Yao‡ , Yuxiang Zhou♦ , Jialin Yu♥ , Philip Torr♥ † School of Computer Science, Nanjing University of Posts and Telecommunications ‡ School of Computer Science and Information Engineering, Hefei University of Technology

arXiv:2604.12663v1 [cs.AI] 14 Apr 2026

♦ School of Electronic Engineering and Computer Science, Queen Mary University of London

♥ Department of Engineering Science, University of Oxford email:{rui_wang, 1223045708, 1224045815, hhp}@njupt.edu.cn, yaoyz@hfut.edu.cn, yuxiang.zhou@qmul.ac.uk, {jialin.yu, philip.torr}@eng.ox.ac.uk ❌Goal-biased Topics

Abstract Existing topic modeling methods, from LDA to recent neural and LLM-based approaches, which focus mainly on statistical coherence, often produce redundant or off-target topics that miss the user’s underlying intent. We introduce “Human-centric Topic Modeling” (Human-TM), a novel task formulation that integrates a humanprovided goal directly into the topic modeling process to produce interpretable, diverse and goal-oriented topics. To tackle this challenge, we propose the Goal-prompted Contrastive Topic Model with Optimal Transport (GCTM-OT), which first uses LLM-based prompting to extract goal candidates from documents, then incorporates these into semantic-aware contrastive learning via optimal transport for topic discovery. Experimental results on three public subreddit datasets show that GCTM-OT outperforms state-of-theart baselines in topic coherence and diversity while significantly improving alignment with human-provided goals, paving the way for more human-centric topic discovery systems.

CCS Concepts • Information systems → Document topic models.

Keywords Human-Centric Topic Modeling, Opinion Mining, Contrastive Learning, Large Language Models ACM Reference Format: Rui Wang† , Yi Zheng† , Dongxin Wang† , Haiping Huang†∗ , Yuanzhi Yao‡ , Yuxiang Zhou♦ , Jialin Yu♥ , Philip Torr♥ . 2026. Human-Centric Topic Modeling with Goal-Prompted Contrastive Learning and Optimal Transport. In . ACM, New York, NY, USA, 11 pages. https://doi.org/10.1145/nnnnnnn. nnnnnnn * Corresponding author.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. Conference’17, Washington, DC, USA © 2026 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-x-xxxx-xxxx-x/YYYY/MM https://doi.org/10.1145/nnnnnnn.nnnnnnn

Corpus

LDA[2003], CAST[2025],etc. Modeling without Goals

whatsbotheringyou What's the thing that's bothering you ? Our GCTM-OT

Human

Human-Centric Topic Modeling

Limitations

Topic 1: house, sister, father, brother... Topic 2: wife, father, marry, house... Topic 3: happy, enjoy, hope, change... Topic 4: college, class, study, learn... ...

👍Goal-oriented Topics

Topic Redundancy Not Related to Goal Low Interpretability

Summary Explanation

[Health Issues] Topic 1: cancer, surgery, medical, injury... Topic 2: cheat, breakup, girlfriend, betray... [Breakup struggles] Topic 3: passion, motivation, purpose, success... [Life Direction]

Topic 4: exam, academic, grade, assignment... [Academic Stress] ...

Figure 1: Key differences between Traditional Topic Modeling and Human-Centric Topic Modeling (with human’s goal).

1

Introduction

Topic models [1, 46], such as the Latent Dirichlet Allocation (LDA)[3] and advanced variants including the Corpus-Aware Self-similarity enhanced Topic Model (CAST) [21], have been widely adopted for open information extraction [40, 41] and knowledge discovery [36, 49]. These approaches uncover coherent semantic patterns from large text corpora without supervision. However, when the analytical objective is guided by a specific human goal, these methods often fall short because they are not designed to adapt topic discovery to user-defined intentions without extensive manual intervention. Consider the example in Figure 1, where the human goal, expressed in question "What’s the thing that’s bothering you?", is to identify the main issues of concern within a collection of text responses. Traditional topic models, which are optimized solely for statistical coherence, often produce topics that, although internally consistent, are redundant, less interpretable, or unrelated to the goal. For example, topics containing terms such as ‘house’, ‘sister’, ‘father’ or ‘happy’, ‘enjoy’, ‘hope’ are coherent in a statistical sense but do not address the intended analytical objective. As a result, important patterns such as ‘breakup struggles’, ‘health issue’ or ‘academic stress’ may be overlooked or obscured, requiring substantial manual filtering and interpretation. To address these limitations, we propose a new task formulation called Human-centric Topic Modeling (Human-TM). As illustrated in Figure 1, given a text corpus (e.g. Subreddit corpus ‘Whatsbotheringyou’) 1 and a human-provided free-text goal (e.g. “What’s the thing that’s bothering you?”), Human-TM aims to automatically generate a set of goal-oriented topics. These topics should satisfy the 1 https://www.reddit.com/r/whatsbotheringyou/

Conference’17, July 2017, Washington, DC, USA

following three requirements: 1). Topics are semantically relevant to the human-provided goal. 2). Topics are semantically coherent and easily interpretable. 3). Topics are diverse (non-redundant) and collectively provide broad coverage of the corpus. To tackle the Human-TM task, we propose the Goal-prompted Contrastive Topic Model with Optimal Transport (GCTM-OT), which formulates topic modeling as a representation learning procedure based on advanced prompt engineering [10] and optimal transport theory [25]. The core idea is to incorporate goal information, identified through LLM-based prompting from texts, into the modeling process for goal-oriented topic extraction. Specifically, GCTM-OT first generates phrase-style goal candidates for each document using a goal-oriented summarization prompt and aggregates them into a corpus-level goal set. To improve topic interpretability, it integrates contextualized word representations from a transformer [9] to build contextualized document representations and models topics with topic representations to infer documenttopic distributions. Moreover, to embed goal information and encourage topic diversity, it forms goal representations by clustering goal candidates in the corpus-level goal set and employs optimal transport [25] to guide topic representation training. After training with the supervision signals provided by contrastive learning and optimal transport, GCTM-OT is capable of identifying interpretable and diverse goal-oriented topics. Additionally, it could generate a phrase-style topic summary for each topic, offering document-level topic interpretation. The main contributions of this work are summarized as: • We introduce Human-TM, a novel topic modeling formulation that takes human goals into account and generates interpretable, diverse and goal-oriented topics. • We propose the novel Goal-prompted Contrastive Topic Model with Optimal Transport (GCTM-OT), which leverages LLM-based prompting to identify goal candidates within texts and incorporates them into the modeling process via contrastive learning and optimal transport, enabling goaloriented topic extraction. • We evaluate GCTM-OT on three Subreddit datasets. Experimental results show that GCTM-OT outperforms state-ofthe-art baselines in topic coherence and diversity. In addition, it generates topics that closely align with human goals, as measured by Goal Similarity (𝐺𝑆), Goal-relevant Topic Rate (𝐺𝑇 𝑅) and Goal Coverage Rate (𝐺𝐶𝑅) metrics.

2

Trovato et al.

network to capture semantic patterns. Inspired by NVDM, Xu et al. proposed the VONT [47] by employing a mixture of von-Mises Fisher distributions as topic prior. Ma et al. utilized the word embeddings to filter out irrelevant words and proposed the Corpus-Aware Self-similarity Topic (CAST) [21] model. Fang et al. incorporated the dynamic word representations into the modeling process and proposed the Contextualized Word Topic Model (CWTM) [11]. To mine the public’s opinion towards ChatGPT, Wang et al. employed a topic disentangle mechanism and proposed the Disentangled Contextualized Topic Model (DisCTM) [38]. However, all of these approaches are not able to incorporate the human’s goal into the modeling process, resulting in generic topics that do not match specific human needs.

2.2

Advances in Large Language Models (LLMs) have made them a thriving research topic and have been explored for their robust capabilities in NLP tasks [32, 44, 50]. Meanwhile, Prompt Engineering [30] has emerged as an indispensable approach to extend the capability of LLMs. Firstly, Radford et al. proposed the Zero-shot prompting [27] to offer a paradigm shift in leveraging LLMs. By providing several input-output examples to induce the target of the given task, Brown et al. proposed the Few-shot Prompting [4]. Along this line, Wei et al. introduced the Chain-of-Thought (CoT) [45] prompting to guide LLMs and form a step-by-step reasoning scheme. Recently, scholars have also explored using prompting for topic extraction. Pham et al. employed the LLM-prompting and proposed the TopicGPT [26] for topic phrase generation. Mu et al. proposed the Large Language Model Topic Extraction (LLM-TE) [23], which is an automatic topic generation framework based on LLM. Chang et al. proposed to improve topic quality by designing a topic refinement mechanism [6] with LLM-based prompting.

3

(1) Goal-Oriented: Each topic 𝑡𝑘 (𝑘 ∈ {1, 2, ..., 𝐾 }) must be semanticlly related to human’s goal. For example, if the goal H𝑔 =“What’s the thing that’s bothering you?”, extracted topics should be “Breakup Struggles”, “Acdamic Stress” and etc. (2) Interpretable: Each topic 𝑡𝑘 (𝑘 ∈ {1, 2, ..., 𝐾 }) must be represented by a list of semantically coherent words 𝑡𝑘𝑤 and a human-readable summary explanation 𝑡𝑘𝑠 , such as [‘cheat’, ‘breakup’, ‘girlfriend’, ‘betray’, ‘ring’, ‘break’, ‘divorce’, ‘relationship’, ‘partner’, ‘devastate’] and “breakup struggles”. (3) Diverse and Maximum Coverage: The topics in 𝑇 should be semantically diverse, and their union should cover the majority of goal candidates retained in the corpus.

Related Work

Neural Topic Modeling

Modeling topics using generative neural networks, such as Variational Autoencoders (VAE) [17] and Generative Adversarial Networks (GAN) [14], has emerged as an active area of research. Miao et al. proposed the Neural Variational Document Model (NVDM) [22], which is the pioneering work in this field, based on the VAE. Likewise, Wang et al. proposed the Adversarial-neural Topic Model (ATM) [39] and the Bidirectional Adversarial Topic (BAT) [37] model in an adversarial manner and utilized a generator

Human-TM Task Formulation

Given a collection of 𝑁 documents 𝐷 ={𝑥 1, 𝑥 2, ..., 𝑥 𝑁 } and a humanprovided goal description H𝑔 , human-TM aims to mine a set of 𝐾 topics 𝑇 ={𝑡 1, 𝑡 2, ..., 𝑡𝐾 } that satisfy the following requirements:

Our work is relevant to neural topic modeling and Large Language Model based prompting.

2.1

Large Language Model based Prompting

4

Methodology

As illustrated in Figure 2 (a), our proposed Goal-prompted Contrastive Topic Model with Optimal Transport (GCTM-OT) comprises five components: 1). Goal-oriented Summary Generation (topleft): Extracts human-goal–related phrases from each document

Human-Centric Topic Modeling with Goal-Prompted Contrastive Learning and Optimal Transport workflow of summary knowledge workflow of OT Alignment

Conference’17, July 2017, Washington, DC, USA

Detailed Design of Text Representation Procedure

Goal-oriented Summary Generation

Goal-oriented Optimal Transport Alignment

Document

Document-goal Distributions

Goal Clusters

Goal-candidates Set

GoalSummarization Prompt

Goal Representation Matrix

OT Alignment

.

Clustering

{

Summary

}:

"I'm trapped by student debt, supporting four post-dad's illness on minimum wage with crushing workload leaving no social life.

:

{Student debt limiting independence; Supporting family on minimum wage; No social life due to workload}

Pretrained Transformer

.

OT Alignment

...

...

Maximize Similarity

.

Student debt ...

0.1

0.7

0.1

0.2

0.1

0.3

Supporting family...

0.1

0.3

0.1

0.1

0.2

0.5

No social life...

0.1

0.3

0.1

0.3

0.3

0.7

Similarity Matrix

Topic Representation Matrix

...

Maximum and Normalize

Weight sum

...

Document-topic Distributions

Augmented Documents

n so o ci al lif e

.

...

Weight Sharing

I'm

...

Text Augmentation Prompt

tra pp e ... d

OT Cost Matrix

0.1

0.3

0.1

0.1

0.1

0.3

Attention Weight

Document Representations

Text Augmentation and Representation

Topic Inference and Prior Matching

Contrastive Learning

(a) Overall framework of Goal-prompted Contrastive Topic Model with Optimal Transport (GCTM-OT).

(b) Goal-oriented Text Representation Mechanism.

Figure 2: The framework of GCTM-OT (a) and details of Goal-oriented Text Representation Mechanism (b). so that later topic discovery stays focused on the target intent. It uses the goal summarization prompt P𝑠 to generate 𝐺𝑥 for each document 𝑥, which are then aggregated into the corpus-level goal set 𝐺. 2). Text Augmentation and Representation (bottom-left): Produces diverse yet semantically consistent views of each document so that contrastive learning is effective. Using the text augmentation prompt P𝑎 , it generates pair (𝑥 𝑎 , 𝑥 𝑏 ), then integrates goal information from 𝐺𝑥 into a transformer T to produce goal-oriented representations (ℎ®𝑎𝑥 , ℎ®𝑏𝑥 ). 3). Topic Inference and Prior Matching (bottommiddle): Maps goal-aware document representations to topic distributions while maintaining interpretability. It models topics using the trainable topic representation matrix 𝐸𝑡 and infers the document-topic distributions 𝜃®𝑎 and 𝜃®𝑏 for 𝑥 𝑎 and 𝑥 𝑏 , respectively. These distributions are then matched to a Dirichlet prior, leveraging its multiple-peak property to improve topic interpretability. 4). Semantic-aware Contrastive Learning (bottom-right): Encourages topic representations to capture distinctive semantic boundaries between documents. It optimizes 𝐸𝑡 through contrastive learning, bringing semantically similar document pairs closer and separating unrelated pairs in the learned topic space. 5). Goal-oriented Optimal Transport Alignment (top-right): Aligns the learned topics with human goals to ensure relevance and diversity of topics. It first clusters the corpus-level goal set 𝐺 to form a goal representation matrix 𝐸𝑔 , infers document–goal distributions (𝜆®𝑎 , 𝜆®𝑏 ), and then applies optimal transport to align the document–topic distributions (𝜃®𝑎 , 𝜃®𝑏 ) with the corresponding document–goal distributions (𝜆®𝑎 , 𝜆®𝑏 ), directly injecting goal structure into topic learning. Meanwhile, Figure 2 (b) illustrates the details of the goal-oriented text representation mechanism. The functionalities of these components are described in detail below.

4.1

Goal-oriented Summary Generation

To ensure semantic alignment between the mined topics and the human-provided goal H𝑔 , we first identify and extract potential goal candidates from the corpus. For each document 𝑥 in the corpus 𝐷, we use an LLM [12] with a goal-summarization prompt P𝑠 to 𝑁 generate document-specific goal candidates 𝐺𝑥 = {𝑔𝑥1 , 𝑔𝑥2 , ..., 𝑔𝑥 𝑔𝑥 }.

Detailed Prompt for Goal-oriented Summarization <s>[INST] messages: [ {role: system, content: Act as a subreddit content framework designer. Your task is to identify documents (posts) in "r/{subreddit_name}" that align with the goal: "{goal}".}, {role: user, content: Process this document in two steps: Step 1:Relevance Check -If the text is **NOT relevant** to the goal, output exactly: `[irrelevant]` -If relevant, proceed to Step 2. Step 2 - Core Extraction -Extract 3-5 Core Elements from this text related to the goal({goal}): Format: {{elements1|elements2|...}}.each elements follows "<Entity Action/Status>" structure, without explanations information.}, {role: assistant, content: Sure, please provide the document.}, {role: user, content: {Text}} ] Output: [irrelevant] / {element1|element2|element3} [/INST]

Figure 3: Details of Goal Summarization Prompt P𝑠 . Detailed Prompt for Text Augmentation <s>[INST] messages: [ {role: system, content: You are a helpful assistant that extract critical information, rephrase text and make sentence smooth.}, {role: user, content: I will give you a document . Please rephrase it as little as possible, ensure the output starts with the prefix 'REPHRASED:' and is free of any additional labels, headers, or explanatory text.}, {role: assistant, content: Sure, please provide the sentence you would like me to rephrase.}, {role: user, content: {Text}} ] [/INST] Output: <Augmented Text>

Figure 4: Details of Text Augmentation Prompt P𝑎 .

Here, 𝑁𝑔𝑥 denotes the number of goals in document 𝑥, ranging from 3 to 5 in our experiments. If a document is deemed irrelevant to the goal H𝑔 , it is annotated as ‘irrelevant’ and excluded from the corpus. The details of the prompt P𝑠 are shown in Figure 3, which follows the OpenAI-released API guidelines2 . We then aggregate document-specific goal sets 𝐺𝑥𝑛 (𝑛 ∈ {1,2,. . . ,𝑁 }) to construct the corpus-level goal set 𝐺 = 𝐺𝑥1 ∪ 𝐺𝑥2 ∪ · · · ∪ 𝐺𝑥𝑁 , which is later used to produce the goal representation matrix 𝐸𝑔 (Section 4.5). 2 https://platform.openai.com/docs/guides/prompt-engineering

Conference’17, July 2017, Washington, DC, USA

Trovato et al.

where 𝑒®𝑛′ (𝑛 ∈ {1, 2, ..., 𝑁𝑥′ }) denotes the contextualized word repre-

representation matrix 𝐸𝑡 ∈ R𝐾 ×𝐻 , where 𝐻 denotes the dimensionality of the transformer representation. Specifically, for the augmented document 𝑥 𝑎 , GCTM-OT infers its document-topic distribution 𝜃®𝑎 with: 𝜃®𝑎 = softmax([cos(ℎ®𝑎 , 𝑒®𝑡1 ), cos(ℎ®𝑎 , 𝑒®𝑡2 ), ..., cos(ℎ®𝑎 , 𝑒®𝑡𝐾 )]) (6) where 𝑒®𝑡𝑘 (𝑘 ∈ {1, 2, ..., 𝐾 }) represents the topic representation of the 𝑘-th topic. Likewise, the document-topic distribution of 𝑥 𝑏 could be obtained by feeding it with ℎ®𝑏 . Moreover, to ensure topic interpretability, we match the inferred document-topic distributions to the Dirichlet prior via Maximum Mean Discrepancy (MMD) [15]. Specifically, given two batches of inferred document-topic distributions Θ ={𝜃®1, 𝜃®2, ..., 𝜃®2𝑀 } =Θ𝑎 ∪ Θ𝑏 = 𝑎 , 𝜃®𝑏 , 𝜃®𝑏 , ..., 𝜃®𝑏 } and two batches of random samples {𝜃®1𝑎 , 𝜃®2𝑎 , ..., 𝜃®𝑀 1 2 𝑀 ′ ′ ′ ′ } drawn from the Dirichlet prior Dir(𝜃®′ |𝛼), Θ = {𝜃®1, 𝜃®2, ..., 𝜃®2𝑀 ® we match the inferred document-topic distributions Θ to the Θ′ with the formula: 1 ∑︁ ® ® 1 ∑︁ ® ®′ 𝑀𝑀𝐷 (Θ, Θ′ ) = [k(𝜃 𝑖 , 𝜃 𝑗 ) +k(𝜃®𝑖′, 𝜃®′𝑗 )] − k(𝜃 𝑖 , 𝜃 𝑗 ) (7) 𝐴 𝑖≠𝑗 2𝑀 2 𝑖,𝑗

sentation of the 𝑛-th word in the document. Next, we encode the document-specific goal candidates with:

where 𝐴 =2𝑀 (2𝑀 − 1), 𝑀 denotes the batch size, k(·, ·) is the kernel function [33], and 𝛼® is the hyperparameter of the Dirichlet prior.

4.2

Text Augmentation and Representation

To ensure that the augmented documents preserve the semantic meaning of the original documents, which is critical for effective contrastive representation learning [42], we propose using a text augmentation prompt P𝑎 to generate the augmented documents. Specifically, for each document 𝑥 in the batch 𝑋 , we employ a text augmentation prompt P𝑎 to generate semantically consistent documents 𝑥 𝑎 and 𝑥 𝑏 by rephrasing 𝑥 using a large language model. The details of the augmentation prompt P𝑎 are illustrated in Figure 4, which also follows the OpenAI released API guideline 2 . Then, for each augmented document 𝑥 ′ ∈ {𝑥 𝑎 , 𝑥 𝑏 }, we generate a goal-oriented contextualized representation ℎ®𝑥′ using a transformer [9] T (Figure 2b). This model integrates the contextual information from the document with its corresponding goal can𝑁 didates 𝐺𝑥 = {𝑔𝑥1 , 𝑔𝑥2 , . . . , 𝑔𝑥 𝑔 }. Specifically, we first represent the document 𝑥 ′ = [𝑤 1′ , 𝑤 2′ , . . . , 𝑤 𝑁′ ′ ] using transformer T : 𝑥

[® 𝑒 1′ , 𝑒®2′ , ..., 𝑒®𝑁′ ′ ] = T ([𝑤 1′ , 𝑤 2′ , ..., 𝑤 𝑁′ ′ ]) 𝑥

′

′

(1)

𝑥

′𝑁

[® 𝑒𝑔1 ; 𝑒®𝑔2 ; ...; 𝑒®𝑔 𝑔𝑥 ] = [ T̄ (𝑔𝑥1 ); T̄ (𝑔𝑥2 ); ...; T̄ (𝑔𝑥 𝑔𝑥 )] 𝑁

(2)

′

where 𝑒®𝑔𝑖 (𝑖 ∈ {1, 2, . . . , 𝑁𝑔𝑥 }) is the contextualized embedding of the 𝑖-th goal 𝑔𝑥𝑖 , and T̄ (·) denotes the mean of token embeddings produced by the model T for that goal phrase. Then, for the 𝑛th (𝑛 ∈ {1, 2, ..., 𝑁𝑥′ }) word 𝑤𝑛′ , we could calculate its semantic correlations with each goal candidate and assign its attention weight 𝑎𝑛 as the maximum cosine similarity: ′

′

′𝑁

𝑎𝑛 = max( [cos(® 𝑒𝑛′ , 𝑒®𝑔1 ), cos(® 𝑒𝑛′ , 𝑒®𝑔2 ), ..., cos(® 𝑒𝑛′ , 𝑒®𝑔 𝑔 )]) (3) where cos(·, ·) means the cosine similarity. The use of the maximum operation ensures that each word is associated with its most relevant goal, preserving the strongest semantic link without dilution from less-related goals. This highlights words with at least one strong goal alignment while ignoring weaker associations. Finally, we obtain the contextual representation ℎ®𝑥′ of document 𝑥 ′ with: [𝑎ˆ1, 𝑎ˆ2, ..., 𝑎ˆ𝑁𝑥′ ] =softmax([𝑎 1, 𝑎 2, ..., 𝑎 𝑁𝑥′ ]) (4) ∑︁𝑁𝑥′ ℎ®𝑥′ = 𝑎ˆ𝑛 · 𝑒®𝑛′ (5) 𝑛=1

where softmax(·) means the softmax normalization. Thus, we could generate contextualized document representations ℎ®𝑎 and ℎ®𝑏 for the LLM-augmented documents 𝑥 𝑎 and 𝑥 𝑏 . By incorporating contextual and goal candidate information in this way, these representations emphasize goal-relevant semantics and facilitate the extraction of interpretable and goal-oriented topics.

4.4

Semantic-aware Contrastive Learning

To enable the topic representation matrix 𝐸𝑡 to capture semantic patterns and group semantically similar words into clusters, we train GCTM-OT using semantic-aware contrastive learning [7]. Concretely, given two batches of inferred document-topic dis𝑎 , 𝜃®𝑏 , 𝜃®𝑏 , ..., 𝜃®𝑏 } from augtributions Θ = Θ𝑎 ∪ Θ𝑏 = {𝜃®1𝑎 , 𝜃®2𝑎 , ..., 𝜃®𝑀 1 2 𝑀 𝑎 𝑏 𝑎 𝑎 , ..., 𝑥 𝑏 , 𝑥 𝑏 , ..., 𝑥 𝑏 }, mented documents 𝑋 = 𝑋 ∪ 𝑋 = {𝑥 1 , 𝑥 2𝑎 , ..., 𝑥 𝑀 1 2 𝑀 for each document-topic distribution 𝜃®𝑖𝑎 , we first select the corresponding 𝜃®𝑖𝑏 to form the positive pair (𝜃®𝑖𝑎 , 𝜃®𝑖𝑏 ). Then, we select 𝜃®¬ ∈ Θ from a semantically irrelevant 𝑥 ¬ ∈ 𝑋 to form negative pairs (𝜃®𝑖𝑎 , 𝜃®¬ ). Here, 𝜃®¬ should satisfy the following requirements: 1). 𝜃®¬ ∉ {𝜃®𝑖𝑎 , 𝜃®𝑖𝑏 }. 2). cos( T̂ (𝑥𝑖𝑎 ), T̂ (𝑥 ¬ )) < 𝛿, where T̂ (·) denotes the document representation obtained by averaging the contextualized word representations produced by the transformer T , 𝛿 is the similarity threshold hyperparameter, set to 0.6 in the experiments. Thus, the contrastive objective of the augmented document 𝑥𝑖𝑎 could be computed with: ®𝑎 ®𝑏

𝑙𝑖𝑎 =−log

e [s(𝜃𝑖 ,𝜃𝑖 )/𝜏 ] 𝑀 Í [s(𝜃®𝑎,𝜃®𝑎 )/𝜏] [s(𝜃®𝑎,𝜃®𝑏 )/𝜏] Δ+ [ind(𝜃®𝑖𝑎,𝜃®𝑎𝑗 )e 𝑖 𝑗 +ind(𝜃®𝑖𝑎,𝜃®𝑏𝑗 )e 𝑖 𝑗 ] 𝑗=1 ®𝑎 ®𝑏

4.3

Topic Inference and Prior Matching

Given the contextualized document representations (ℎ®𝑎 and ℎ®𝑏 ) from the previous stage, this module aims to infer document-topic distributions (𝜃®𝑎 and 𝜃®𝑏 ) for the augmented documents (𝑥 𝑎 and 𝑥 𝑏 ). Additionally, it matches the inferred document-topic distributions to the Dirichlet prior [19], leveraging the multiple peaks property of its density to improve topic interpretability [35]. To capture semantic patterns in the contextualized document representations, GCTM-OT models topics using a trainable topic

(8)

where s(·, ·) is the cosine similarity, Δ = e [s(𝜃𝑖 ,𝜃𝑖 )/𝜏 ] , and 𝜏 represents the temperature parameter (set to 0.05), the indicator function ind(𝜃®𝑖𝑢 , 𝜃®𝑣𝑗 ) is defined as: ( 1, 𝑖 ≠ 𝑗 𝑎𝑛𝑑 cos( T̂ (𝑥𝑖𝑢 ), T̂ (𝑥 𝑣𝑗 )) < 𝛿 𝑢 ®𝑣 ® ind(𝜃 𝑖 , 𝜃 𝑗 ) = (9) 0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 where 𝑢, 𝑣 ∈ {𝑎, 𝑏}. Overall, the contrastive objective of two batches of augmented documents 𝑋 = 𝑋 𝑎 ∪ 𝑋 𝑏 could be computed with: 𝑀 1 ∑︁ 𝑎 𝑏 𝐿𝑋 = (𝑙 + 𝑙𝑖 ) (10) 2𝑀 𝑖=1 𝑖 where 𝑀 denotes the batch size.

Human-Centric Topic Modeling with Goal-Prompted Contrastive Learning and Optimal Transport

4.5

Goal-oriented Optimal Transport Alignment

To inject goal information into the learned topics more directly and ensure topic diversity, we identify key goal candidates from the corpus-level goal set 𝐺 via clustering and use them to guide model training through goal-oriented optimal transport alignment. Specifically, given the constructed goal candidates set 𝐺, we first identify a set of 𝐾 key goals by grouping them into clusters with: [𝐶 1, 𝐶 2, ..., 𝐶𝐾 ] = KMeans( T̄ (𝐺)) (11) where T̄ (·) denotes the mean of token embeddings produced by T for goal candidates in 𝐺. 𝐶𝑘 (𝑘 ∈ {1, 2, . . . , 𝐾 }) represents the 𝑘-th cluster of goal candidates. For each cluster 𝐶𝑘 , we use its centroid to form the 𝑘-th goal representation 𝑒®𝑔𝑘 ∈ 𝐸𝑔 . Here, 𝐸𝑔 ∈ R𝐾 ×𝐻 denotes the goal representation matrix, 𝐻 is the dimensionality of the transformer representations. Then, for each augmented document 𝑥 𝑎 , we infer the corresponding document-goal distribution 𝜆®𝑎 with: 𝜆®𝑎 = softmax( [cos(ℎ®𝑎 , 𝑒®1 ), cos(ℎ®𝑎 , 𝑒®2 ), ..., cos(ℎ®𝑎 , 𝑒®𝐾 )]) (12) 𝑔

𝑔

𝑔

where ℎ®𝑎 denotes the contextualized representation of the augmented document 𝑥 𝑎 , softmax(·) and cos(·, ·) denote the softmax normalization and cosine similarity. Thus, to inject the goal information into the modeling process and influence the learning of the topic representation matrix 𝐸𝑡 , we align the document-topic distribution 𝜃®𝑎 with the documentgoal distribution 𝜆®𝑎 using optimal transport (OT) [25], where the distance 𝑑 M̂ (𝜃®𝑎 , 𝜆®𝑎 ) is formulated as: 𝑑 (𝜃®𝑎 , 𝜆®𝑎 ) = min ⟨P, M̂⟩ (13) M̂

P∈𝑈 (𝜃®𝑎 ,𝜆®𝑎 )

where ⟨·, ·⟩ means the Frobenius dot-product, P ∈ R𝐾>0×𝐾 is the transport matrix, 𝑈 (𝜃®𝑎 , 𝜆®𝑎 ) is the transport polytope of 𝜃®𝑎 and 𝜆®𝑎 , which is the polyhedral set of 𝐾 × 𝐾 matrix: 𝑈 (𝜃®𝑎 , 𝜆®𝑎 ) :={𝑃 ∈ R𝐾>0×𝐾 |𝑃 1®𝐾 = 𝜃®𝑎 , 𝑃 𝑇 1®𝐾 = 𝜆®𝑎 }, and 1®𝐾 is the 𝐾-dimensional vector of ones. M̂ ∈ R𝐾>0×𝐾 is the cost matrix of the transport, where the element 𝑚𝑖,𝑗 in the 𝑖-th row and 𝑗-th column is formed with: 𝑚𝑖,𝑗 = 1 − cos(® 𝑒𝑡𝑖 , 𝑒®𝑔𝑗 ) (14) where cos(·, ·) denotes cosine similarity, and 𝑚𝑖,𝑗 ∈ [0, 2], 𝑒®𝑡𝑖 and 𝑒®𝑔𝑗 denote the 𝑖-th topic representation and the 𝑗-th goal representation, respectively. Similarly, we could calculate the OT distance for the augmented document 𝑥 𝑏 using Eq. 13 by feeding in 𝜃®𝑏 and 𝜆®𝑏 . As direct computation via Eq. 13 is time-consuming for large-scale problems, we follow an entropy-regularized estimator [8] and utilize the Sinkhorn iteration [24] for the OT distance estimation. Overall, given two batches of document-topic distributions Θ = Θ𝑎 ∪ Θ𝑏 and the corresponding document-goal distributions Λ = Λ𝑎 ∪ Λ𝑏 , generated from the augmented documents 𝑋 𝑎 ∪ 𝑋 𝑏 , their optimal transport objective could be formulated as: 𝑀 1 ∑︁ 𝑂𝑇 (Θ, Λ) = [𝑑 (𝜃®𝑎 , 𝜆®𝑎 ) + 𝑑 M̂ (𝜃®𝑖𝑏 , 𝜆®𝑖𝑏 )] (15) 2𝑀 𝑖=1 M̂ 𝑖 𝑖 where 𝑀 denotes the batch size, 𝜃®𝑖𝑎 and 𝜃®𝑖𝑏 are the 𝑖-th documenttopic distributions in Θ𝑎 and Θ𝑏 , 𝜆®𝑖𝑎 and 𝜆®𝑖𝑏 denote the 𝑖-th documentgoal distributions in Λ𝑎 and Λ𝑏 .

4.6

Conference’17, July 2017, Washington, DC, USA

Training Objective and Learning Procedure

To extract goal-relevant, interpretable and diverse topics, our proposed GCTM-OT should take the following factors into account: • Maximizing the similarities between positive pairs 3 and minimizing similarities between negative pairs 4 using semanticaware contrastive learning, enabling topic representation matrix 𝐸𝑡 to capture semantic patterns. • Matching the inferred document-topic distributions Θ = Θ𝑎 ∪ Θ𝑏 to the Dirichlet prior Dir(𝜃®′ |𝛼), ® ensuring topic interpretability. • Aligning the inferred document-topic distributions 𝜃®𝑎 and 𝜃®𝑏 with the document-goal distributions 𝜆®𝑎 and 𝜆®𝑏 using optimal transport, ensuring both semantic relevance to the goals and topic diversity. Thus, we formulate the training objective of GCTM-OT as: L = L𝑐 + 𝜂L𝑃𝑀 + 𝜁 L𝑂𝑇 (16) where L𝑐 denotes the contrastive learning objective, computed using Eq. 10. L𝑃𝑀 and L𝑂𝑇 represent the prior matching and optimal transport objectives, calculated using Eq. 7 and Eq. 15. 𝜂 and 𝜁 are coefficient hyperparameters, we set them to 1.0 in our experiments. The detailed training procedure of GCTM-OT is shown in Algorithm 1. In our experiment, batch size 𝑀 is set to 32, learning rate 𝛼 1 is set to 2e-3, the optimal transport objective L𝑂𝑇 is computed using the GeomLoss 5 library with default configuration. GCTM-OT is optimized by Adam [16] optimizer. Algorithm 1 The training procedure of GCTM-OT. Input: Corpus 𝐷 , human’s goal H𝑔 , topic number 𝐾 , batch size 𝑀 , learning rate 𝛼 1 . 1: for each document 𝑥 in 𝐷 do 2: Generate the document-specific goal candidates set 𝐺𝑥 = P𝑠 (𝑥 ) . 3: Add goal candidates in 𝐺𝑥 to the corpus-level goal set 𝐺 . 4: end for 5: Randomly initialize the topic representation matrix 𝐸𝑡 . 6: Obtain goal representation matrix 𝐸𝑔 via Eq.11. 7: Calculate the cost matrix M̂ of optimal transport using Eq.14. 8: for each batch of documents 𝑋 in 𝐷 do 9: for each document 𝑥 in 𝑋 do 10: Generate the augmented document pair via 𝑥 𝑎 = P𝑎 (𝑥 ), 𝑥 𝑏 = P𝑎 (𝑥 ) . 11: Construct document representations ℎ®𝑎 and ℎ®𝑏 for 𝑥 𝑎 and 𝑥 𝑏 via Eq. 5. 12: Infer document-topic distributions 𝜃®𝑎 and 𝜃®𝑏 for 𝑥 𝑎 and 𝑥 𝑏 via Eq.6. 13: Infer document-goal distributions 𝜆®𝑎 and 𝜆®𝑏 for 𝑥 𝑎 and 𝑥 𝑏 via Eq.12. 14: end for 15: Calculate contrastive objective L𝑐 via Eq. 10. 16: Draw a set of 2𝑀 random samples Θ′ from 𝐷𝑖𝑟 (𝜃®′ | 𝛼® ) . 17: Calculate the prior matching objective L𝑃 𝑀 via Eq. 7. 18: Calculate the OT objective L𝑂𝑇 in Eq. 13 through Sinkhorn Iterations [25]. 19: Calculate the overall training objective L via Eq. 16. 20: Update topic representation matrix 𝐸𝑡 with gradient descent. 21: end for Output: Learned topic representation matrix 𝐸𝑡 .

4.7

Topic Extraction

Leveraging the learned topic representation matrix 𝐸𝑡 and the constructed goal representation matrix 𝐸𝑔 , we could extract goalrelevant topics represented by semantically coherent keywords and interpretable topic summaries. 3 Both (𝜃®𝑎 , 𝜃®𝑏 ) and (𝜃®𝑏 , 𝜃®𝑎 ). 4 Both (𝜃®𝑎 , 𝜃® ) and (𝜃®𝑏 , 𝜃® ). ¬ ¬ 5 https://www.kernel-operations.io/geomloss/

Conference’17, July 2017, Washington, DC, USA

Trovato et al.

Specifically, for the 𝑣-th (𝑣 ∈ {1, 2, ..., 𝑉 }) word 𝑤 𝑣 in the vocabulary, we first use the transformer T to generate the contextualized ′ word representations 𝑒®𝑛𝑣 (𝑛 ′ ∈ {1, 2, ..., 𝑁 𝑣′ }) of all its appearances ′ and collect them into the word representation matrix 𝐸 𝑣 ∈ R𝐻 ×𝑁 𝑣 . ′ Here, 𝑁 𝑣 denotes the number of times 𝑤 𝑣 appears in the corpus, 𝐻 is the dimensionality of word representations. We then compute the semantic correlations 𝑐®𝑣 between word 𝑤 𝑣 and topics using: ′ ′ ′ ′ 𝑐®𝑛𝑣 = softmax( [cos(® 𝑒 𝑛𝑣 , 𝑒®𝑡1 ), cos(® 𝑒 𝑛𝑣 , 𝑒®𝑡2 ), ..., cos(® 𝑒 𝑛𝑣 , 𝑒®𝑡𝐾 )]) (17) 1 ∑︁𝑁 𝑣′ 𝑛′ 𝑐®𝑣 = ′ 𝑐® (18) 𝑛 ′ =1 𝑣 𝑁𝑣 ′ 𝑛 where 𝑐®𝑣 denotes the local semantic correlations between the word 𝑤 𝑣 and topics in its 𝑛 ′ -th appearance, and 𝑒®𝑡𝑘 denotes the topic representation of the 𝑘-th topic. Similarly, we construct a correlation matrix Ĉ ∈ R𝐾 ×𝑉 , which stores the semantic correlations between topics and words in the vocabulary. The topic-word distribution 𝜙®𝑘 of the 𝑘-th topic (𝑘 ∈ {1, 2, . . . , 𝐾 }) is then computed with: 𝜙®𝑘 = norm(® 𝑐𝑘,· ) (19) where 𝑐®𝑘,· denotes the 𝑘-th row of Ĉ, and norm(·) denotes the normalization function. Moreover, to enhance topic interpretability, GCTM-OT generates a human-readable summary 𝑡𝑘𝑠 for the 𝑘-th topic using: ˆ 𝑒®𝑔𝑘 )) 𝑡𝑘𝑠 = arg max (cos( T̄ (𝑔), ˆ 𝑘 𝑔∈𝐶

(20)

ˆ is the where 𝑔ˆ denotes a goal candidate in the 𝑘-th cluster 𝐶𝑘 , T̄ (𝑔) ˆ and mean of token embeddings produced by transformer T for 𝑔, 𝑒®𝑔𝑘 is the 𝑘-th goal representation.

5

Experiments

We first introduce the experimental setup, including descriptions of the datasets, baselines, metrics and implementation details. We then present the topic evaluation results along with the corresponding analysis, followed by hyperparameter analysis and ablation studies.

5.1

Experimental Setup

5.1.1 Datasets. We evaluate the performance of GCTM-OT for Human-TM task on three subreddit datasets: ‘Bothering’ 6 , ‘TeslaModel3’ 7 and ‘AskAcademia’ 8 . Specifically, the ‘Whatsbotheringyou’ dataset is a collection of posts about personal concerns, such as ‘breakup struggle’ and ‘health issue’, and is abbreviated as ‘Bothering’ in our experiments. ‘TeslaModel3’ dataset contains user-generated posts covering various aspects of this electric vehicle, such as ‘charging speed’ and ‘autopilot’. ‘AskAcademia’ dataset contains posts about academic life, including topics like ‘work–life balance’ and ‘research challenges’. All datasets are sourced from Pushshift’s Reddit data dumps9 . For each dataset, to ensure data quality, we remove duplicate posts with the same post_id, lowengagement posts with scores below 10, and irrelevant posts tagged by moderators. We then perform a series of preprocessing steps on

each selected post, including spell checking, stemming and stopword removal, using the SpaCy10 library. The statistics of the processed datasets are presented in Table 1. Datasets

#Doc

#Vocab

Avg_Len

Bothering TeslaModel3 AskAcademia

1,483 1,991 3,348

7,700 5,634 8,122

56.48 32.82 36.16

Table 1: The statistics of processed datasets. 5.1.2 Baselines. We choose the following approaches as baselines: • LDA [3] is a probabilistic topic model, which models documents with a mixture of topics. We use the Mallet 11 implementation in the experiments. • BAT [37] is a neural topic modeling approach based on bidirectional adversarial training. We implement the approach following the default parameter configurations. • CTMNeg [2] is a neural topic model based on VAE and negative sampling, we use the official implementation 12 . • vONT [47] is a neural topic model, which models topics with a mixture of von-Mises Fisher distributions. We use the official implementation 13 . • CWTM [11] is a topic model based on the contextualized representations, we use the official implementation 14 . • DisCTM[38] is a contextualized neural topic model for short text modeling. We implement the approach following the default parameter configurations. • HiCOT[34] is a neural topic model based on optimal transport, we use the official implementation 15 . • CAST[21] is a topic model that leverages word embeddings to remove functional words for topic extraction16 . • LLM-TE [23] is a topic mining approach based on prompt engineering, we use the official implementation 17 . • LLM-ITL [48] is a neural topic model that incorporates an LLM-based topic refinement mechanism 18 . 5.1.3 Evaluation Metrics. Human-TM’s performance is evaluated using topic coherence and diversity metrics. In addition, we propose three novel metrics to assess the relevance between humanprovided goals and the extracted topics. To measure topic interpretability, we follow [29] and employ four widely used topic coherence metrics: 𝐶𝑃 , 𝐶𝐴 , 𝑁 𝑃𝑀𝐼 and 𝑈 𝐶𝐼 . All coherence values are computed using the Palmetto 19 library, with higher values indicating greater interpretability. For topic diversity evaluation, we follow [41] and use the Unique Term (𝑈𝑇 ) metric, defined as: 𝑈𝑇 = #𝑁𝑢 /(10 × 𝐾) (21) where #𝑁𝑢 denotes the number of unique words among the topic words, and 𝐾 is the number of topics. 10 https://spacy.io/ 11 https://github.com/mimno/Mallet 12 https://github.com/adhyasuman/ctmneg 13 https://github.com/xuweijieshuai/Neural-Topic-Modeling-vmf 14 DBLP:conf/coling/00110P24

6 https://www.reddit.com/r/whatsbotheringyou/

15 https://github.com/HoangTran223/HiCOT

7 https://www.reddit.com/r/TeslaModel3/

16 https://github.com/yananma1029/CAST

8 https://www.reddit.com/r/AskAcademia/

17 https://github.com/GateNLP/LLMs-for-Topic-Modeling

9 https://www.reddit.com/r/pushshift/comments/1akrhg3/separate_dump_files_for_

18 https://github.com/Xiaohao-Yang/LLM-ITL

the_top_40k_subreddits/

19 https://github.com/dice-group/Palmetto

 















 

  







   





/'$







   



   



%$7





NPMIRQ$VN$FDGHPLD



CARQ$VN$FDGHPLD

CPRQ$VN$FDGHPLD



&701HJ









Y217

&:70

  























    

    



'LV&70













 



 



 





NPMIRQ7HVOD0RGHO

CARQ7HVOD0RGHO

















 

CPRQ7HVOD0RGHO







UCIRQ%RWKHULQJ





UCIRQ7HVOD0RGHO



Conference’17, July 2017, Washington, DC, USA

UCIRQ$VN$FDGHPLD





NPMIRQ%RWKHULQJ



CARQ%RWKHULQJ

CPRQ%RWKHULQJ

Human-Centric Topic Modeling with Goal-Prompted Contrastive Learning and Optimal Transport



+L&27



&$67



//07(



//0,7/

*&7027

Figure 5: The comparison of topic coherence values vs. different topic number settings [20, 30, 40, 50]. Dataset Model LDA BAT CTMNeg vONT CWTM DisCTM HiCOT CAST LLM-TE LLM-ITL GCTM-OT

Bothering

TeslaModel3

AskAcademia

𝑪𝑷

𝑪𝑨

𝑵 𝑷 𝑴𝑰

𝑼 𝑪𝑰

𝑪𝑷

𝑪𝑨

𝑵 𝑷 𝑴𝑰

𝑼 𝑪𝑰

𝑪𝑷

𝑪𝑨

𝑵 𝑷 𝑴𝑰

𝑼 𝑪𝑰

0.1214 0.0127 -0.1319 0.1535 0.0175 0.1316 -0.1226 0.2097 0.0164 -0.0493 0.3326

0.1286 0.1324 0.1189 0.1353 0.1132 0.1509 0.1209 0.1584 0.1414 0.1293 0.1916

0.0047 -0.0022 -0.0575 0.0020 -0.0107 0.0163 -0.0599 0.0254 -0.0502 -0.0232 0.0419

-0.3226 -0.8499 -1.9725 -0.5920 -0.6486 -0.4035 -2.1051 -0.2828 -1.9701 -1.1970 -0.1020

0.1143 -0.0246 -0.1510 0.0995 0.0417 0.1067 -0.0881 0.1212 -0.0480 -0.0419 0.3031

0.1451 0.1441 0.1195 0.1363 0.1320 0.1464 0.1298 0.1716 0.1389 0.1408 0.2011

-0.0056 -0.0058 -0.0611 -0.0093 -0.0265 -0.0020 -0.0753 0.0136 -0.0519 -0.0296 0.0377

-0.6976 -0.7193 -2.0211 -0.8079 -1.0433 -0.7551 -2.5121 -0.5318 -1.9261 -1.2793 -0.1889

0.0997 -0.0786 -0.0156 0.0883 0.1039 0.1362 0.0372 0.1337 0.0696 0.0561 0.2990

0.1422 0.1439 0.1319 0.1187 0.1414 0.1540 0.1527 0.1683 0.1502 0.1559 0.1946

-0.0114 -0.0031 -0.0267 -0.0018 -0.0042 0.0124 -0.0494 0.0142 -0.0348 0.0010 0.0405

-0.6964 -0.6011 -1.2121 -0.3956 -0.6252 -0.3812 -1.9659 -0.4551 -1.6095 -0.6401 -0.1173

Table 2: Average topic coherence values with four topic number settings [20, 30, 40, 50].

Furthermore, to assess the relevance between the human-provided goal H𝑔 and the extracted topics, we propose three novel metrics: Goal Similarity (𝐺𝑆), Goal-relevant Topic Rate (𝐺𝑇 𝑅) and Goal Coverage Rate (𝐺𝐶𝑅), all computed based on OpenAI embeddings 20 . Concretely, given a corpus 𝐷 and the human-provided goal H𝑔, we first identify a set of 𝑁𝑔 ground-truth goal candidates ˆ 𝑔2, ˆ . . . , 𝑔𝑁 ˆ 𝑔 } through clustering and human annotation. 𝐺ˆ = {𝑔1, To measure the semantic relevance between the extracted topics ˆ we 𝑇 = {𝑡 1, 𝑡 2, . . . , 𝑡𝐾 } and the ground-truth goal candidates 𝐺, define the Goal Similarity (𝐺𝑆) metric with: ∑︁𝐾 ˆ = 1 𝐺𝑆 (𝑇 , 𝐺) max cos(® 𝑒𝑡𝑘 , 𝑒®𝑔ˆ) (22) 𝑘=1 𝑔∈ 𝐾 ˆ 𝐺ˆ where 𝑒®𝑡𝑘 and 𝑒®𝑔ˆ denote the OpenAI embeddings of the 𝑘-th topic ˆ respectively. On the other hand, to 𝑡𝑘 and the goal candidate 𝑔, measure the proportion of topics that are relevant to the goals, we define the Goal-relevant Topic Rate (𝐺𝑇 𝑅) metric as: ∑︁𝐾 ˆ = 1 𝐺𝑇 𝑅(𝑇 , 𝐺) max(0, sgn(max cos(® 𝑒𝑡𝑘 , 𝑒®𝑔ˆ) − 𝜎1 )) (23) 𝑘=1 𝐾 ˆ 𝐺ˆ 𝑔∈ where cos(® 𝑒𝑡𝑘 , 𝑒®𝑔ˆ) means the cosine similarity between OpenAI emˆ sgn(·) denotes beddings of the 𝑘-th topic and the goal candidate 𝑔, 20 https://platform.openai.com/docs/guides/embeddings

the sign function, and 𝜎1 is the threshold hyperparameter. Similarly, to measure the proportion of goal candidates in 𝐺ˆ that the extracted topics can discover, we define the Goal Coverage Rate (𝐺𝐶𝑅) metric as: ∑︁𝑁𝑔 ˆ = 1 𝐺𝐶𝑅(𝑇 , 𝐺) max(0, sgn(max cos(® 𝑒𝑔ˆ𝑖 , 𝑒®𝑡 ) − 𝜎2 )) (24) 𝑖=1 𝑡 ∈𝑇 𝑁𝑔 where cos(® 𝑒𝑔ˆ𝑖 , 𝑒®𝑡 ) is the cosine similarity between OpenAI embeddings of the 𝑖-th goal candidate 𝑔ˆ𝑖 and topic 𝑡, and 𝜎2 denotes the threshold hyperparameter.

5.1.4 Implementation Details. In our experiments, we use the MPNet [31] (T ) to generate contextualized representations. For LLMbased prompting (goal summarization prompt P𝑠 and text augmentation prompt P𝑎 ), we employ the GPT-3.5-turbo. For three datasets, the analytical objectives (goal) are defined with: “What’s the thing that’s bothering you?”, “What is the specific aspect of TeslaModel3?” and “What is the specific concern about academia?”, respectively. All experiments are conducted on a machine running Ubuntu, equipped with an NVIDIA RTX 4090 GPU (CUDA 12.2) and 64GB RAM.

Conference’17, July 2017, Washington, DC, USA

Trovato et al. 7RSLF1XP*75

Model

20

30

40

50

20

30

40

50

20

30

40

50

LDA BAT CTMNeg vONT CWTM DisCTM HiCOT CAST LLM-TE LLM-ITL GCTM-OT

0.6950 0.5150 0.7250 0.7700 0.4850 0.7100 0.7400 0.6300 0.6750 0.7000 0.9800

0.6000 0.6067 0.6933 0.6600 0.3767 0.6800 0.6433 0.6900 0.4167 0.6100 0.9200

0.5750 0.5525 0.6800 0.6350 0.4000 0.5600 0.5300 0.5350 0.7875 0.5950 0.8175

0.5620 0.6480 0.6120 0.6320 0.4140 0.5120 0.6000 0.5600 0.7460 0.5520 0.7400

0.6150 0.7050 0.7050 0.6600 0.6400 0.8000 0.6950 0.8300 0.8000 0.6500 0.9750

0.5867 0.7633 0.7133 0.5267 0.5133 0.6567 0.6600 0.6800 0.3467 0.5900 0.9133

0.5600 0.6675 0.6725 0.5325 0.4050 0.6200 0.6050 0.6375 0.7800 0.5675 0.8125

0.5260 0.6680 0.6500 0.5100 0.5200 0.5620 0.5400 0.5880 0.6860 0.5420 0.7260

0.6650 0.6550 0.7650 0.8750 0.7000 0.8650 0.6800 0.8250 0.7000 0.7150 1.0000

0.6333 0.5767 0.7033 0.7967 0.6533 0.8067 0.7433 0.7100 0.7267 0.5933 0.9400

0.5675 0.5525 0.6950 0.6750 0.6125 0.7400 0.5300 0.6975 0.5375 0.6425 0.9000

0.5240 0.5160 0.6140 0.6180 0.5200 0.6720 0.5260 0.6000 0.7740 0.5920 0.8380

Table 3: Comparison on 𝑈𝑇 metric across different topic number settings. Dataset

LDA

BAT

CTMNeg

vONT

CWTM

DisCTM

HiCOT

CAST

LLM-TE

LLM-ITL

GCTM-OT

Bothering TeslaModel3 AskAcademia

0.3611 0.4607 0.3890

0.3386 0.3496 0.3248

0.3654 0.4131 0.3933

0.3593 0.4367 0.3789

0.3557 0.4188 0.3635

0.3557 0.4110 0.3915

0.3356 0.4051 0.4014

0.3845 0.4696 0.3943

0.3616 0.4689 0.3971

0.3653 0.4736 0.4104

0.4253 0.5100 0.4530

Table 4: Comparison results on Goal Similarity (𝐺𝑆) metric.

5.2

Topic Evaluation and Results Analysis

In this section, we conduct extensive experiments on the ‘Bothering’, ‘TeslaModel3’ and ‘AskAcademia’ datasets, and present the results along with an analysis of topic coherence, diversity and goal-relevance metrics. For each dataset, we run the model with four different topic number configurations [20, 30, 40, 50], and we select the top 10 words according to the topic-word probabilities to represent the meaning of each topic. 5.2.1 Topic Coherence and Diversity Evaluation. To assess the topic interpretability and diversity, we perform comparisons on topic coherence (𝐶𝑃 , 𝐶𝐴 , 𝑁 𝑃𝑀𝐼 and 𝑈𝐶𝐼 ) and diversity (𝑈𝑇 ) metrics. Coherence: Table 2 lists the average topic coherence values for the 𝐶𝑃 , 𝐶𝐴 , 𝑁 𝑃𝑀𝐼 and 𝑈𝐶𝐼 metrics across the three datasets. Each value is obtained by averaging the coherence scores from different topic-number configurations. For example, GPTM’s 𝑁 𝑃𝑀𝐼 value (0.0405) on the ‘AskAcademia’ dataset is the mean value of 0.04779 (20 topics), 0.04328 (30 topics), 0.03338 (40 topics) and 0.03751 (50 topics). To examine whether GCTM-OT is sensitive to variations in the topic-number setting, we also present a comparison of topic coherence vs. topic number in Figure 5. The statistics show that GCTM-OT outperforms state-of-the-art baselines across the three datasets under different topic-number settings. These improvements may be attributed to the following aspects: 1). Modeling topics with the Dirichlet prior helps improve topic quality. 2). Incorporating goal information into the text representation process facilitates the generation of meaningful text representations. 3). Incorporating contextualized word representations into topic word extraction is crucial for mining coherent topics. Diversity: To compare topic diversity, we report the 𝑈𝑇 values in Table 3, with the optimal results highlighted in bold. The statistics show that GCTM-OT produces more diverse topics than state-of-the-art baselines across the three datasets under different topic-number configurations. The superior performance of GCTMOT on the 𝑈𝑇 metric may be attributed to the following two factors: 1). Semantic-aware contrastive learning tends to separate documents with different semantics into distinct clusters, facilitating the discovery of diverse patterns. 2). The goal-oriented optimal transport mechanism aligns topics with different goals, thereby enhancing topic diversity.

%RWKHULQJ

AskAcademia

7HVOD0RGHO

TeslaModel3

$VN$FDGHPLD

Bothering

7RSLF1XP*&5

































 

 















 

    6LPLODULW\7KUHVKROG 1 /'$ %$7

&701HJ Y217



&:70 'LV&70

 +L&27 &$67

    6LPLODULW\7KUHVKROG 2 //07( //0,7/



*&7027

Figure 6: Comparision on 𝐺𝑇 𝑅 and 𝐺𝐶𝑅 metrics.

Dataset Bothering TeslaModel3 AskAcademia

Human vs. Human

Human vs. LLM

𝑱𝑺

𝑺𝑴 𝑪

𝑫𝑺𝑪

𝑱𝑺

𝑺𝑴 𝑪

𝑫𝑺𝑪

0.7579 0.9192 0.9600

0.7700 0.9200 0.9600

0.8623 0.9579 0.9796

0.7738 0.8800 0.9196

0.7750 0.8800 0.9200

0.8722 0.9357 0.9581

Table 5: Coefficients between two experts’ annotation results. 5.2.2 Goal Annotation and Relevence Evluation. To assess the relevance between goal H𝑔 and extracted topics 𝑇 , we should first identify a set of 𝑁𝑔 ground-truth goal candidates 𝐺ˆ = {𝑔ˆ1, 𝑔ˆ2, ..., 𝑔𝑁ˆ 𝑔 }. Goal Annotation: For each dataset, we first perform clustering on its corpus-level goal candidate set 𝐺 and obtain a set of 𝑁𝑐 21 goal candidate clusters 𝐶𝑔 = {𝐶 1, 𝐶 2, ..., 𝐶 𝑁𝑐 }. We then recruit two experts (human or LLM) to identify if the 𝑖-th (𝑖 ∈ {1, 2, ..., 𝑁𝑐 }) cluster 𝐶𝑖 is a reasonable candidate for H𝑔 . In our experiments, we employ the Jaccard Similarity Coefficient (𝐽𝑆𝐶) [18], Simple Matching Coefficient (𝑆𝑀𝐶) [13] and the Sorensen-Dice Coefficient (𝑆𝐷𝐶) [5] to measure the agreement between the two experts’ annotation results, with the coefficient values listed in Table 5. Only clusters labeled as true by both human experts are added to the ground-truth ˆ resulting in 72, 91 and 96 ground-truth goal goal candidate set 𝐺, candidates for ‘Bothering’, ‘TeslaModel3’ and ‘AskAcademia’. Goal-relevance Evaluation: To assess the semantic relevance ˆ we between the topics and the annotated goal candidate set 𝐺, calculate the Goal Similarity (𝐺𝑆) values using Eq. 22 and present the results in Table 4. Each value is the mean 𝐺𝑆 value across four topic number setting, and 𝐺𝑆 value is obtained by averaging the maximum semantic similarity between each topic 𝑡𝑘 ∈ 𝑇 and the ˆ The statistics show that our proposed GCTMgoal candidates in 𝐺. OT consistently generates topics with higher semantic relevance to human-provided goals than the baselines, across all three datasets and under various topic number settings. Similarly, to assess the proportion of topics that are relevant to goals and the proportion of goals in 𝐺ˆ that extracted topics could discover, we calculate the Goal-relevant Topic Rate (𝐺𝑇 𝑅) and the Goal Coverage Rate (𝐺𝐶𝑅) values using Eq. 23 and Eq. 24. For evaluation, we set the threshold hyperparameters (𝜎1 and 𝜎2 ) to six different values: [0.4, 0.425, 0.45, 0.475, 0.5, 0.525], columns mean the metrics, and rows denote datasets. The comparison results for 21 𝑁

𝑐 is set to 100 in our evaluation.

Human-Centric Topic Modeling with Goal-Prompted Contrastive Learning and Optimal Transport

Conference’17, July 2017, Washington, DC, USA

Model

Topic Summary

Topic Words

LDA

— — —

completely, walk, girlfriend, hang, answer, finally, happy, honestly text couple pain, issue, tired, doctor, knee, sister, sleep, clean, house, tear money, college, afford, bill, rent, apply, house, plan, student, boss

LLM-ITL

— — —

loved, relationship, past, old, text, feel, love, emotional, life, talk doctor, hospital, mental, cancer, symptom, helping, degree, currently, training, surgery money, work, need, wanted, open, using, month, job, know, asking

GCTM-OT

Breakup Struggles Health Issue Financial Strain

cheat, breakup, girlfriend, betray, ring, break, divorce, relationship, partner, devastate lung, surgery, cancer, knee, blood, diagnosis, medical, condition, doctor, injury financially, financial, expense, afford, money, payment, debt, income, bill, expensive

Table 6: Topic Examples extracted from the ‘Bothering’ dataset, italic means out-of-topic words.

Parameter

Setting

𝑪𝑷

𝑪𝑨

𝑵 𝑷 𝑴𝑰

𝑼 𝑪𝑰

𝑼𝑻

0.2575

0.1592

0.0258

-0.3314

0.6300

0.2 0.4 0.6 0.8 1

0.3161 0.3431 0.3518 0.3371 0.2959

0.2001 0.1891 0.2084 0.1962 0.1957

0.0408 0.0414 0.0524 0.0416 0.0329

-0.1491 -0.2578 0.0568 -0.1541 -0.2770

0.9700 0.9650 0.9800 0.9850 0.9450

0.6 0.8 1 1.2 1.4

0.3251 0.3593 0.3518 0.3492 0.3472

0.2134 0.2108 0.2084 0.2023 0.1942

0.0387 0.0491 0.0524 0.0406 0.0433

-0.2748 -0.1147 0.0568 -0.2481 -0.1644

0.9800 0.9600 0.9800 0.9750 0.9800

0.6 0.8 1 1.2 1.4

0.3318 0.3274 0.3518 0.3293 0.3348

0.2057 0.2037 0.2084 0.1917 0.2125

0.0436 0.0396 0.0524 0.0381 0.0448

-0.1256 -0.2125 0.0568 -0.2889 -0.1573

0.9650 0.9700 0.9800 0.9450 0.9550

CAST

𝛿

𝜂

𝜁

Table 7: Comparison results of hyperparameter analysis.

𝐺𝑇 𝑅 and 𝐺𝐶𝑅 in Figure 6 show that GCTM-OT generates more goal-oriented topics than the baselines and that these topics cover ˆ a broader range of goal candidates in 𝐺. The superior performance of GCTM-OT on the 𝐺𝑆, 𝐺𝑇 𝑅 and 𝐺𝐶𝑅 metrics may be attributed to the following factors: 1). Goaloriented text representation mechanism injects goal information into text representations and topics. 2). Goal-oriented optimal transport mechanism aligns the trainable topic representation matrix 𝐸𝑡 with the LLM-generated goal representation matrix 𝐸𝑔 via optimal transport, thereby facilitating goal-relevant topic generation. To provide an intuitive comparison of topic interpretability, we present three example topics—‘Breakup Struggle’, ‘Health Issue’ and ‘Financial Strain’—in Table 6. We observe that GCTM-OT generates interpretable, goal-oriented topics accompanied by automatically produced, human-readable topic summaries. The symbol ‘−’ indicates that topic summaries are not available for LDA and LLM-ITL.

5.3

Hyperparameter Analysis

To explore the impact of the similarity threshold 𝛿, prior matching coefficient 𝜂 and optimal transport coefficient 𝜁 on GCTM-OT, we conduct a parameter analysis on the ‘Bothering’ dataset using a 20-topic configuration.

T

Strategy

LLM

Model

𝑪𝑷

𝑪𝑨

𝑵 𝑷 𝑴𝑰

𝑼 𝑪𝑰

𝑼𝑻

CAST

0.2575

0.1592

0.0258

-0.3314

0.6300

GCTM-OT (RoBERTa) GCTM-OT(BERT) GCTM-OT(Sentence-BERT) GCTM-OT(MiniLM) GCTM-OT

0.3460 0.2760 0.2760 0.3359 0.3518

0.1944 0.1673 0.1668 0.1939 0.2084

0.0489 0.0381 0.0371 0.0506 0.0524

0.0414 0.0639 -0.0707 0.0312 0.0568

0.9900 0.9450 0.9500 0.9600 0.9800

GCTM-OT w/o CL GCTM-OT w/o GR GCTM-OT w/o OT GCTM-OT

0.2959 0.3028 0.3148 0.3518

0.1957 0.1993 0.2017 0.2084

0.0329 0.0428 0.0436 0.0524

-0.2770 -0.0584 -0.0586 0.0568

0.9450 0.9900 0.9800 0.9800

GCTM-OT(DeepSeek-V2.5) GCTM-OT(GPT-4o-mini) GCTM-OT(Gemini) GCTM-OT(Mistral) GCTM-OT

0.3276 0.3802 0.2704 0.3031 0.3518

0.1928 0.2182 0.1818 0.1836 0.2084

0.0403 0.0613 0.0413 0.0456 0.0524

-0.1403 0.2174 0.1093 0.0824 0.0568

0.9400 0.9800 0.9050 0.9050 0.9800

Table 8: Comparison results of ablation studies. For each parameter, we test five different values: 𝛿 ∈ {0.2, 0.4, 0.6, 0.8, 1.0}, 𝜂 ∈ {0.6, 0.8, 1.0, 1.2, 1.4}, and 𝜁 ∈ {0.6, 0.8, 1.0, 1.2, 1.4}, while keeping other parameters at their default settings. Detailed results are listed in Table 7. For clarity, we present only CAST’s results as the comparison baseline due to its competitive performance. We observe that our proposed GCTM-OT consistently outperforms the competitive CAST across various parameter combinations, demonstrating its robustness to these hyperparameters.

5.4

Ablation Study

To explore the impact of the transformer T , large language models (LLMs) and different strategies (such as optimal transport, semanticaware contrastive learning and goal-oriented text representation) on GCTM-OT, we conduct ablation studies on the ‘Bothering’ dataset using a 20-topic configuration. For the transformer T ablation, we utilize five variants: BERT [9], MiniLM [43], MPNet [31], Sentence-BERT [28] and RoBERTa [20]. For strategy ablation, we use variants of GCTM-OT that exclude optimal transport (GCTM-OT w/o OT), goal-oriented text representation (GCTM-OT w/o GR) and the semantic-aware mechanism in contrastive learning (GCTM-OT w/o CL). For LLM ablation, we explore various models: DeepSeek-V2.5 22 , GPT-4o-mini 23 , GPT-3.5 24 , Gemini-1.5-pro 25 and Mistral 26 . 22 https://api-docs.deepseek.com/news/news0905 23 https://platform.openai.com/docs/models/gpt-4o-mini 24 https://platform.openai.com/docs/models/gpt-3.5-turbo 25 https://ai.google.dev/gemini-api/docs/models#gemini-1.5-pro 26 https://mistral.ai/news/mixtral-8x22b

Conference’17, July 2017, Washington, DC, USA

As shown in Table 8, GCTM-OT with different transformers, strategies and LLMs consistently outperforms the competitive baseline in terms of topic coherence and diversity metrics. For clarity, we present only CAST’s results as the baseline.

6

Conclusion

We introduce Human-TM, a novel topic modeling task formulation that embeds human-provided goals into topic discovery. To tackle this task, we propose the GCTM-OT, a goal-prompted contrastive topic model with optimal transport to generate interpretable, diverse and goal-oriented topics. Experiments on three public subreddit datasets show that GCTM-OT consistently outperforms stateof-the-art baselines in coherence, diversity and goal alignment. By directly embedding user intent into the modeling process, HumanTM reduces the need for manual filtering and improves the practical utility of topic models for targeted analysis. Future work will explore hierarchical extensions of Human-TM, enabling the generation of topic hierarchies aligned with human-provided goals.

Acknowledgement This work is supported by the National Natural Science Foundation of China (Grant No. 62102192), the fellowship of China Postdoctoral Science Foundation (Grant No. 2022M710071), the Fundamental Research Funds for the Central Universities of China (Grant Nos. PA2025IISL0107 and aiia-24-01) and the Innovation and Entrepreneurship Program of Jiangsu Province (Grant No. JSSCBS20210530).

References [1] Aly Abdelrazek, Yomna Eid, Eman Gawish, Walaa Medhat, and Ahmed Hassan Yousef. 2023. Topic modeling algorithms and applications: A survey. Information Systems 112 (2023), 102131. [2] Suman Adhya, Avishek Lahiri, Debarshi Kumar Sanyal, and Partha Pratim Das. 2022. Improving Contextualized Topic Models with Negative Sampling. In Proceedings of the 19th International Conference on Natural Language Processing, ICON 2022, New Delhi, India, December 15-18, 2022. Association for Computational Linguistics, 128–138. [3] David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent Dirichlet Allocation. Journal of Machine Learning Research 3 (2003), 993–1022. [4] Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020). [5] Aaron Carass, Snehashis Roy, Adrian Gherman, Jacob C Reinhold, Andrew Jesson, Tal Arbel, Oskar Maier, Heinz Handels, Mohsen Ghafoorian, Bram Platel, et al. 2020. Evaluating white matter lesion segmentations with refined Sørensen-Dice analysis. Scientific reports 10, 1 (2020), 8242. [6] Shuyu Chang, Rui Wang, Peng Ren, Qi Wang, and Haiping Huang. 2024. A Large Language Model Guided Topic Refinement Mechanism for Short Text Modeling. arXiv preprint arXiv:2403.17706 (2024). [7] Ching-Yao Chuang, Joshua Robinson, Yen-Chen Lin, Antonio Torralba, and Stefanie Jegelka. 2020. Debiased Contrastive Learning. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual. [8] Marco Cuturi. 2013. Sinkhorn Distances: Lightspeed Computation of Optimal Transport. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States. 2292–2300. [9] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers). Association for Computational Linguistics, 4171–4186. [10] Kennedy Edemacu and Xintao Wu. 2025. Privacy Preserving Prompt Engineering: A Survey. Comput. Surveys 57, 10 (2025), 255:1–255:36. [11] Zheng Fang, Yulan He, and Rob Procter. 2024. CWTM: Leveraging Contextualized Word Embeddings from BERT for Neural Topic Modeling. In Proceedings of

Trovato et al.

the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC/COLING 2024, 20-25 May, 2024, Torino, Italy. ELRA and ICCL, 4273–4286. [12] Fiona Fui-Hoon Nah, Ruilin Zheng, Jingyuan Cai, Keng Siau, and Langtao Chen. 2023. Generative AI and ChatGPT: Applications, challenges, and AI-human collaboration. 277–304 pages. [13] David W Goodall. 1967. The distribution of the matching coefficient. Biometrics (1967), 647–656. [14] Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David WardeFarley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada. 2672–2680. [15] Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander J. Smola. 2012. A Kernel Two-Sample Test. Journal of Machine Learning Research 13 (2012), 723–773. [16] Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. [17] Diederik P. Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings. [18] Sven Kosub. 2019. A note on the triangle inequality for the Jaccard distance. Pattern Recognition Letters 120 (2019), 36–38. [19] Jiayu Lin. 2016. On the dirichlet distribution. Department of Mathematics and Statistics, Queens University 40 (2016). [20] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv:1907.11692 [21] Yanan Ma, Chenghao Xiao, Chenhan Yuan, Sabine N. van der Veer, Lamiece Hassan, Chenghua Lin, and Goran Nenadic. 2025. CAST: Corpus-Aware Selfsimilarity Enhanced Topic modelling. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2025 - Volume 1: Long Papers, Albuquerque, New Mexico, USA, April 29 - May 4, 2025. Association for Computational Linguistics, 7548–7561. [22] Yishu Miao, Lei Yu, and Phil Blunsom. 2016. Neural Variational Inference for Text Processing. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016 (JMLR Workshop and Conference Proceedings, Vol. 48). JMLR.org, 1727–1736. [23] Yida Mu, Chun Dong, Kalina Bontcheva, and Xingyi Song. 2024. Large Language Models Offer an Alternative to the Traditional Approach of Topic Modelling. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC/COLING 2024, 20-25 May, 2024, Torino, Italy. ELRA and ICCL, 10160–10171. [24] Giorgio Patrini, Rianne van den Berg, Patrick Forré, Marcello Carioni, Samarth Bhargav, Max Welling, Tim Genewein, and Frank Nielsen. 2019. Sinkhorn AutoEncoders. In Proceedings of the Thirty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI 2019, Tel Aviv, Israel, July 22-25, 2019 (Proceedings of Machine Learning Research, Vol. 115). AUAI Press, 733–743. [25] Gabriel Peyré and Marco Cuturi. 2019. Computational Optimal Transport. Foundations and Trends in Machine Learning 11, 5-6 (2019), 355–607. [26] Chau Pham, Alexander Miserlis Hoyle, Simeng Sun, Philip Resnik, and Mohit Iyyer. 2024. TopicGPT: A Prompt-based Topic Modeling Framework. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), NAACL 2024, Mexico City, Mexico, June 16-21, 2024. Association for Computational Linguistics, 2956–2984. [27] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9. [28] Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019. Association for Computational Linguistics, 3980–3990. [29] Michael Röder, Andreas Both, and Alexander Hinneburg. 2015. Exploring the Space of Topic Coherence Measures. In Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, WSDM 2015, Shanghai, China, February 2-6, 2015. ACM, 399–408. [30] Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. 2024. A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications. CoRR abs/2402.07927 (2024). [31] Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. MPNet: Masked and Permuted Pre-training for Language Understanding. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.

Human-Centric Topic Modeling with Goal-Prompted Contrastive Learning and Optimal Transport

[32] Marek Strong, Rami Aly, and Andreas Vlachos. 2024. Zero-Shot Fact Verification via Natural Logic and Large Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024. Association for Computational Linguistics, 17021–17035. [33] George R Terrell and David W Scott. 1992. Variable kernel density estimation. The Annals of Statistics (1992), 1236–1265. [34] Hoang Tran Vuong, Tue Le, Tu Vu, Tung Nguyen, Linh Ngo Van, Sang Dinh, and Thien Huu Nguyen. 2025. HiCOT: Improving Neural Topic Models via Optimal Transport and Contrastive Learning. In Findings of the Association for Computational Linguistics: ACL 2025. 13894–13920. [35] Hanna M. Wallach, David M. Mimno, and Andrew McCallum. 2009. Rethinking LDA: Why Priors Matter. In Advances in Neural Information Processing Systems 22: 23rd Annual Conference on Neural Information Processing Systems 2009. Proceedings of a meeting held 7-10 December 2009, Vancouver, British Columbia, Canada. Curran Associates, Inc., 1973–1981. [36] Mengting Wan, Tara Safavi, Sujay Kumar Jauhar, Yujin Kim, Scott Counts, Jennifer Neville, Siddharth Suri, Chirag Shah, Ryen W. White, Longqi Yang, Reid Andersen, Georg Buscher, Dhruv Joshi, and Nagu Rangan. 2024. TnT-LLM: Text Mining at Scale with Large Language Models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2024, Barcelona, Spain, August 25-29, 2024. ACM, 5836–5847. [37] Rui Wang, Xuemeng Hu, Deyu Zhou, Yulan He, Yuxuan Xiong, Chenchen Ye, and Haiyang Xu. 2020. Neural Topic Modeling with Bidirectional Adversarial Training. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020. Association for Computational Linguistics, 340–350. [38] Rui Wang, Xing Liu, Yanan Wang, Shuyu Chang, Yuanzhi Yao, and Haiping Huang. 2025. Mining Topics towards ChatGPT Using a Disentangled Contextualizedneural Topic Model. In Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining, WSDM 2025, Hannover, Germany, March 10-14, 2025. ACM, 539–548. [39] Rui Wang, Deyu Zhou, and Yulan He. 2019. ATM: Adversarial-neural Topic Model. Information Processing & Management 56, 6 (2019). [40] Rui Wang, Deyu Zhou, and Yulan He. 2019. Open Event Extraction from Online Text using a Generative Adversarial Network. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019. Association for Computational Linguistics, 282–291. [41] Rui Wang, Deyu Zhou, Haiping Huang, and Yongquan Zhou. 2025. MIT: Mutual Information Topic Model for Diverse Topic Extraction. IEEE Transactions on Neural Networks and Learning Systems 36, 2 (2025), 2523–2537. [42] Tongzhou Wang and Phillip Isola. 2020. Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event (Proceedings of Machine Learning Research, Vol. 119). PMLR, 9929–9939. [43] Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. 2020. MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of PreTrained Transformers. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual. [44] Zihan Wang, Jingbo Shang, and Ruiqi Zhong. 2023. Goal-Driven Explainable Clustering via Language Descriptions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023. Association for Computational Linguistics, 10626–10649. [45] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022. [46] Xiaobao Wu, Thong Nguyen, and Anh Tuan Luu. 2024. A survey on neural topic models: methods, applications, and challenges. Artificial Intelligence Review 57, 2 (2024), 18. [47] Weijie Xu, Xiaoyu Jiang, Srinivasan Sengamedu Hanumantha Rao, Francis Iannacci, and Jinjin Zhao. 2023. vONTSS: vMF based semi-supervised neural topic modeling with optimal transport. In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2023. Association for Computational Linguistics, 4433–4457. [48] Xiaohao Yang, He Zhao, Weijie Xu, Yuanyuan Qi, Jueqing Lu, Dinh Phung, and Lan Du. 2025. Neural Topic Modeling with Large Language Models in the Loop. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2025, Vienna, Austria, July 27 - August 1, 2025. Association for Computational Linguistics, 1377–1401. [49] Dejian Yu and Bo Xiang. 2023. Discovering topics and trends in the field of Artificial Intelligence: Using LDA topic modeling. Expert Systems with Applications 225 (2023), 120114. [50] Yuwei Zhang, Zihan Wang, and Jingbo Shang. 2023. ClusterLLM: Large Language Models as a Guide for Text Clustering. In Proceedings of the 2023 Conference

Conference’17, July 2017, Washington, DC, USA

on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023. Association for Computational Linguistics, 13903–13920.

Related documents

• Don&apos;t Play Favorites: Minority Guidance for Diffusion Models#430495
• Transformers as In-Context Samplers: From Closed-Form Diffusion to Estimation-Free Sampling#999895
• Low-Budget Active Learning through Entropic Optimal Transport#1156471
• LocUS: Head Selection and Subspace Projection for Targeted Activation Steering#1105680
• Concept-Level Risk and Calibration for Governance in Diffusion Foundation Models#985864