arXiv:2606.11476v1 [cs.SE] 9 Jun 2026
SentTrack: Sentiment-Driven Bottleneck Detection in GitHub Issue Repositories Xinyu Hu
Ali Behbahani
Daniel Moon
University of Tennessee Knoxville, TN, USA [email protected]
University of Tennessee Knoxville, TN, USA [email protected]
University of Tennessee Knoxville, TN, USA [email protected]
Yaren Dogan
Nasir U. Eisty
University of Tennessee Knoxville, TN, USA [email protected]
University of Tennessee Knoxville, TN, USA [email protected]
Abstract—Software engineering teams increasingly depend on GitHub issue threads to coordinate work, report bugs, and negotiate technical decisions, yet most repository health tools focus on code metrics and ignore the conversational dynamics that drive or stall development. This paper presents SentTrack, a dual-lens framework for detecting socio-technical bottlenecks from GitHub issue discussions. Applied to the AvaloniaUI opensource repository across approximately 9,000 issue threads, the framework addresses three questions: how to automate workflowinefficiency detection from real-time conversational data, whether sentiment signals can surface risk earlier than traditional labelbased methods, and how to isolate human narrative from machine-generated noise in mixed-media issue text. SentTrack combines two complementary pipelines. A horizontal pipeline translates raw issue reports into clean summaries using a large language model, extracts mid-level concern phrases, and clusters them through UMAP and HDBSCAN, producing 613 semantic clusters from the first 3,608 issues processed. A vertical pipeline applies the ABCDE collaborative interaction framework to classify each comment and infer thread-level outcomes. Across the full corpus, 49% of threads ended in stagnation and only 13% reached resolution, with the resolution gap identified as the dominant bottleneck signal. A weighted scoring engine that combines negativity, stagnation, resolution gap, and thread length gives maintainers an interpretable prioritization tool for highfriction discussions before they stall development. Index Terms—sentiment analysis, large language models, topic modeling, aspect-based sentiment analysis, GitHub repositories, software engineering automation, bottleneck detection, developer workflow, collaborative interaction
I. I NTRODUCTION Software development teams increasingly rely on GitHub Issues to coordinate work, report bugs, and negotiate technical decisions [1], [2]. These threads constitute a continuous, realtime record of both technical problems and the communication patterns that shape how projects evolve. Yet most repository health tools focus narrowly on code metrics, treating issue discussions as secondary. Teams still depend on periodic retrospectives or manually curated labels to surface friction, an approach that is reactive by nature and slow to catch emerging problems.
This paper investigates whether that gap can be closed through automated analysis of GitHub issue conversations. We focus on three research questions: RQ1: How can an automated framework identify workflow inefficiencies using real-time conversational data from GitHub repositories? • RQ2: Can sentiment analysis surface risk signals earlier than traditional label-based repository management? • RQ3: How can human narrative be effectively isolated from machine-generated noise in mixed-media issue threads to ensure topic extraction reflects actual developer intent? •
We analyze issue discussions from the AvaloniaUI opensource repository through a dual-lens approach: a horizontal pipeline for global topic modeling and a vertical pipeline for per-thread interaction analysis. Public repositories such as AvaloniaUI involve a mix of core maintainers and external contributors whose discussions span technical debt, workflow friction, feature prioritization, and documentation gaps. Two characteristics of GitHub issues make automated analysis particularly challenging. First, repository labels such as bug, enhancement, and documentation are frequently inconsistent, overly specific, or missing context about why issues remain unresolved. Labels like UI-fix and Backend-refactor may both stem from a deeper systemic cause, yet no native mechanism aggregates them into a higher-level insight. Second, GitHub issues are mixed-media documents in which stack traces, log dumps, and code snippets dominate text volume but carry no user intent, causing language models to surface topics rooted in code artifacts rather than developer concerns. SentTrack addresses both characteristics. The horizontal pipeline removes machine noise through intent-focused LLM summarization before extracting and clustering topics. The vertical pipeline models each issue as a chronological interaction sequence, classifying comments under the ABCDE framework to detect whether discussions converge toward resolution or accumulate friction. Together, the two lenses capture both
what developers discuss and how those discussions progress, producing interpretable bottleneck scores that maintainers can act on directly.
C. Semantic Summarization, Topic Extraction, and Clustering
II. R ELATED W ORK
Xi −−→ Si −−→ Ki −−→ zi ,
This study draws on five bodies of literature: sentiment and emotion mining in software engineering, issue mining and machine-noise removal, semantic topic modeling, thread-level conversation analysis, and LLM-assisted developer tooling.
where Xi is raw issue text, Si is a clean summary, Ki is a set of extracted concern phrases, and zi is the cluster assignment. Evaluation metrics such as ROUGE [21], factual-consistency scoring [22], SEAHORSE [23], and GPTScore [24] motivate assessing summaries beyond surface fluency. Keyphrase extraction work [25] motivates transforming long technical documents into compact concepts before clustering. Topic modeling has been applied to large developer corpora such as Stack Overflow [26]. Embedding-based approaches such as BERTopic [27] combine semantic encoders with density-based clustering. SentTrack follows this direction but separates topic extraction from clustering so that overfragmentation can be diagnosed independently at each stage. UMAP preserves local and global neighborhood structure in low dimensions [28], HDBSCAN identifies density-based clusters without a predefined count [29], and DBCV provides an intrinsic measure of cluster quality [30].
T
A. Sentiment and Emotion Mining in Software Engineering Pang and Lee [3] established the foundational vocabulary of sentiment analysis and opinion mining, treating them as closely related tasks over subjective text. Liu [4] formalized Aspect-Based Sentiment Analysis (ABSA), representing an opinion as a quintuple o = (e, a, s, h, t),
(1)
where e is the target entity, a is the aspect, s is sentiment orientation, h is the opinion holder, and t is time. This formulation is well suited to GitHub because a contributor may express negative sentiment about one aspect, such as rendering or input handling, while remaining neutral toward the project overall. Affective signals have been identified in commits, pull requests, issue trackers, and security discussions [5]–[9]. However, general-purpose sentiment tools perform poorly on technical text because words that appear negative in ordinary language may describe neutral debugging facts. SE-specific tools such as SentiStrength-SE, Senti4SD, and SentiCR address this mismatch, though comparative studies show that tool choice can still alter empirical conclusions [10]–[16]. Accordingly, SentTrack treats negativity as one term in a multi-factor risk model rather than as a direct bottleneck signal: Bi = f (Ni , Si , Ri , Li ),
(2)
where Ni is negativity, Si is stagnation, Ri is resolution gap, and Li is normalized thread length. B. Issue Mining, Machine Noise, and Pipeline Error GitHub labels provide useful metadata but are often sparse, inconsistent, or too coarse for repository-health analysis [17]. A further complication is the mixed-media nature of issue reports: Issue Text = Human Narrative + Code and Logs.
The horizontal component transforms raw issues through three sequential stages:
(3)
Non-natural-language artifacts degrade NLP pipelines when they are not removed or normalized [18], and pipeline studies demonstrate that early-stage errors propagate into later components [19], [20]. These findings motivate SentTrack’s translation stage, which converts raw issue text into intentfocused summaries before topic extraction begins.
E
C
(4)
D. Thread-Level Conversation Analysis and LLM Assistance Stavrianou et al. [31] integrated social-network analysis with opinion mining to model web forum dynamics, distinguishing user-based from opinion-based graph structures. Their work supports treating a GitHub issue as a chronological sequence of replies rather than a single document. Ravi et al. [32] introduce a collaborative interaction framework structured around the ABCDE taxonomy: Agree, Build, Chatting, Different Perspective, and Elicit. SentTrack adapts this taxonomy to GitHub issue threads, where labels A and B indicate convergence, D and E indicate unresolved concern or open clarification, and C represents weakly task-moving text. LLMs support a broad range of software-engineering tasks but remain vulnerable to hallucination and missing project context [33]. Retrieval-augmented generation grounds model outputs in retrieved artifacts [34], and studies of developer assistance tools show that RAG-based responses can match human helpfulness, though verbosity remains a limitation [35], [36]. SentTrack therefore uses LLMs only in bounded, welldefined roles: summarization, concern phrase extraction, and cluster labeling. E. Positioning of SentTrack Prior work typically studies sentiment, issue classification, summarization, topic modeling, or conversation structure as independent problems. SentTrack combines these streams by simultaneously estimating what is {z discussed} | horizontal topics
and
evolve} |how discussions {z
(5)
vertical thread dynamics.
The contribution is an interpretable end-to-end pipeline that links technical topics, thread outcomes, and bottleneck risk scores into a unified maintainer signal.
TABLE I S ENT T RACK PIPELINE STAGES AND OUTPUTS . Stage
Operation
Output
Translation
Remove machine noise while preserving user intent Generate mid-level concern phrases using the project profile Embed, reduce, and densitycluster topics Classify comments as an ABCDE sequence Fuse negativity, stagnation, resolution gap, and thread length
Si
Topic extraction Clustering Vertical analysis Scoring
TABLE II ABCDE CODING FRAMEWORK FOR VERTICAL THREAD ANALYSIS . Code
Meaning
Thread signal
A B
Agree Build
C D E
Chat Different Perspective Elicit
Confirmation or convergence Added evidence, proposed fix, or explanation Neutral or weakly task-moving text Disagreement, concern, or blocker Question or clarification request
Ki qi ai , y i Bi
2) Context-Aware Topic Extraction: From each clean summary, the pipeline extracts a set of mid-level concern phrases: Ki = Eθ (Si ; P ) = {ki1 , . . . , kiri },
III. A PPROACH SentTrack is a dual-lens framework for detecting sociotechnical bottlenecks in GitHub repositories. For issue i, the input is Ii = (ti , bi , Ci , mi ),
Ci = (ci1 , . . . , cini ),
(6)
where each kij is a three-to-six-word noun phrase. The project profile guides the model toward domain-specific concerns such as rendering, cross-platform input, data binding, tooling, documentation, performance, and native OS integration. The target abstraction level satisfies label < kij < raw stack trace,
where ti is the issue title, bi is the body, Ci is the ordered comment sequence, and mi contains metadata such as state and closure time. The horizontal lens maps (ti , bi ) to topic clusters; the vertical lens maps Ci to a thread outcome. A bottleneck scoring engine then fuses both lenses with sentiment-derived risk into a single interpretable score. A. Pipeline Overview The two sub-pipelines process each issue in parallel: T
E
g, U, H
θ θ (ti , bi ) −→ Si −−→ Ki −−−−→ qi ,
h
Ci − → ai → yi ,
(7)
where Tθ performs intent-focused translation, Si is the resulting clean summary, Eθ extracts concern phrases Ki , g embeds phrases, U applies UMAP, H applies HDBSCAN, qi is the cluster assignment, h assigns ABCDE labels, ai is the interaction-label sequence, and yi is the inferred thread outcome. Table I summarizes each stage.
(9)
(10)
meaning each phrase is more specific than a GitHub label but more reusable than a one-off error token. 3) Embedding and Density-Based Clustering: Each extracted phrase is embedded and assigned to a cluster through vij = g(kij ),
uij = U (vij ),
qij = H(uij ),
(11)
where g is the instruction-tuned encoder, U is UMAP, and H is HDBSCAN. Because the number of recurring concerns in any repository is not known in advance, HDBSCAN is preferred over clustering methods that require a predefined count. The noise label qij = −1 is treated as informative: it marks rare or isolated concerns rather than a clustering failure. C. Vertical Thread Analysis The vertical pipeline treats each issue as a chronological discussion rather than a single block of text. For comment cij , a classifier assigns an ABCDE label: aij = h(cij ) ∈ {A, B, C, D, E},
B. Horizontal Topic Clustering The horizontal pipeline identifies macro-level repository trends and recurring technical themes across the issue corpus. 1) Intent-Focused Summarization: GitHub issues blend human narrative with machine-layer content such as stack traces and log dumps. SentTrack first produces a clean summary
ai = (ai1 , . . . , aini ). (12) Table II defines each label and its thread-level interpretation. Thread-level counts and proportions are computed as (r) ni =
(8)
where P is a structured project profile. The LLM is instructed to retain the developer’s intent, normalize terminology, and suppress file paths, hexadecimal codes, stack traces, and raw logs. This prevents machine artifacts from dominating downstream topic extraction.
1[aij = r],
(13)
j=1 (r)
(r)
Si = Tθ (ti , bi ; P ),
ni X
ρi
=
ni , ni
r ∈ {A, B, C, D, E}.
(14)
The thread outcome is determined by the final interaction in the sequence: resolved, aini ∈ {A, B}, yi = stagnant, aini ∈ {D, E}, (15) neutral, aini = C.
A sequence such as E → B → A signals a question followed by a constructive response and agreement; D → E signals a concern followed by an unresolved clarification request. D. Pipeline Diagnostic Metrics To distinguish weak downstream NLP signals caused by model limitations from those caused by sparse or fragmented source conversations, SentTrack computes four diagnostic rates over N threads: N
RG = 1 −
1 X 1[yi = resolved], N i=1
(16)
#(E → {A, B}) , (17) #E #({D, E} → {A, B}) CCR = , (18) #{D, E} #shallow or isolated threads . (19) FS = N Here RG is the resolution gap, QRR is the question-resolution rate, CCR is the concern-capture rate, and FS is the fragmentation score. High RG or low QRR and CCR indicates that issue threads provide weak conversational signals for downstream summarization and clustering, a property of the corpus rather than of the models. QRR =
E. Bottleneck Detection Engine Rather than relying on a single indicator, the bottleneck engine combines sentiment-based features with thread-dynamics features. Comment-level negativity is aggregated per thread as + 0.20 ri− , Ni = 0.50 ni + 0.30 nmax i
(20)
where ni is average negativity, nmax is maximum negativity, i and ri− is the negative-comment ratio. The final bottleneck score is Bi = 0.35Ni + 0.30Si + 0.25Ri + 0.10Li ,
(21)
where Si is a binary stagnation indicator, Ri is the resolutiongap score, and Li is normalized thread length. The weight on length is deliberately smallest because a long discussion may reflect legitimate technical complexity rather than communication failure. In addition to the scalar score, the engine assigns each issue a compound bottleneck category: Gi ⊆ {Negative, Stagnant, Long, Unresolved}.
(22)
The score Bi is intended as a prioritization signal for maintainer triage, not as a ground-truth label that an issue is objectively blocked. IV. I MPLEMENTATION A. Horizontal Analysis 1) Project Profiling: To ensure the downstream LLM extracts domain-relevant topics rather than generic keywords, the pipeline begins with a static project profile stored in
project_profile.md. This profile was generated automatically by directing an AI agent to process the full AvaloniaUI repository and produce a structured document covering: the project’s domain and typical users, its major user-facing features, the technology categories most likely to attract opinions, and the kinds of concerns users would plausibly raise in issue threads. The resulting profile defines AvaloniaUI as a cross-platform .NET UI framework and organizes expected feedback into technology pillars such as UI and Rendering, Performance and Resource Usage, and Tooling Integration, along with concrete representative concerns such as memory leaks, visual inconsistencies, and missing native OS integration. 2) Intent-Focused Summarization: Each raw issue title and body is processed by gemini-2.5-flash via langchain_google_genai, configured at temperature 0.2 to minimize hallucination. The model is prompted to act as an expert triage engineer: it strips machine-layer noise including file paths, memory addresses, hexadecimal codes, and stack traces, and produces a two-to-five sentence plainEnglish summary focused on the developer’s core intent and the technical failure they experienced. A Pydantic schema enforces consistent output structure across the corpus. 3) Topic Extraction: Clean summaries are passed back through gemini-2.5-flash with a secondary Pydantic schema enforcing noun-phrase output of three to six words per phrase. This constraint keeps topics at the intended abstraction level, avoiding phrases that are too broad (such as “Performance Bug”) or too specific (such as “Slider track right side broken”). Extracted topics are deduplicated across the corpus to build a compact set of unique concern phrases for embedding. 4) Semantic Embedding and Clustering: Unique concern phrases are embedded with Alibaba-NLP/gte-Qwen2-1.5B-instruct via SentenceTransformers, producing high-dimensional vectors that capture contextual meaning rather than surface keyword overlap. UMAP reduces the embedding space while preserving local and global structure. HDBSCAN is then applied to the reduced representations, identifying natural-density clusters without a predefined count and labeling isolated phrases as noise. B. Vertical Thread Classification 1) Data Inputs: The vertical pipeline processes scraped GitHub issue data containing issue identifiers, comment text, timestamps, issue state, and closure metadata where available. The full dataset covers approximately 9,000 parent issues, both open and closed. Comments are organized by issue number, sorted by creation timestamp, and analyzed at two levels: individually for interaction signals and collectively for threadlevel aggregation. 2) ABCDE Classification: Each comment is converted to lowercase and matched against keyword lists for each interaction category. Phrases such as “I agree,” “looks good,” and
“that works” are classified as A; proposed fixes, added evidence, and explanations as B; question marks and interrogative phrases such as “why,” “could you,” and “what if” as E; and expressions of concern or disagreement such as “however,” “this breaks,” and “not sure” as D. Comments not matching any category default to C. This rule-based approach provides a lightweight, transparent baseline for classifying comment roles without the computational cost of a large language model. 3) Thread-Level Metrics: After classification, the pipeline aggregates ABCDE labels into per-issue metrics: total comment count, per-label frequency, D and E ratios, a questionpresence flag, the full chronological ABCDE sequence, and the thread outcome derived from Eq. 15. A transition matrix records how frequently each label type follows another across the corpus, enabling analysis of common conversation flow patterns.
Fig. 1. Discovery rates of unique LLM topics and HDBSCAN clusters across the first 2,700 issues.
C. Bottleneck Detection The bottleneck engine merges thread-dynamics metrics from the vertical pipeline with comment-level sentiment scores produced by the VADER analyzer [37]. VADER assigns a negative score, a positive score, a neutral score, and a compound score to each comment independently. Per-thread aggregates including average negativity, maximum negativity, and negative-comment ratio are combined into Ni via Eq. 20. Thread length is normalized by min-max scaling over all issues. Boolean flags for high negativity, stagnant outcome, long thread, and low resolution are derived and combined into the compound label set Gi . The final bottleneck score Bi from Eq. 21 ranks all issues, with open and closed threads producing independent output files and visualization directories. V. E VALUATION A. Horizontal Topic Clustering Evaluating unsupervised NLP pipelines is inherently challenging without objective ground-truth labels. Because SentTrack intentionally replaces GitHub’s fragile manual tagging system, no pre-existing baseline is available for direct comparison. We therefore evaluate the horizontal pipeline through qualitative assessment of individual extraction accuracy and quantitative analysis of discovery rates at scale. 1) Qualitative Assessment: On individual issues, human inspection confirms that the summarization and extraction constraints capture the core socio-technical friction. For a complex discussion about INotifyPropertyChanged and WPF DataContext equivalents, the pipeline generated clean midlevel phrases such as “External Model Data Binding Implementation” and “MVVM Library Integration Guidance,” successfully distilling a highly technical thread into structural concepts accessible to a project manager. 2) Topic Fragmentation Analysis: To evaluate pipeline behavior at scale, we plotted the cumulative discovery rate of unique LLM-generated topics and unique HDBSCAN clusters against the number of issues processed. Figure 1 shows results for the first 2,700 issues and Figure 2 shows results for the full corpus of 6,629 issues.
Fig. 2. Discovery rates of unique LLM topics and HDBSCAN clusters across the full corpus of 6,629 issues.
The unique-topic curve is nearly linear: processing approximately 8,000 rows yielded over 16,000 unique phrases, roughly two novel phrasings per issue. If the model produced consistent terminology, this curve would eventually plateau. Instead, the LLM generates fresh phrasings for essentially every issue, producing variants such as “XAML previewer startup failure,” “designer process crash during initialization,” and “visual studio design view launch error” for the same underlying problem. The cluster curve behaves differently, exhibiting an S-shape that plateaus. This confirms that the semantic encoder and HDBSCAN do group many varied phrasings together. However, the plateau settles at approximately 620 clusters for 2,700 issues and over 1,800 clusters for the full corpus. A repository of AvaloniaUI’s scale realistically contains 50 to 200 distinct macro-level concerns, so the pipeline produces roughly an order of magnitude more clusters than are actionable for maintainers. Figure 3 illustrates the full horizontal pipeline on a representative issue, showing how noise removal, phrase extraction, and cluster assignment interact. The root cause of over-fragmentation is the LLM’s unbounded extraction vocabulary. While the encoder can bridge moderate phrasing variation, linking “dependency injection” to “IoC container” for example, it cannot reliably consolidate 16,000 highly specific variants. Constraining the LLM to a curated vocabulary during extraction or introducing a dedicated LLM-driven topic-reduction step between clustering and reporting are the two most promising paths forward, and we treat this as the primary open problem in the horizontal pipeline.
Fig. 3. Representative horizontal pipeline execution showing: (a) LLM noise removal and summarization, (b) structured mid-level phrase extraction, and (c) mathematical cluster assignment. Fig. 6. Interaction transition heatmap for all issue threads combined.
B. Vertical Thread Analysis The transition heatmaps in Figures 4, 5, and 6 show interaction label transitions for closed threads, open threads, and the combined corpus respectively.
The ABCDE classifier reveals that most threads do not reach resolution: the majority end in disagreement, elicitation, or neutral commentary with no indication of subsequent progress. The resolution pattern differs only marginally between open and closed issues, so the combined corpus serves as the primary analytical unit. Across all threads, 49% ended in stagnation (D or E), 38% in neutral commentary (C), and 13% in resolution (A or B). Figure 7 shows that the presence of questions is associated with stagnant outcomes: many resolved and neutral threads contain no elicitation at all.
Fig. 4. Interaction transition heatmap for closed issue threads.
Fig. 7. Thread outcome distribution conditioned on the presence of at least one elicitation comment.
Fig. 5. Interaction transition heatmap for open issue threads.
Figure 8 shows the four diagnostic metrics computed over the full corpus. The resolution gap is the highest-scoring metric, confirming that unresolved discussions are the dominant bottleneck signal. The question-resolution rate and concerncapture rate are lower but meaningful, indicating that questions and concerns appear in threads but are not the primary friction source. The low fragmentation score suggests that conversations are not generally shallow or scattered; bottlenecks arise from resolution difficulty rather than from conversational disorganization.
Fig. 8. Pipeline diagnostic metrics over the combined open and closed corpus.
C. Bottleneck Detection Engine The bottleneck detection engine is evaluated through diagnostic visualizations that examine whether the pipeline identifies meaningful patterns of communication friction. Rather than treating detection as a supervised classification task, the evaluation focuses on interpretability and internal consistency between the scoring components. Figure 9 compares each issue’s negativity score against its resolution status, with points colored by bottleneck classification. Resolved issues cluster at low negativity and are predominantly classified as non-bottlenecks. Unresolved issues show a wider spread of negativity values and account for most detected bottlenecks, particularly as negativity exceeds the lower range. Point sizes reflect comment counts, and larger, more negative, unresolved threads tend to receive higher bottleneck scores. Negativity alone, however, does not characterize most unresolved threads, which underscores the importance of the multi-factor scoring approach.
Fig. 10. Distribution of compound bottleneck types across the issue corpus.
Figure 11 ranks the highest-scoring issues from both open and closed threads. The top issue reaches a bottleneck score of approximately 0.70, with the remaining high-priority issues ranging from 0.65 to 0.67. These represent discussions where multiple warning signals co-occur and where targeted maintainer review would likely yield the highest return.
Fig. 11. Top-ranked issues by bottleneck score across open and closed threads.
Fig. 9. Issue negativity score versus resolution status, colored by bottleneck classification. Point size reflects comment count.
Figure 10 shows the distribution of compound bottleneck categories across the corpus. The most common pattern is Stagnant and Long and Unresolved, confirming that the dominant signal is not emotional negativity but prolonged, directionless discussion. The categories Negative and Unresolved and Negative and Stagnant and Unresolved are the next most frequent, indicating that negative language contributes to bottleneck detection primarily when paired with unresolved outcomes. Multi-factor combinations involving all four signals are rare, suggesting that severe simultaneous failures are uncommon in the corpus.
VI. C ONCLUSION This paper presented SentTrack, a dual-lens framework for detecting socio-technical bottlenecks in GitHub repositories through automated analysis of issue thread conversations. Applied to the AvaloniaUI open-source repository across approximately 9,000 issue threads, the system demonstrated that real-time conversational data can surface workflow friction that traditional labels and periodic reviews consistently miss. The horizontal pipeline addressed RQ3 by introducing an intent-focused LLM summarization stage that isolates human narrative from machine-generated noise in mixed-media issue text, enabling downstream topic extraction to reflect actual developer concerns rather than code artifacts. The pipeline produced 613 semantic clusters from the first 3,608 issues. However, the discovery-rate analysis exposed a structural limitation: the LLM’s creative phrasing generates roughly two unique topic variants per issue, resulting in far more clusters than are actionable for a repository of AvaloniaUI’s scale. Constraining the extraction vocabulary or introducing a dedicated topic-reduction step after clustering are the most direct paths to resolving this fragmentation.
The vertical pipeline addressed RQ1 and RQ2 by demonstrating that comment-level ABCDE classification produces interpretable thread-level signals without requiring a large language model. Across the full corpus, 49% of threads ended in stagnation and only 13% reached resolution, with questions frequently unanswered and concerns rarely propagating into constructive follow-up. These patterns affect downstream pipeline quality as a cross-stage effect, confirming that conversation structure is as important as text content for NLP reliability. The bottleneck detection engine showed that the dominant risk pattern is not emotional negativity but rather issues that remain open while accumulating long, unproductive discussion, with Stagnant and Long and Unresolved as the most frequently detected compound type. Together, these results support the core premise of SentTrack: combining horizontal topic modeling with vertical thread analysis produces earlier and more interpretable signals of workflow breakdown than code metrics or manual labels alone. The primary path forward is consolidating the cluster space through a topic-reduction phase, after which bottleneck scores can be mapped directly to specific technical themes and give maintainers a unified view of both what is breaking down and where in the community it is happening. R EFERENCES [1] GitHub, About Issues, GitHub Docs, n.d., accessed: Mar. 6, 2026. [Online]. Available: https://docs.github.com/en/issues/trackingyour-work-with-issues/learning-about-issues/about-issues [2] ——, About Discussions, GitHub Docs, n.d., accessed: Mar. 6, 2026. [Online]. Available: https://docs.github.com/en/discussions/collaborating-with-yourcommunity-using-discussions/about-discussions [3] B. Pang and L. Lee, “Opinion mining and sentiment analysis,” Foundations and Trends in Information Retrieval, vol. 2, no. 1–2, pp. 1–135, 2008. [4] B. Liu, Sentiment Analysis and Opinion Mining, ser. Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers, 2012. [5] F. Jurado and P. Rodriguez, “Sentiment analysis in monitoring software development processes: An exploratory case study on GitHub’s project issues,” Journal of Systems and Software, vol. 104, pp. 82–89, 2015. [6] E. Guzman, D. Azócar, and Y. Li, “Sentiment analysis of commit comments in GitHub: An empirical study,” in Proceedings of the 11th Working Conference on Mining Software Repositories. ACM, 2014, pp. 352–355. [7] D. Pletea, B. Vasilescu, and A. Serebrenik, “Security and emotion: Sentiment analysis of security discussions on GitHub,” in Proceedings of the 11th Working Conference on Mining Software Repositories. ACM, 2014, pp. 348–351. [8] M. Ortu, B. Adams, G. Destefanis, P. Tourani, M. Marchesi, and R. Tonelli, “Are bullies more productive? empirical study of affectiveness vs. issue fixing time,” in 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories. IEEE, 2015, pp. 303–313. [9] M. Ortu, A. Murgia, G. Destefanis, P. Tourani, R. Tonelli, M. Marchesi, and B. Adams, “The emotional side of software developers in JIRA,” in Proceedings of the 13th International Conference on Mining Software Repositories. ACM, 2016, pp. 480–483. [10] M. R. Islam and M. F. Zibran, “SentiStrength-SE: Exploiting domain specificity for improved sentiment analysis in software engineering text,” Journal of Systems and Software, vol. 145, pp. 125–146, 2018. [11] F. Calefato, F. Lanubile, F. Maiorano, and N. Novielli, “Sentiment polarity detection for software development,” Empirical Software Engineering, vol. 23, no. 3, pp. 1352–1382, 2018.
[12] T. Ahmed, A. Bosu, A. Iqbal, and S. Rahimi, “SentiCR: A customized sentiment analysis tool for code review interactions,” in 2017 32nd IEEE/ACM International Conference on Automated Software Engineering. IEEE, 2017, pp. 106–111. [13] R. Jongeling, P. Sarkar, S. Datta, and A. Serebrenik, “On negative results when using sentiment analysis tools for software engineering research,” Empirical Software Engineering, vol. 22, pp. 2543–2584, 2017. [14] N. Novielli, F. Calefato, F. Lanubile, and A. Serebrenik, “Assessment of off-the-shelf SE-specific sentiment analysis tools: An extended replication study,” Empirical Software Engineering, vol. 26, no. 4, p. 77, 2021. [15] M. Imran, A. Menon, M. Abubakar, W. Mahmood, and M. Niazi, “Sentiment analysis tools in software engineering: A systematic mapping study,” Information and Software Technology, vol. 151, p. 107018, 2022. [16] D. Coutinho, L. Cito, M. V. Lima, B. Arantes, J. Alves Pereira, J. Arriel, J. Godinho, V. Martins, P. V. C. F. Libório, L. Leite, A. Garcia, W. K. G. Assunção, I. Steinmacher, A. Baffa, and B. Fonseca, ““looks good to me ;-)”: Assessing sentiment analysis tools for pull request discussions,” in Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering. ACM, 2024, pp. 211–221. [17] G. Colavito, F. Lanubile, N. Novielli, and L. Quaranta, “Impact of data quality for automatic issue classification using pre-trained language models,” Journal of Systems and Software, vol. 210, p. 111838, 2024. [18] T. Hirsch and B. Hofer, “Detecting non-natural language artifacts for denoising bug reports,” Automated Software Engineering, vol. 29, no. 2, p. 52, 2022. [19] J. R. Finkel, C. D. Manning, and A. Y. Ng, “Solving the problem of cascading errors: Approximate bayesian inference for linguistic annotation pipelines,” in Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2006, pp. 618–626. [20] T. Caselli, P. Vossen, M. van Erp, A. Fokkens, F. Ilievski, R. Izquierdo, M. Le, R. Morante, and M. Postma, “When it’s all piling up: Investigating error propagation in an NLP pipeline,” in Proceedings of the 20th International Conference on Applications of Natural Language to Information Systems. Springer, 2015, pp. 417–428. [21] C.-Y. Lin, “ROUGE: A package for automatic evaluation of summaries,” in Text Summarization Branches Out: Proceedings of the ACL-04 Workshop. Association for Computational Linguistics, 2004, pp. 74–81. [22] W. Kryściński, B. McCann, C. Xiong, and R. Socher, “Evaluating the factual consistency of abstractive text summarization,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2020, pp. 9332– 9346. [23] E. Clark, S. Rijhwani, S. Gehrmann, J. Maynez, R. Aharoni, V. Nikolaev, T. Sellam, A. Siddhant, D. Das, and A. Parikh, “SEAHORSE: A multilingual, multifaceted dataset for summarization evaluation,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2023, pp. 9397–9413. [24] J. Fu, S.-K. Ng, Z. Jiang, and P. Liu, “GPTScore: Evaluate as you desire,” arXiv preprint arXiv:2302.04166, 2023. [25] I. Augenstein, M. Das, S. Riedel, L. Vikraman, and A. McCallum, “SemEval 2017 task 10: ScienceIE – extracting keyphrases and relations from scientific publications,” in Proceedings of the 11th International Workshop on Semantic Evaluation. Association for Computational Linguistics, 2017, pp. 546–555. [26] A. Barua, S. W. Thomas, and A. E. Hassan, “What are developers talking about? an analysis of topics and trends in Stack Overflow,” Empirical Software Engineering, vol. 19, no. 3, pp. 619–654, 2014. [27] M. Grootendorst, “BERTopic: Neural topic modeling with a class-based TF-IDF procedure,” arXiv preprint arXiv:2203.05794, 2022. [28] L. McInnes, J. Healy, and J. Melville, “UMAP: Uniform manifold approximation and projection for dimension reduction,” arXiv preprint arXiv:1802.03426, 2018. [29] R. J. G. B. Campello, D. Moulavi, and J. Sander, “Density-based clustering based on hierarchical density estimates,” in Advances in Knowledge Discovery and Data Mining. Springer, 2013, pp. 160–172. [30] D. Moulavi, P. A. Jaskowiak, R. J. G. B. Campello, A. Zimek, and J. Sander, “Density-based clustering validation,” in Proceedings of the 2014 SIAM International Conference on Data Mining. SIAM, 2014, pp. 839–847. [31] A. Stavrianou, J. Velcin, and J.-H. Chauchat, “A combination of opinion mining and social network techniques for discussion analysis,” Revue
des Nouvelles Technologies de l’Information, vol. RNTI-E-17, pp. 25– 44, 2009. [32] P. Ravi, D. W. Lee, B. Flamia, J. David, B. Hanks, C. Breazeal, E. Anderson, and G. Lin, “Leveraging large language models to identify conversation threads in collaborative learning,” arXiv preprint arXiv:2510.22844, 2025. [33] A. Fan, B. Gokkaya, M. Harman, M. Lyubarskiy, S. Sengupta, S. Yoo, and J. M. Zhang, “Large language models for software engineering: Survey and open problems,” in 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering. IEEE, 2023, pp. 31–53. [34] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, S. Riedel, and D. Kiela, “Retrieval-augmented generation for knowledge-intensive NLP tasks,” in Advances in Neural Information Processing Systems, vol. 33, 2020, pp.
9459–9474. [35] J. Correia, M. C. Nicholson, D. Coutinho, C. Barbosa, M. Castelluccio, M. Gerosa, A. Garcia, and I. Steinmacher, “Unveiling the potential of a conversational agent in developer support: Insights from mozilla’s PDF.js project,” in Proceedings of the 1st ACM International Conference on AIPowered Software. ACM, 2024, pp. 10–18. [36] J. Correia, D. Coutinho, M. Castelluccio, C. Barbosa, R. de Mello, A. Sarma, A. Garcia, M. Gerosa, and I. Steinmacher, “A comparison of conversational models and humans in answering technical questions: The Firefox case,” arXiv preprint arXiv:2510.21933, 2025. [37] C. J. Hutto and E. Gilbert, “VADER: A parsimonious rule-based model for sentiment analysis of social media text,” in Proceedings of the International AAAI Conference on Web and Social Media, vol. 8, no. 1, 2014, pp. 216–225.