arXiv:2604.12123v1 [cs.SE] 13 Apr 2026
Mining Implicit Requirements Signals from Developer Crowds: Language Co-Usage Patterns on Stack Overflow Bachan Ghimire
Nitin Gupta
University of Victoria Victoria, British Columbia, Canada [email protected]
University of Victoria Victoria, British Columbia, Canada [email protected]
Abstract—Developer communities on platforms such as Stack Overflow constitute a crowd whose behavioral traces encode implicit requirements signals that are never explicitly stated yet continuously revealed through usage patterns. We present a three-phase pipeline that mines these signals from 36 million Stack Overflow posts by 435,803 developers across 186 programming languages, applying FP-Growth frequent itemset mining, Latent Dirichlet Allocation (LDA) topic modeling, and Louvain community detection on a weighted co-usage graph. We address two CrowdRE research questions: (RQ1) Can implicit behavioral traces from developer crowds be mined to identify stakeholder archetypes and their associated requirement constraints? (RQ2) Do crowd-derived community structures provide empirically grounded scoping boundaries that prevent inappropriate generalization of requirements across developer sub-populations? Results show that LDA yields 25 interpretable developer archetypes with distinct implied needs; FP-Growth rules with high lift encode requirement inseparability constraints; and Louvain partitions the language space into three communities that define valid requirement scoping boundaries. All three methods independently converge on the same macro-structure, providing a consistent and stable crowd-derived stakeholder map. This work contributes a novel CrowdRE approach that operates on passive behavioral data rather than explicit user feedback, addressing a gap identified in prior literature. Index Terms—Crowd-Based Requirements Engineering, Stack Overflow, Stakeholder Segmentation, Implicit Feedback, FPGrowth, LDA, Community Detection, Requirements Elicitation
I. I NTRODUCTION Requirements Engineering (RE) is the process by which the needs, constraints, and goals of stakeholders are elicited, analyzed, and documented as the foundation for software development. When done well, RE ensures that what is built actually serves the people who use it; when done poorly, it is one of the most persistent sources of project failure [1]. Traditional RE techniques, interviews, workshops, and ethnographic observation assume a small, reachable, and cooperative stakeholder population. As software systems increasingly serve millions of distributed users, these assumptions break down, and new approaches to elicitation and stakeholder understanding become necessary.
Developers rarely articulate what they need from tools, platforms, or languages. They reveal their needs through behavior: the questions they ask, the problems they encounter, and critically, the combinations of technologies they consistently use together. A developer who routinely posts about Python and SQL is implicitly signaling a requirement that their toolchain support both in combination, a constraint obvious to practitioners but invisible to any requirements process relying on explicit elicitation. Crowd-Based Requirements Engineering (CrowdRE) addresses precisely this scalability challenge, targeting settings where the stakeholder population is too large and distributed for traditional RE techniques [1]. It collects and analyzes feedback from crowds of users to surface requirements for software-intensive systems. To date, CrowdRE has focused predominantly on explicit feedback: app store reviews, forum posts, and structured user stories [2], [3]. Behavioral and usage data representing implicit feedback remain severely underexplored, despite their abundance and resistance to the response biases inherent in active elicitation [2], [4]. Stack Overflow1 is the largest publicly available record of developer behavior. Its question-and-answer corpus, tagged with programming language names, constitutes a crowdgenerated behavioral dataset that encodes implicit requirements at a scale no active elicitation method can reach. A developer who posts about Python and SQL is, with high probability, one who requires these technologies to interoperate. Aggregated across hundreds of thousands of users, this signal becomes a crowd-level requirements constraint, one that is continuously generated and freely available. We exploit this signal through three complementary analyses. FP-Growth frequent itemset mining surfaces requirement inseparability constraints: language combinations the crowd consistently uses together, indicating any tool targeting this crowd must support them jointly. LDA topic modeling produces a stakeholder segmentation: interpretable developer archetypes whose implied needs differ systematically. Louvain community detection on a co-usage graph produces a 1 https://stackoverflow.com
requirements scoping map: empirically grounded community boundaries within which requirements can be safely generalized. Together, these form a passive crowd feedback pipeline requiring no active elicitation and no prior knowledge of the developer domain.This leads to two research questions:
III. M ETHODOLOGY A. Dataset We use a Stack Overflow data dump 2 (upto 2024) comprising four tables: posts (36.1M rows), poststags (41.6M rows), users (7.25M rows), and a curated languages list of 262 valid programming language tag names used to filter out non-language tags such as frameworks and operating systems. Table I summarizes the data.
RQ1. Can implicit behavioral traces from developer crowds on Stack Overflow be mined to identify stakeholder archetypes and their associated requirement constraints? RQ2. Do crowd-derived community structures provide empirically grounded scoping boundaries that prevent inappropriate generalization of requirements across developer subpopulations?
TABLE I R AW DATASET FILES File posts poststags users languages
II. BACKGROUND AND R ELATED W ORK Crowd-Based Requirements Engineering. CrowdRE integrates methods for obtaining and analyzing user feedback from large, distributed stakeholder populations [1], [5]. Groen et al. identify four core CrowdRE activities: motivating crowd members, eliciting feedback, analyzing feedback, and monitoring context and usage data. Stakeholder segmentation, partitioning the crowd into sub-populations with coherent needs, is a central challenge: requirements derived from one segment may be inappropriate or contradictory for another [1]. Existing deployments address this through explicit pull feedback such as user stories and idea forums [3], or through argumentation over conflicting crowd opinions [1]. Implicit Feedback Gap. Wang et al.’s systematic mapping of crowdsourced RE [2] examined 44 papers and found a strong focus on crowd-generated textual comments, with usage-generated and behavioral data severely underrepresented, and explicitly calls for research leveraging implicit feedback alongside explicit sources. Radeck et al. [4] recently proposed combining implicit behavioral feedback with explicit pull feedback in CrowdRE, but their approach requires instrumented applications with direct user access. Lim et al.’s data-driven RE survey [6] similarly identifies process-mediated behavioral data as underexplored, noting that automated elicitation focuses almost exclusively on textual reviews. Mining Stack Overflow for Requirements. Ahmad et al.’s systematic review [7] examined ML-based approaches, including LDA topic modeling, for identifying software requirements on Stack Overflow, confirming its viability as a large-scale requirements source. Abdalkareem et al. [8] showed empirically that developers use Stack Overflow as crowd-generated knowledge, with programming language topics yielding the highest helpfulness scores among all categories studied. Research Gap. No existing work uses language co-usage behavioral patterns on Stack Overflow as a CrowdRE artifact. Prior SO mining targets textual content for explicit requirements [7], not structural co-occurrence patterns that reveal implicit stakeholder constraints. We address this gap by treating developer activity traces as passive crowd feedback and extracting stakeholder archetypes and scoping boundaries directly from co-usage structure.
Rows
Key columns
36,149,134 41,605,024 7,250,739 262
postid, posttypeid, parentid, userid postid, tag userid language
B. Processing Pipeline We implement a SQL-based analytical pipeline that transforms raw posts into per-user language baskets. Both questions (posttypeid=1) and answers (posttypeid=2) carry language signal: a user who answers a C++ question demonstrates C++ familiarity. Answers are mapped to their parent question via parentid to inherit the question’s tags. We join against the language list to retain only recognized programming language tags, dropping framework, OS, and tool tags. We count per-user language occurrences and apply two filters: (i) drop any (user, language) pair with usage count of one, removing incidental encounters; and (ii) drop any user with fewer than two qualifying languages, since single-language users carry no co-usage signal. Table II shows the effect of the filters. The pipeline is expressed in relational algebra as follows. post user languages: πpid,uid,tag σφQ (ptags ▷◁ptags.pid=q.pid ρq (posts)) ⊎all πpid,uid,tag σφA (ptags ▷◁ptags.pid=a.parid ρa (posts))
where φQ : tag ∈ πlang (langs) ∧ uid ∼ ˆ\d+$ ∧ pid ∼ ˆ\d+$; and φA : φQ ∧ parid ∼ ˆ\d+$ user results: σcnt>1 γuid,tag; cnt(∗)→c (ul )
final query: σcnt(uid)>1 γuid; agg(tag)→langs (ur )
2 https://shorturl.at/ogkXd
2
LDA’s probabilistic topics capture these mixtures, surfacing broader stakeholder archetypes, such as enterprise web developers, that cannot be reduced to simple itemsets. Graph community detection adds a global structural view by treating languages as nodes linked by weighted co-usage edges, revealing macro-level scoping boundaries that arise from the overall topology rather than local pairs. No single method provides all three perspectives; together they offer a richer and more actionable characterization of the requirement structure.
TABLE II P IPELINE OUTPUT STATISTICS Metric Total users with language activity Users after filtering Unique languages in baskets
Value 2,378,706 435,803 186
C. Analysis Methods FP-Growth – Requirement Inseparability Mining. Frequent itemset mining [9] finds all item sets whose support, the fraction of baskets containing the set, exceeds a threshold σ. Association rules A ⇒ B require minimum confidence P (B | A) ≥ γ. FP-Growth [10] avoids candidate generation by compressing transactions into a prefix tree (FP-tree), achieving significant speedups on dense datasets. We use the mlxtend library [11] with σ = 0.02 and γ = 0.5. In the RE context, lift, how much more likely a consequent is given the antecedent versus its base rate encodes requirement inseparability: high lift signals that the crowd demands joint support for two technologies. LDA – Stakeholder Archetype Discovery. LDA [12] models each document as a mixture of k latent topics, each a distribution over words. We treat each user’s basket as a bag-of-languages document and fit LDA with k = 25 using scikit-learn’s batch variational Bayes (random seed 0, 20 iterations). In the CrowdRE context, topics are latent stakeholder archetypes: segments of the developer crowd whose implied technology requirements differ systematically. Because users receive a mixture of topics, LDA captures developers who span multiple segments, a soft membership that discrete clustering cannot represent, and that mirrors the blended, multi-role nature of real developer profiles. Louvain Community Detection – Requirements Scoping. We construct a weighted co-usage graph G = (V, E, ω) where V is the set of 186 languages, E contains an edge for every co-occurring pair, and ω(u, v) counts users who use both languages. The resulting graph has 6,502 edges. We load it into Neo4j3 and run the Louvain algorithm [13], which maximizes modularity Q [14] by greedy merging. In the CrowdRE context, communities partition the crowd into sub-populations: requirements derived within one community cannot be safely generalized across community boundaries without empirical justification.
IV. R ESULTS A. RQ1: Stakeholder Archetypes and Requirement Constraints FP-Growth yields 106 frequent itemsets and 62 association rules. Table III shows the top rules by lift. TABLE III T OP ASSOCIATION RULES BY LIFT ( REQUIREMENT INSEPARABILITY ) Antecedent
Consequent
Supp.
Conf.
Lift
shell swift c, c# c++, python c, python c
bash objective-c c++ c c++ c++
0.025 0.033 0.027 0.031 0.031 0.078
0.610 0.670 0.746 0.514 0.661 0.613
11.06 8.35 4.20 4.04 3.72 3.45
Three distinct requirement inseparability clusters emerge. The shell scripting constraint (shell ⇒ bash, lift 11.06) is the strongest in the dataset: 61% of users who tag shell also tag bash. Any tool or documentation system targeting this crowd that treats shell and bash as separable components is misspecifying the requirement for this segment. The Apple platform constraint (swift ⇒ objective-c, lift 8.35) reflects the iOS/macOS developer profile, where both Swift and Objective-C are required across legacy and modern codebases simultaneously, a constraint that remains active as long as any production codebase contains Objective-C. The systems programming constraints (c ⇒ c++, lifts 3.45-4.20) appear in multiple multi-antecedent variants (c+python, c+c#), confirming that the demand for C and C++ joint support is robust and not an artifact of any single usage pattern. Java appears as a consequent in nearly a dozen rules whose antecedents span systems languages (c, c++), web languages (php, javascript), and enterprise languages (c#). This positions Java as a cross-segment requirement attractor: not the defining language of any single stakeholder archetype, but the language that most crowd segments demand simultaneously. Python and JavaScript appear frequently but with modest lift, indicating broad adoption across heterogeneous segments. For RE, highlift languages carry specific inseparability constraints, while low-lift languages represent generic capabilities required everywhere but constraining nowhere in particular. Fig. 1 visualizes the association rules with lift ≥ 2. The shell/bash and swift/objective-c dyads are isolated on the periphery, confirming their self-containment and the sharpness of their segment boundaries. The central cluster shows Cfamily rules converging on c++ and java.
D. Methodological Justification The three methods are complementary because each captures a different structural property of the crowd’s requirement space. FP-Growth identifies strong, discrete requirement inseparability constraints, pairs or small sets of languages the crowd consistently demands together, making it effective for tightly coupled stakeholder constraints. LDA complements this by modeling soft structure: developers often span multiple roles, and their language baskets reflect blended expertise. 3 https://neo4j.com/
3
Notably, Topics 3 and 17 both center on Unix scripting but differ in weight distribution: Topic 3 is Perl-heavy and Tool-oriented (sed, awk), while Topic 17 is Python-heavy. This reveals two sub-profiles within what a coarser analysis would treat as a single segment, with meaningfully different documentation and toolchain support needs. Topic 11 (not shown, ∼66k users, the largest) spreads weight broadly across PHP, JavaScript, and Java. This is the generic web developer profile: the largest crowd but the one whose requirements are hardest to characterize through co-usage alone, suggesting that explicit elicitation is more necessary here than for tightly bounded archetypes. Fig. 2 shows the topic-language weight heatmap for the 12 largest topics. Most topics concentrate on 2-4 languages, confirming that archetypes are interpretable and not artifacts of the k = 25 parameterization. Topic 9’s near-total concentration on swift and objective-c independently corroborates the FP-Growth inseparability rule. Topic 11’s broad distribution illustrates the undifferentiated web developer profile whose requirements resist compact characterization from behavioral data alone.
Fig. 1. Association rule network (lift ≥ 2). Edge color encodes lift: yellow = low, red = high. Peripheral isolated dyads represent the strongest requirement inseparability constraints. The central cluster shows C-family rules converging on c++ and java, with java acting as the cross-ecosystem attractor.
LDA produces 25 topics. Table IV shows selected topics with their top languages and user counts. TABLE IV S ELECTED LDA STAKEHOLDER ARCHETYPES ( TOP 5 LANGUAGES EACH ) Topic
Top languages
Users
3 9 14 17 20 22 23
perl, bash, sed, awk, python swift, objective-c haskell, lisp, scheme, rust, ocaml bash, shell, python typescript, javascript python, fortran, vhdl, verilog powershell, vbscript
10,781 15,933 5,644 20,007 6,762 28,892 6,893
Fig. 2. LDA topic-language weight heatmap (12 largest topics, top 8 languages each). Each row sums to approximately 1. Concentrated rows correspond to tightly scoped stakeholder archetypes with specific implied requirements; diffuse rows indicate generic segments requiring explicit elicitation to surface actionable constraints.
Each archetype represents a distinct crowd segment with unstated but derivable requirements. Topic 9 (Apple Platform Developer, ∼16k users) demands simultaneous Swift and Objective-C support in any IDE, debugger, or build system, treating either as optional is a requirements error for this segment, a conclusion corroborated independently by the FP-Growth rule. Topic 22 (Scientific/Hardware Developer, ∼29k users) implies Python-Fortran interoperability and co-simulation between high-level scripting and hardware description languages (VHDL, Verilog); tools that silo these languages poorly serve this segment’s fundamental workflow. Topic 14 (Functional/Academic Programmer, ∼6k users) is the smallest archetype but most sensitive to type system and runtime support quality, a niche whose requirements are specific and non-negotiable. Topic 23 (Windows Enterprise Automation, ∼7k users) is isolated from the Unix scripting community, with no meaningful co-usage with Bash or Shell, and requires entirely distinct Windows-native automation tooling.
B. RQ2: Community Structures as Requirement Scoping Boundaries Centrality. Table V ranks the top ten languages by total co-usage weight (sum of all edge weights incident to a node), capturing both breadth and intensity of co-occurrence. JavaScript has the highest total co-usage weight despite not having the highest degree, reflecting that its pairings are particularly frequent rather than merely numerous. Java has the highest degree (181 out of 185 possible neighbors), confirming its role as a cross-segment attractor identified by FP-Growth. For RE, this positions Java as a requirements conflict hotspot: a platform targeting Java faces simultaneous demands from web, systems, and enterprise crowd segments whose constraints can directly contradict each other. For example, a lightweight scripting-friendly API versus verbose enterprise boilerplate. Tools that ignore this conflict while attempting to serve all Java users simultaneously risk satisfying no segment well.
4
whose common thread is a Unix/open-source heritage rather than a single application domain. LDA reveals that this community contains at least two internally coherent subsegments: Topic 22 (scientific computing: Python, Fortran, VHDL, SPSS) and Topics 3 and 17 (Unix scripting: Perl, Bash, Sed, Awk). The community detector merges these because they share dense cross-usage through Python and Bash. For RE, the implication is direct: requirements targeting Community 2 should be further sub-scoped using LDA archetypes before any generalization is attempted, because a requirement valid for the scientific computing sub-segment may be irrelevant or harmful for the Unix scripting sub-segment. The low modularity (Q = 0.096) reflects the dense connectivity of the full graph: most of the 186 languages co-occur at least weakly with most others across the 435,803-user corpus, limiting the maximum achievable Q and underscoring that the three communities represent the strongest available topological signal rather than a weakly defined partition. Fig. 3 shows the co-occurrence heatmap for the top 20 languages. The javascript/java/php block and the python/c++ secondary cluster are visually separated, with C sitting at the boundary and co-occurring moderately with both communities. The asymmetry is meaningful for RE: the web cluster is dense internally but weakly connected to the systems cluster, confirming that requirements do not transfer across the Community 0/2 boundary.
TABLE V T OP 10 LANGUAGES BY TOTAL CO - USAGE WEIGHT Language
Degree
Total weight
javascript java php c# sql python c++ c bash ruby
177 181 175 174 173 177 177 174 163 158
550,975 418,450 403,473 322,120 299,412 287,996 258,387 211,684 115,720 110,066
Community Detection. The Louvain algorithm yields modularity Q = 0.096 and three communities (Table VI). TABLE VI L OUVAIN COMMUNITIES AS REQUIREMENT SCOPE BOUNDARIES ID
Representative languages
Size
0 1 2
javascript, java, php, c#, sql, ruby objective-c, swift, applescript python, c++, c, bash, shell, perl, r
43 10 133
Ecosystem boundaries are sharp for niche platforms. Community 1 (Apple Ecosystem, 10 languages) exhibits the clearest scoping boundary: all ten members are Apple-platform languages, including legacy tools (AppleScript, Automator) alongside modern ones (Swift), and no cross-community cousage edges exceed the 500-user threshold. Requirements derived from this community’s crowd, on IDE support, debugger behavior, or language interoperability, are valid within Community 1 and must not be generalized to Communities 0 or 2. The sharpness of this boundary is confirmed by all three methods independently: FP-Growth’s highest lift, LDA’s most concentrated topic, and Louvain’s most isolated community all identify the same segment, a convergence that strongly validates the boundary’s RE significance. The web and enterprise stack is large but loosely coupled. Community 0 (Web/Enterprise, 43 languages) is the most populous and contains the four highest-centrality languages: javascript, java, php, and c#. However, their pairwise association rule lifts are modest (1.0–1.5), and the dominant LDA topic (Topic 11, ∼66k users) spreads weight broadly without concentrating on any subset. This reflects a crowd that is large and interconnected but not tightly specialized: developers here tend to use many of these languages rather than pairing a specific two or three. From a CrowdRE perspective, this community yields no sharp inseparability constraints and its requirements are necessarily more diffuse, suggesting it is a strong candidate for active explicit elicitation to complement the passive behavioral signal. Systems and scientific computing share an underlying affinity. Community 2 (Systems/Scientific, 133 languages) is the largest and most heterogeneous, grouping C, C++, Python, Bash, R, Fortran, VHDL, and Verilog, languages
Fig. 3. Co-occurrence heatmap (top 20 languages). Darker cells indicate more shared users. The web/enterprise cluster (upper left) and systems/scientific cluster (lower right) are separated, with C bridging both communities and sitting at the requirement scoping boundary.
Filtering and crowd quality. The two-step filter reduces the user pool from 2.38M to 435k (an 82% reduction). This is itself a CrowdRE finding: the majority of Stack Overflow users who engaged with a programming language tag did so only briefly and incidentally, without demonstrated multi-language commitment. The 435k retained users represent developers with sustained, recurrent activity across multiple languages,
5
making the co-usage signal substantially cleaner than a naive union would provide. For CrowdRE practitioners, this establishes that raw participation count significantly overstates the size of the signal-bearing crowd, and that behavioral data quality improves substantially when filtered for sustained engagement. Synthesis. Across all three methods, a consistent structure emerges: a few tightly bounded niches (Apple ecosystem, Unix shell scripting) with sharp inseparability constraints; a broad but loosely organized web/enterprise region whose requirements are diffuse and cross-cutting; and a large systems/scientific cluster unified by Unix/open-source heritage that requires LDA sub-segmentation before requirements can be scoped. FP-Growth, LDA, and Louvain independently highlight the same boundaries, confirming the stability of the crowd-derived stakeholder map across methods with different theoretical assumptions. Java consistently acts as a crossecosystem connector, and the filtering step ensures the analysis reflects developers with sustained multi-language commitment rather than incidental crowd participants.
to the choice of k. Graph community detection provides a global structural view but loses the directionality of association rules and merges sub-segments that share cross-usage. Their convergence on the same macro-structure across different theoretical assumptions increases confidence in the resulting stakeholder map and demonstrates that the findings are not artifacts of any single method. Passive vs. Active Crowd Feedback. A key distinguishing property of this approach is that it requires no crowd engagement. The 435,803 developers contributed behavioral traces passively; no survey or review form was required. Passive data is abundant and unbiased by elicitation framing, but it captures what developers do rather than what they want changed. The most productive CrowdRE integration would combine this passive segmentation step with targeted explicit elicitation within the identified segments, following the hybrid model proposed by Radeck et al. [4]. The present work provides the segmentation layer; coupling it with active feedback mechanisms within each identified archetype remains a natural and important next step.
V. D ISCUSSION
VI. T HREATS TO VALIDITY
Implications for CrowdRE Practice. The pipeline addresses a concrete CrowdRE problem: before analyzing what a crowd says, one must know who the crowd is. Traditional CrowdRE methods require the crowd to self-identify through surveys or forum participation. Our approach derives a stakeholder taxonomy automatically from behavioral traces, requiring no active crowd engagement. For a tooling vendor, platform designer, or language maintainer, the archetypes in Table IV provide actionable starting points for requirements elicitation: rather than soliciting feedback from an undifferentiated developer crowd, requirements can be elicited segmentby-segment with domain-specific prompts. The scientific computing segment (Topic 22) calls for different elicitation instruments than the Windows enterprise automation segment (Topic 23) treating them as a single crowd would suppress the requirements of the smaller segment entirely. The community scoping map (Table VI) addresses the complementary problem of generalization [1]. CrowdRE practitioners who mine Stack Overflow without first segmenting by community risk conflating requirements from incompatible stakeholder groups. The three communities provide an empirical basis for scoping: feedback from the Apple ecosystem community is valid only within Community 1 and must not be projected onto the systems/scientific crowd. The LDA-derived sub-segmentation of Community 2 adds a second scoping layer for the largest community, which the graph topology alone does not resolve. Complementarity of Methods as a CrowdRE Contribution. The three methods provide views that no single technique can supply. FP-Growth identifies specific highconfidence inseparability constraints but cannot capture gradual or partial stakeholder membership. LDA captures soft developer archetypes and multi-segment membership but requires interpretation of probabilistic weights and is sensitive
Stack Overflow users are not representative of all developers. The platform over-represents web and systems programmers while under-representing embedded engineers and developers working in proprietary environments. This sampling bias may influence which languages appear central in the co-usage graph and which archetypes emerge from LDA, potentially making niche but important developer populations invisible to the analysis. The analysis relies on user-generated tags, which may contain noise. Although the curated language list filters nonlanguage tags, mis-tagging remains possible for languages with ambiguous names or overlapping ecosystems. The assumption that posting about a language implies active usage does not hold uniformly: some users ask about unfamiliar languages, while others answer questions outside their primary expertise. The two-step filter removes 82% of users, emphasizing active multi-language developers and potentially excluding casual or novice contributors whose requirements may differ systematically. The analysis also treats the dataset as static, ignoring temporal dynamics: trends such as the rise of TypeScript or the decline of Objective-C are invisible in the aggregate and could alter the archetypes and community structure if the dataset were segmented by year. Finally, methodological choices introduce their own limits. FP-Growth captures discrete associations and may miss diffuse requirements. LDA results are sensitive to the number of topics k; different values may yield alternative archetypes. Graph community detection is sensitive to graph density and modularity resolution limits, particularly in highly interconnected networks where the maximum achievable Q is structurally constrained. These threats do not invalidate the RE interpretations but warrant caution in applying the stakeholder profiles to rapidly evolving or specialized ecosystems.
6
VII. C ONCLUSION
but does not include a user study, industrial case, or concrete demonstration of a practitioner using these segments to improve an elicitation or prioritization decision. For a workshop contribution this is acceptable as a research-inprogress artifact, but a full empirical evaluation is necessary before the approach can be recommended for adoption. Future work should therefore report an empirical validation study in which requirements engineers or product managers are given the mined archetypes and scoping map as inputs to a real elicitation task, measuring whether segmented elicitation produces more specific, less contradictory, or better-prioritized requirements than an unsegmented baseline. Such a study would transform the current methodological contribution into a practically validated CrowdRE technique.
We presented a CrowdRE pipeline that transforms passive developer behavioral traces on Stack Overflow into structured RE artifacts: stakeholder archetypes with implied requirement profiles, requirement inseparability constraints from association rule lift, and a community-grounded scoping map. Across 435,803 developers and 186 languages, the three-method pipeline answers both research questions affirmatively: behavioral co-usage traces yield interpretable stakeholder segments with actionable implied needs (RQ1), and crowd-derived community structure provides empirically grounded requirement scoping boundaries that prevent inappropriate generalization across developer sub-populations (RQ2). The convergence of all three methods on the same macro-structure, tight niche communities, loose web/enterprise, merged systems/scientific requiring LDA sub-segmentation, increases confidence in the stability of the resulting stakeholder map. The contribution to CrowdRE is methodological: passive behavioral data, long identified as underexplored [2], can serve as a first-stage elicitation input that segments the crowd before any explicit feedback is solicited. The approach scales to any platform where developers produce observable behavioral traces, suggesting applicability beyond Stack Overflow to GitHub activity, IDE telemetry, or package dependency graphs.
R EFERENCES [1] E. C. Groen, N. Seyff, R. Ali, F. Dalpiaz, J. Doerr, E. Guzman, M. Hosseini, J. Marco, M. Oriol, A. Perini, and M. Stade, “The crowd in requirements engineering: The landscape and challenges,” IEEE Softw., vol. 34, no. 2, pp. 44–52, 2017. [2] C. Wang, C. Dan, P. Liang, M. Daneva, and M. van Sinderen, “A systematic mapping study on crowdsourced requirements engineering using user feedback,” J. Softw. Evol. Process, vol. 31, no. 10, 2019. [3] J. Wouters, R. Pinchuk, J. Doerr, and K. Vanderhulst, “Crowd-based requirements elicitation via pull feedback: method and case studies,” Requirements Eng., vol. 27, no. 3, pp. 307–332, 2022. [4] L. Radeck and N. Seyff, “Integrating implicit feedback into crowd requirements engineering: A research preview,” in Proc. IEEE RE’23 Workshops (REW), 2023. [5] E. C. Groen, “Crowd-based requirements engineering,” Ph.D. dissertation, University of Zurich, 2025. [6] S. L. Lim, D. Damian, and A. Finkelstein, “Data-driven requirements elicitation: A systematic literature review,” SN Comput. Sci., vol. 2, no. 1, 2021. [7] A. Ahmad, C. Feng, I. Ullah, A. Ullah, and S. Nazir, “A systematic literature review on using machine learning algorithms for software requirements identification on Stack Overflow,” Secur. Commun. Netw., vol. 2020, 2020. [8] R. Abdalkareem, E. Shihab, and J. Rilling, “What do developers use the crowd for? A study using Stack Overflow,” IEEE Softw., vol. 34, no. 2, pp. 53–60, 2017. [9] R. Agrawal and R. Srikant, “Fast algorithms for mining association rules,” in Proc. 20th Int. Conf. Very Large Data Bases (VLDB), 1994, pp. 487–499. [10] J. Han, J. Pei, and Y. Yin, “Mining frequent patterns without candidate generation,” in Proc. ACM SIGMOD Int. Conf. Management of Data, 2000, pp. 1–12. [11] S. Raschka, “Mlxtend: Providing machine learning and data science utilities and extensions to Python’s scientific computing stack,” J. Open Source Softw., vol. 3, no. 24, 2018. [12] D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent dirichlet allocation,” Journal of Machine Learning Research, vol. 3, pp. 993–1022, 2003. [13] V. D. Blondel, J.-L. Guillaume, R. Lambiotte, and E. Lefebvre, “Fast unfolding of communities in large networks,” Journal of Statistical Mechanics: Theory and Experiment, vol. 2008, no. 10, p. P10008, 2008. [14] M. E. J. Newman and M. Girvan, “Finding and evaluating community structure in networks,” Physical Review E, vol. 69, no. 2, p. 026113, 2004.
A. Future Work Future work should combine this passive segmentation with active elicitation within the identified segments, extend the pipeline to temporal slices to track how archetypes evolve, and incorporate user reputation and post scores as weighting signals to distinguish incidental language exposure from expert-level co-usage. Temporal analysis. The current pipeline treats the dataset as a static snapshot, aggregating activity across many years. Applying the same pipeline to year-segmented data would reveal how stakeholder archetypes evolve over time – for example, whether the TypeScript archetype (Topic 20) is displacing the generic JavaScript segment, or whether Rust is drawing developers away from the C/C++ inseparability cluster. Temporal archetypes would also make the scoping map responsive to ecosystem shifts rather than fixed to a single historical window. Integration with active elicitation. The present work provides the segmentation layer of a hybrid CrowdRE pipeline but does not close the loop to active feedback. A natural extension is to use the mined archetypes as crowd selection criteria for targeted pull feedback [3], [4]: rather than soliciting open-ended requirements from an undifferentiated developer population, elicitation instruments can be designed per archetype and distributed to the corresponding Stack Overflow sub-community. This would operationalize the segmentation results as inputs to a full, end-to-end CrowdRE process. Practitioner validation. The most significant limitation of this work is the absence of downstream validation. The paper derives stakeholder archetypes and scoping boundaries
7