Multi-Agent Systems are Mixtures of Experts: Who Becomes an Influencer?

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

Multi-Agent Systems are Mixtures of Experts: Who Becomes an Influencer?

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

machine learning, deep learning, neural networks

arXiv:2605.25929v1 [cs.MA] 25 May 2026

Multi-Agent Systems are Mixtures of Experts: Who Becomes an Influencer? Franka Bause1 Jonas Niederle1 Martin Pawelczyk2 Rebekka Burkholz1 1 CISPA Helmholtz Center for Information Security, Saarbrücken, Germany 2 Faculty of Computer Science, University of Vienna, Vienna, Austria {franka.bause, jonas.niederle, burkholz} @cispa.de [email protected]

Abstract The effectiveness of multi-agent LLM deliberation depends not only on the agents’ individual predictions, but also on how they communicate and collaborate. We study this mechanism through the lens of Friedkin-Johnsen (FJ) opinion dynamics, a tractable model for analyzing stubbornness, influence, and opinion change in multi-agent systems that captures empirically observed deliberation patterns. We show that the FJ parameters are input-dependent, turning multi-agent deliberation into a mixture of experts. This perspective implies that multi-agent systems can outperform single agents and static ensembles when routing reflects agent competence. Since competence is latent in practice, we analyze how influence is established through observable proxies: agents’ self-assessed confidence, their perceived confidence, and initial alignment with other agents’ views.

Introduction

Large language models (LLMs) have led to significant advancements in natural language processing across various tasks. Recently, multi-agent systems (MASs) composed of interacting LLMs have attracted attention for their potential to improve performance, particularly in tasks involving strategic reasoning, negotiation, and generative design [7, 38, 23, 3, 11]. In MASs, multiple agents communicate iteratively to deliberate and refine predictions, with the promise that diverse agents can contribute complementary expertise and improve decision-making. However, the empirical benefits of MASs over single-agent models or static ensembles are mixed [27, 39, 31]. A key factor in the success of MASs is understanding how influence is distributed during deliberation. Not all agents are equally persuasive, and the central question is: What makes some agents more influential than others in a multi-agent deliberation? We analyze this problem through the Friedkin-Johnsen (FJ) model of opinion dynamics [8], which has been successfully applied to model belief propagation in social networks and, more recently, in LLM deliberations [1]. While FJ dynamics have been used to assess security risks in MASs, we focus on the observation that the FJ model’s parameters are input-dependent, which leads to an important insight: multi-agent deliberation can be interpreted as a mixture of experts (MoE) system [13]. In this framework, each agent contributes to the final decision based on its initial belief and its influence weight, which varies depending on the input. Accordingly the MAS implicitly implements an adaptive routing mechanism. Its performance hinges on the degree to which influence is directed toward the most competent agents for a given input. Since true competence is latent, we focus on observable proxies for competence, such as self-assessed confidence, peer influence, and initial alignment with the group. Our findings demonstrate that relative confidence is the primary factor influencing routing, with initial alignment and social behavior also playing relevant roles. Preprint.

Disagreement W5

5 W1

Parameter value

1.0 0.8 0.6 0.4 0.2 0.0 (a) Parameter variability. W: weighted in-degrees (avg. over senders).

0.6 0.4 0.2 0.0

Initial

Final round

(b) Belief disagreement.

Figure 1: FJ induces MoE. (a) FJ parameter variability across MMLU-Pro questions for GPT-5.4 Mini. (b) Despite diverse initial opinions (left), consensus is often reached (right).

Contributions. 1) We use the Friedkin–Johnsen model to provide a tractable description of belief propagation, stubbornness, and peer influence in LLM-based MASs. 2) We show that input-dependent FJ parameters induce a mixture-of-experts interpretation of multi-agent deliberation. 3) We theoretically and empirically analyze how influence emerges from observable proxies for latent competence, identifying confidence and opinion alignment as central drivers of the implicit routing mechanism. Together, these results suggest that successful MAS design requires not only diverse agents, but also reliable mechanisms for routing influence toward agents that are locally competent. 1.1

Related work

LLM-based MASs and deliberation. Recent work explores the use of LLMs interacting through structured or unstructured communication to improve reasoning and robustness [38, 19]. Techniques such as self-consistency [33] and chain-of-thought prompting [35] demonstrate that aggregating diverse opinions and reasoning can outperform single-pass inference. However, existing methods largely lack a principled framework on how opinions evolve through communication and how influence shapes the outcome. Opinion dynamics and social influence models. The Friedkin-Johnsen (FJ) model [5, 8, 9] has been applied extensively in the social sciences to understand how group consensus is formed, how stubbornness and belief retention interact with peer influence, and how changes in network structure can affect collective decision-making [21, 30, 1]. Recently, it has been used to model opinions co-evolving with platform learning systems [37] and creating simulation environments of human opinion formation [10, 32]. Recent work has demonstrated that FJ dynamics can effectively capture the belief formation process in MASs to study systemic risk induced by stubborn agents [1]. Interpretability of MASs. Research on the interpretability of MASs has explored how agents’ behavior can be explained, particularly in the context of collaborative problem-solving and collective decision-making [36, 26, 18]. A challenge in multi-agent settings is understanding the emergent behaviors that arise from complex inter-agent interactions. Our work adds to this growing body of research by providing a framework for explaining influence in MASs through the lens of FriedkinJohnsen belief propagation [1] and a mixture-of-experts interpretation. Mixture of Experts. The concept of mixture-of-experts (MoE) models has been widely studied in the machine learning literature, where multiple models (or experts) are assigned responsibility for different regions of the input space. In these models, a gating network determines which expert(s) to trust based on the input, effectively routing the problem to the most relevant expert. MoE models have been used to improve performance on complex tasks by allowing specialized models to handle different subproblems [13, 15, 4, 28]. In contrast to the standard MoE setting, we focus on collaborative MAS, where routing is not explicitly learned in a supervised setting but implicitly realized through deliberation.

Theoretical framework: Agentic deliberation as mixtures of experts

Our work builds on the observation that LLM deliberation dynamics can be described by the FriedkinJohnsen model (see also Table 1), which enables us to cast multi-agent LLM systems as mixture of 2

experts. Accordingly, influence emerges as the result of a routing mechanism, which we set out to understand based on the confidence of agents and communication behavior. Friedkin-Johnsen (FJ) Model. Let each agent i ∈ V in an agentic network G = (V, E) with with n = |V | agents hold a belief bi (t) ∈ ∆d , where ∆d ⊂ [0, 1]d is the d-dimensional simplex representing a probability distribution over potential outcomes, e.g., answers to a multiple-choice question. Each agent i is characterized by an innate belief bi (0) = si ∈ ∆d , which may be interpreted as the agent’s individual prediction before deliberation. The belief update at time t + 1 in the Friedkin-Johnsen model [8] for belief dynamics is defined as [1]: bi (t + 1) =

X + (1 − γi )αi bi (t) + (1 − γi )(1 − αi ) wij bj (t), | {z } j∈N i Prior Belief Pull Belief Retention | {z } γi si |{z}

(1)

Peer Influence Pull

where γi ∈ [0, 1] denotes the attachment to innate beliefs also called stubbornness, αi ∈ [0, 1] represents P the weight given to the previous state, and W = [wij ] is a row-stochastic influence matrix where j wij = 1 (and wii = 0, as self-loops are covered by αi ). The term (1−γi )(1−αi ) represents the agent’s susceptibility to external influence. We can also formulate this in matrix notation. Let B(t) ∈ Rn×d denote the matrix whose i-th row is bi (t)⊤ , and let S ∈ Rn×d denote the matrix whose i-th row is s⊤ matrices Γ = diag(γ1 , . . . , γn ), A = diag(α1 , . . . , αn ), and H = i . Define diagonal (I − Γ) A + (I − A)W . Then the dynamics can be written compactly as B(t + 1) = ΓS + HB(t). As LLM deliberation dynamics are captured by the FJ model, the multi-agent system (MAS) computes a convex combination of the initial beliefs of its agents. Proposition 2.1 (Graph-induced convex combination [8, 22]). Assume that ρ(H) < 1, where ρ(·) denotes the spectral radius. Then the FJ dynamics converge to the unique equilibrium B ⋆ = (I − H)−1 ΓS. M := (I − H)−1 Γ = [mij ] is nonnegative and row-stochastic. Consequently, each Pn equilibrium belief is a convex combination of the innate beliefs: b⋆i = j=1 mij sj . If the agentic Pn Pn network belief b⋆out = i=1 ηi b⋆i = j=1 πj sj with π ⊤ = η ⊤ M is formed by a linear aggregation of all agents’ beliefs (typically ηi = 1/N ), the agentic network is equivalent to an ensemble with weights πj that depend on the FJ parameters (Γ, A, W ). A system that follows Friedkin-Johnsen dynamics with static parameters (Γ, A, W ) would therefore implement an ensemble. This can be already beneficial compared to a single-agent model, as ensembles can combine the opinion of diverse agents, achieve a variance reduction and are potentially more robust to distribution shift [6, 24, 25]. A direct implication of this insight is that agentic networks tend to benefit from high agent diversity. Yet, our empirical findings suggest that MASs can be even more powerful, as they do not rely on static FJ parameters, but adapt them to the input questions x, resulting in FJ parameters (Γ(x), A(x), W (x)) that induce input-dependent routing of agent influences πi (x). Hypothesis 2.2. A multi-agent LLM system (MAS) can be cast as mixture of experts (MoEs) b⋆ (x) = Pn j=1 πj (x)sj (x), where the router πj (x) depends on the input x. Fig. 1a provides empirical evidence for this hypothesis. It implies that the two central quantities of interest that determine the performance of the MAS are a) the diversity of agents and b) the routing. While a) is a critical design choice, it is often given and only mildly influenced by prompting in our LLM deliberation experiments. b) Routing is performed automatically by the LLM agents. The main purpose of the introduced FJ framework is to facilitate the discovery of its main underlying mechanisms. Our analysis reveals that routing primarily relies on initial beliefs si (x), even though these are not explicitly communicated by the LLM agents. However, they serve as proxy for the agents’ states that also informs their behavior. They define their confidence, competence, and to which degree their opinions align initially. Based on this hypothesis, i.e. an MAS acts as an MoE that performs routing based on initial agent beliefs (π = π(S)), for which we provide empirical evidence, our following theoretical discussion aims to derive conditions when MASs outperform single agents and even ensembles. 3

2.1

When do MASs outperform single agents and ensembles?

MASs have shown promising performance gains over single agents. This is especially true when the task has composition structure, agents contribute diverse information or reasoning strategies, and when communication allows for error correction [7, 38, 23, 11]. However, recent evaluations find only modest or inconsistent gains over strong single-agent baselines [27, 39, 2, 31], calling for a deeper understanding of successful MAS design principles. Our link to MoEs allows us to draw on theoretical insights. MoEs are a special case of ensembles. For ensembles, the routing is constant across inputs. While it is well known that agent diversity is critical to their function [6], theoretical studies of MoEs have focused on router properties [13, 15, 14] or how clustered structure in the input data can be exploited [4, 28] in the context of supervised learning problems, where the router has access to predictive features of labels. In our context of LLM deliberation, no router has been trained explicitly and the MAS does not have access to label information. Intuitively, an ideal router would assign high influence to competent agents. Yet, agent competence is input question dependent, latent, and therefore unavailable information. Agent beliefs, however, can be interrogated. We therefore ask the question: When can belief-derived signals identify which agent is locally competent and drive MAS performance? To answer this question, our analysis decomposes performance into three terms: global ensemble diversity, local competence diversity, and routing regret. This lets us compare single agents, static ensembles, and MoE-style deliberation in terms of agent diversity, complementary information, and confidence-based routing that rests on sufficiently well-calibrated agents, whose confidence reflects competence. In particular, we show that adaptive deliberation improves over fixed ensembling only when the gain from local specialization exceeds the diversity lost by no longer averaging all agents and the regret incurred by imperfect confidence-based routing. Belief-dependent routing, diversity, and performance. To formalize this intuition, we assume that each agent j ∈ [n] outputs an initial belief sj (X) ∈ ∆d about the right answer to question X, which is a random variable over the set of possible questions, andP write S(X) = (s1 (X), . . . , sn (X)) n for the collection of all beliefs. A MAS outputs f (X) = j=1 πj (S(X))sj (X) and its performance is measured with the Brier loss ℓ(y, p) = ∥p − ey ∥22 , where ey denotes the one-hot vector for belief in the correct answer y. For each agent, define the belief-conditional risk rj (S) = E ∥sj (X) − eY ∥22 | S(X) = S . This is the true local competence of agent j given the observable beliefs S,P which is generally unknown to the system. For any weight vector a ∈ ∆n , n define the mixture s̄a = j=1 aj sj and the local initial belief diversity, which measures how diverse the agents’ beliefs are under weights a: Da (S) =

n X

aj ∥sj − s̄a ∥22 =

j=1

1 X ai aj ∥si − sj ∥22 . 2 i,j=1

(2)

Lemma 2.3 (Local ambiguity decomposition). For any belief-dependent weights a(S) ∈ ∆n , 2 Pn Pn E | S = j=1 aj (S)rj (S) − Da(S) (S). j=1 aj (S)sj (X) − eY 2

This composition has the following interpretation: The first term rewards putting weight on locally competent agents. The second term rewards averaging diverse beliefs. MoE beats the best single agent if specialization gain plus local diversity exceeds routing regret. Comparing MAS, single agents, and ensembles. This composition enables us to compare different routing choices a(S) ∈ ∆n over set of all questions by taking an average also over X and S Pthe n additionally to Y , yielding L = j=1 E[aj (S)rj (S)] − E[Da(S) (S)]. Theorem 2.4 (MAS vs. single agent). A multi-agent system with the mixture of experts routing π(S) outperforms the best single agent j ∗ with lowest risk minj∈[n] E[rj (S)] if E[rj ∗ (S) − min rj (S)] + E Dπ(S) (S) > E[δπ (S)] , (3) j | {z } | {z } | {z } routing regret local diversity specialization gain

where δπ (S) =

j=1 πj (S)rj (S) − minj∈[n] rj (S) defines the routing regret.

The theorem is verified experimentally by Fig. 17. The proof follows directly from taking the average over the respective losses and demanding the loss of the MoE to be lower. The first term is the specialization advantage over the best global agent. It is large when no single agent is best everywhere. The second term is the local diversity benefit retained by the MoE. The right-hand side is the cost of imperfect routing. Thus, MoE beats the best single agent if specialization gain plus local diversity exceeds routing regret. This implies that a multi-agent system does not improve simply by combining many agents. Its agents have to be locally competent, complementary, and the routing mechanism can identify them from observable belief signals. However, note that this analysis does not exclude the existence of another more powerful and potentially more complex single expert model, which could e.g. be obtained by distillation of the MAS. It could also implement a similar mechanism as routing by choosing good answers from a sample like self self-consistency (SC) [39]. At the same time, such single models could potentially also be valuable experts in a MAS if they are combined with other complementary experts. Note that ensembles are special cases of MoEs and therefore follow a similar logic. However, they cannot achieve Pna local specialization gain through routing and the above condition simplifies to E[Da (S)] > j=1 aj E[rj (S)] − minj∈[n] E[rj (S)], highlighting the relevance of expert diversity. The difference in this analysis points also out how MoEs can potentially outperform ensembles, i.e. by exploiting local diversity. Theorem 2.5 (MAS vs. ensemble). A multi-agent system with the mixture of experts routing π(S) outperforms an ensemble with constant routing weights a if   n X E aj − πj (S) rj (S) > E Da (S) − Dπ(S) (S) . (4) | {z } j=1 diversity loss | {z } local routing gain

The left-hand side is the gain (resulting from lower average risk) from assigning more weight to locally competent agents. The right-hand side is the diversity that could be lost by moving away from the fixed ensemble. Thus, MoE wins if local competence routing gain exceeds lost ensemble diversity.

1.0 Belief in correct answer

Hard routing towards the most competent agent. As a special case, we can also consider hard routing, where ′ a single agent hPj (S) receives all weight,i leading to the n condition E j=1 aj rj (S) − rj ′ (S) (S) > E[Da (S)],

0.8

Correct Incorrect

because the local diversity Dπ(S) (S) = 0. Thus, hard routing must compensate for the diversity benefit with 0.6 a sufficiently better local experts. Our analysis of real LLM deliberation reveals a preference of MAS to select a 0.4 single or only a few experts, suggesting a strong tendency towards local specialization rather than exploiting diversity 0.2 r per MAS: 0.63 ± 0.61 through ensembling. Interestingly, even though the agents do not appear to be very diverse with respect to their 0.0 average performance, their intrinsic randomness leads to 0.00 0.25 0.50 0.75 1.00 local diversity in the their initial beliefs, which is exploited Confidence by the routing. In the following, we discuss scenarios how such routing is enabled by proxies for local expert Figure 2: Association of competence competence in the initial beliefs. and confidence for ChatGPT-5.4 Mini on MMLU-Pro. 2.2 Confidence-based routing as a competence proxy The local risks rj (S) are not observed during deliberation. The router must therefore use proxies derived from the beliefs. A natural self-assessment of competence is their confidence Cj (S) ∈ [0, 1] 5

concise

Answers

1 .75 .5 .25 0

emotional

Answers

1 .75 .5 .25 0

elaborate

Answers

1 .75 .5 .25 0

1 step-by-step 1.00 .75 0.75 .5 0.50 .25 0.25 Init Final 0 0.00 Answers Answers Indicator (intensity=conf.)

balanced

W value

1 .75 .5 .25 0

Correct

Incorrect

Figure 3: Example: Initial beliefs of agents (left). FJ weight matrix, γ, and color coded belief in correct answer (right). The most confident agent convinces the majority to change their answer. in a specific answer, which we propose to measure based on the entropy H(sj ) of their initial belief: d 1 X 1 sj,c log sj,c . (5) H(sj ) = 1 + log d log d c=1 Pn A confidence-based router may take the form πj (S) = exp(βCj (S))/ ℓ=1 exp(βCℓ (S)), where β ≥ 0 controls how strongly the router favors confident agents. Fig. 3 presents an example, where the MAS simply picks the opinion of the initially most confident agent. As we show, not only absolute confidence of agents matters but also confidence relative to the other agents. Thus, all predictions are taken into account.

Cj (S) = 1 −

Such routers are beneficial only when confidence is sufficiently calibrated with competence (see also Fig. 2). One way to formalize calibration is to assume that there exists a decreasing function ϕ such that rj (S) ≈ ϕ(Cj (S)). Equivalently, agents with higher confidence should have lower conditional risk. Let jC (S) ∈ arg maxj Cj (S) be the most confident agent, and let j ⋆ (S) ∈ arg minj rj (S) be the most competent agent. For hard confidence routing, the routing regret is δC (S) = rjC (S) (S) − rj ⋆ (S) (S). The confidence-routed MoE beats a fixed ensemble with weights a Pn if E[Ga (S)] > E[δC (S)] + E[Da (S)], where Ga (S) = Gw (S) = j=1 aj rj (S) − minj∈[n] rj (S) measures how much a fixed ensemble wastes probability mass on agents that are not locally optimal. This condition captures both the strength and the limitation of confidence routing. If confidence reliably identifies competence, then δC (S) is small and the MoE can exploit local specialization. If agents are overconfident when wrong, then δC (S) can be large and the MoE may underperform a fixed ensemble. To get an intuitive understanding of this, let us discuss two examples. Case: Mutually exclusive and optimally calibrated agents. Ideal agents of a MAS have a specialized local expertise that is complementary to other agents and signal reliably when they are competent. Optimal agents j with limited performance budget Bj would specialize and thus spend their budget on disjoint sets of questions. This would take the form Sn Let the input space decompose into n disjoint regions, X = j=1 Xj , Xi ∩ Xj = ∅ for i ̸= j with PN Pr(X ∈ Xj ) = ρj , ρj > 0, j=1 ρj = 1. On region Xj , agent j is competent and confident, while all other agents are uninformative. Let p = 1 − ε with ε ∈ (0, 1 − 1/d), and let u = 1/d denote the probability assigned to the true class by the uniform distribution. For every input x ∈ Xj , assume that agent j assigns probability p to the true class, while every other agent assigns probability u to the true class: sj,Y (x) = p, si,Y (x) = u for all i ̸= j. Thus, each agent is competent on exactly one region and uninformative elsewhere. PN A fixed ensemble with input-independent weights has a log loss Lens (a) = − j=1 ρj log u + (p − u)aj , while a confidence-routed MAS can achieve LMoE = − log p if the router assigns weight πj (S) = 1 to the agent with maximum confidence, which probability p to the correct answer. Proposition 2.6 (MoE advantage under mutually exclusive competence). Assume n ≥ 2, ρj > 0 for all j, and p > u. Then LMoE < inf a∈∆N Lens (a). For balanced regions ρj = 1/N for all j, symmetry implies that the optimal fixed ensemble is the uniform ensemble a⋆j = 1/N . It is simple to see that the loss gap then becomes L⋆ens − LMoE = log(1 − ε) − log(1/d + (1 − ε − 1/d)/(N ))]. Thus, this gap increases for a higher number of agents n and more questions d. Compared with the best single agent j ⋆ ∈ arg maxj ρj whose competence 6

region is largest, the loss gap L⋆single − LMoE = (1 − ρj ⋆ ) log up indicates that the MoE strictly improves over the best single agent whenever at least two regions have positive probability and p > u. Effect of routing errors. The preceding result assumes that confidence perfectly identifies the competent agent. Suppose instead that the router selects the correct agent with probability 1 − δ and selects an uninformative agent with probability δ. Then the routed system assigns true-class probability p with probability 1 − δ and u with probability δ, so its log loss is Lroute (δ) = −(1 − δ) log p − δ log u. In the balanced case, this improves over the optimal fixed ensemble if and only log p−log(u+ p−u N ) if δ < . Thus, confidence-based routing is beneficial only when the confidence log p−log u signal is sufficiently reliable. This formalizes the central limitation of task-dependent FJ deliberation: adaptive influence can realize MoE-like gains, but miscalibrated confidence can route trust to the wrong agent and erase the advantage. As experts are mutually exclusive in this ideal case, a MAS can generally route towards the single most competent expert, while the other agents focus their performance budget on other questions. Note that the maximum belief predictions of ensembles are still correct (albeit with lower certainty). As a consequence, expert optimization could make complex MAS interaction unnecessary. The real advantage of MoE routing is realized with imperfect agents, which are more common in practice. Case: Routing with imperfect agents. More interesting cases that highlight the advantages of MoE routing assume imperfect agents. Let us consider a scenario that is comparable with the observed case in Fig. 3, where the majority of agents makes a wrong prediction, confusing an ensemble, but the most confident agents convinces the rest during deliberation. As before, we observe disjoint regions of competence but with less well calibrated agents that correlate their belief in wrong labels. Let yk ∈ [d] denote the correct label on Xk . For every x ∈ Xk , assume that agent k assigns probability p to the true label with sk,yk (x) = p, while every other agent assigns probability u to the true label with sj,yk (x) = u, where 0 < u < 1/d < p < 1. Furthermore, all non-competent agents concentrate their mass on the same wrong label zk ̸= yk with sj,zk (x) = c for all j ̸= k with c > u. The remaining probability mass is distributed over the other labels. The competent agent assigns its remaining mass uniformly across incorrect labels sk,ℓ (x) = (1 − p)/(d − 1) for ℓ ̸= yk . We further assume that the competent agent is the most confident agent so that Ck (S(x)) > Cj (S(x)) for all j ̸= k, x ∈ Xk . It is easy to see that the MoE log loss for a router that focuses Pn on the most confident agent is LMoE = − log p and a fixed ensemble achieves Lens (a) = −1/n k=1 log u + (p − u)ak . Due to symmetry, the optimal ensemble assigns ak = 1/n. Proposition 2.7 (Confidence routing beats fixed ensembling). Assume n ≥ 3, d ≥ 2, 0 < u < 1/d < p < 1, and that the competent agent is the most confident one. Then the MoE strictly improves over p ⋆ the optimal fixed ensemble with gap Lens − LMoE = log p+(n−1)u . Furthermore, the uniform n

fixed ensemble predicts the wrong label if 1−a d−1 + (n − 1)c > a + (n − 1)b, while the MoE predicts correctly on every region. Strengths and limitations. This theoretical perspective clarifies both the strengths and the limitations of FJ-based multi-agent deliberation. Its strength is that deliberation can implement a task-adaptive mixture of agents: when the FJ parameters track competence, the system can route influence toward agents that are locally reliable. This enables gains from specialization that a fixed ensemble cannot realize. Its limitation is that routing depends on imperfect competence signals. If confidence is miscalibrated, if behavioral confidence is uninformative, or if opinion alignment merely reflects correlated errors, then the induced influence weights may amplify the wrong agents. In particular, high agreement among agents can be beneficial when it reflects independent corroboration, but harmful when it reflects shared bias. Similarly, high confidence can be beneficial when calibrated, but harmful when associated with overconfident mistakes.

Experiments

The main goal of our experiments is to demonstrate the utility of the Friedkin-Johnsen modeling approach and mixture of expert interpretation for studying the emergence of influence in MASs. Our experimental evaluation considers the MMLU-Pro [34], BBQ [20] and CommonsenseQA (CSQA) [29] dataset. We adopt the setup and subset of 100 questions of CSQA from [1] and sample 7

300 questions from MMLU-Pro and BBQ each (balanced for the categories). We conduct our experiments with two representative language models, GPT-5.4 Mini and Qwen2.5-14B-Instruct, enabling comparison across different model scales. We run our experiments over 3 different seeds. We use the same base system prompt for each agent and extend it to create diversity by prompting agents to assume different roles [17], like doctor, mathematician, or careless student, or answering in a specific communication style, e.g. concise, balanced, or emotional. Prompting agents to act as experts in supercategories [16] of the MMLU-Pro dataset and giving them access to specific example questions can also introduce agent diversity. The scenario neutral refers to no diversifying prompts being added in the system prompt. For the user prompts, communication between agents, and generating prompts for the next deliberation round, we use the setup of [1]. The prompts can be found in Appendix E. We use 5 agents, communicating in a complete graph structure over 5 rounds. Competence. We measure agent competence by the magnitude of their belief in the correct answer. This measure is unknown to the system but we can evaluate it based on known labels. To understand on what basis a MAS approximates agent competence, we define several other variables. Confidence. The confidence of an agent j is defined in Eq. (5). The relative-confidence of agent j is Cj (S) defined as Rj (S) = C(n−1) (S) where C(n−1) (S) is the second most confident agent in the system. Influence. The influence of agent j, normalized by the maximum agent influence, is defined as π Ij (S) = maxji πi , where πj is defined in Prop. 2.1. Peer Influence. Let B = A + (I − A)W and define B̃ as B with zero diagonal entries, i.e., B̃ij = Bij 1[i ̸= j]. Here, A and W refer to FJ matrices, as defined in Section 2. The peer influence Pn π peer of agent j is Pj = maxkj πpeer , where πjpeer = i=1 B̃ij and πjpeer is the j-th column sum of B̃. k

Disagreement. The disagreement of a systemP S is the mean P distance of each agent’s initial belief n n from the average system opinion: Di(S) = n1 j=1 sj − n1 i=1 si 2 . Alignment. The alignment of anP agent is given by the cosine similarity of its initial belief to the s ·1

i mean opinion: Alj (S) = ∥s ∥j ·∥N1 Pi=1 N s ∥ j 2

i=1

i 2

Alignment score. The alignment score is a binary variable, describing whether an agents initial an( PN 1, if arg max(sj ) = arg max( N1 i=1 si ) swer matches the groups initial answer: Asj (S) = 0, else

We empirically study the FJ dynamics in agentic systems and investigate how the routing weights are dependent on the different variables. If not indicated otherwise, the plots show results from the MMLUpro dataset with communication style prompts. Plots for the other configurations can be found in Appendix C and in the supplementary material.

Agent

Alignment count. The alignment count of an agent gives the number of other agents, that share the PN same initial answer: Acj (S) = i=1,i̸=j δarg max(sj ),arg max(si ) , where δ is the Kronecker delta.

0.42 0.47 0.49 0.36 Agent_0 ±0.09 ±0.15 ±0.13 ±0.03 0.46 0.38 0.48 0.46 Agent_1 ±0.17 ±0.05 ±0.11 ±0.05 0.45 0.50 0.53 0.56 Agent_2 ±0.09 ±0.07 ±0.10 ±0.02 0.35 0.41 0.36 0.43 Agent_3 ±0.12 ±0.13 ±0.08 ±0.05 0.45 0.47 0.42 0.51 Agent_4 ±0.18 ±0.06 ±0.20 ±0.02 s s ie ther ience stem o c anit ial s hum soc

0.55 0.50 0.45

Agentic system often outperforms ensemble. We 0.40 compare the performance of the agentic system to the baseline of taking the answer with maximum belief (averaging over the initial agent beliefs), and using 0.35 a task independent ensemble by fitting constant FJ parameters over all samples, optimizing for predictive performance. Table 3 in the appendix shows that agentic systems can perform better than both baseline and FJ ensemble, underlining the theory that the Figure 4: Influence (mean ± 95% confidence routing is input dependent, suggesting a MoE model. interval) for different communication styles, differences in influence suggest perceived MASs implement MoEs. Table 1 shows that the FJ confidence plays a role in FJ dynamics. model fits the dynamics of the agentic collaboration

well for all datasets, models and prompts styles. We can therefore leverage these fitted parameters to investigate the variability in agent weight between tasks. High variability of the FJ parameters over samples, as seen in Fig. 1a, suggests that the parameters are input dependent. Consequently, the final weighting of the agents’ initial beliefs, π in Proposition 2.1, are also input dependent, empirically confirming Hypothesis 2.2. Tendency towards consensus in MASs. Despite the diversity in initial opinions (see Fig. 1), agents consistently reach consensus in the final round. This is unexpected under FJ dynamics and only happens under specific conditions, e.g. when little or no stubbornness is present. Furthermore, we observe that influence is concentrated on a small subset of agent, as can be seen in Fig. 16 and in the individual samples in Fig. 18. Influence is strongly related to stubbornness (γ), as can be seen in Fig. 14b: Within an agentic system, stubborn agents become the most influential ones. The fact that a small subset of agents dominate the final consensus and that influence is input dependent, shows that agentic systems make strong routing decisions, and motivates the need for understanding this routing behavior. Competence is associated with influence. Fig. 15 in the appendix shows that there is a tendency for competent agents to get more influence, which is an indicator of effective routing. Particularly, confidence relative to other agents is a stronger predictor than absolute confidence, highlighting the social aspect of influence emergence.

2.0 1.5 1.0 0.5 0.0 0.5

Acc (train) = 0.708 ± 0.002 Acc (CV) = 0.733 ± 0.004

co n co fide mp nc ete e stu nce ma te dent the ac ma her tic Re l. C do ian on cto fi r Ali denc Ali gn e g m Ali nm. ent gn Sc m. ore Co un t

Confidence leads to influence. To understand the routing behavior, we investigate the relation between metrics of the initial beliefs and competence with influence of agents by training random forests to regress influence (Fig. 13a, appendix) to classify (Fig. 13b, appendix) if an agent becomes the most influential one. The models achieve a test R2 of 0.7 and accuracy of 0.9 for regression and classification, respectively, which suggests that influence can largely be explained by the considered variables. We also use logistic regression to gain insights into the relative impact of different variables. Fig. 5 shows that confidence both in absolute terms and relative to the second most confident agent is predictive of influence. Furthermore, competence, prompt style and alignment have an effect. Other datasets show similar behavior (see supplementary material). The positive relation of confidence to influence as well as peer-influence can also be observed in Fig. 8 in the appendix.

Coefficient

Interestingly, the communication style of an agent can change its influence, suggesting that perceived confidence also plays a role in the FJ dynamics (see Fig. 4 and Fig. 5.

Figure 5: Logistic regression coefficients classifying the most influential agent. Confidence and competence are the strongest positive predictors. Communication styles coefficients suggest perceived confidence plays a role in FJ dynamics.

Confident agents are more stubborn. Naturally, alignment with the majority opinion plays in a role in the rate to which an agent sticks to its opinion. Agents that align with the majority stick to their opinion more frequently. However, both for aligned and misaligned agents, confident agents tend to stick more to their initial belief, as can be seen in Fig. 10 in the appendix.

Conclusion

We have provided a framework to study the mechanisms of collaboration in multi-agent LLM systems Table 1: Model fit measured in KL divergence (MASs) and gained new insights into the factors gov- and MSE aggregated across datasets, prompt erning the emergence of influence, including the con- types, seeds and models. fidence of agents, their communication behavior and opinion alignment. Building on the observation that Metric Mean ± 95% CI 9

KL Divergence MSE

0.0529 ± 0.0028 0.0021 ± 0.0003

the Friedkin-Johnsen model captures MAS deliberation dynamics, we have cast MASs as mixtures of experts (MoE) with adaptive routing. This has allowed us to derive conditions when MASs can outperform simpler ensembles and single agent models. Accordingly, their strengths arise from adaptive routing and local specialization of diverse and well calibrated agents, while its limitations arise from miscalibrated agent confidence, misleading consensus, and routing errors. Acknowledgments We are grateful for funding from the European Research Council (ERC) under the Horizon Europe Framework Programme (HORIZON) for proposal number 101116395 SPARSEML.

References [1] Samira Abedini, Sina Mavali, Lea Schönherr, Martin Pawelczyk, and Rebekka Burkholz. Don’t trust stubborn neighbors: A security framework for agentic networks. CoRR, abs/2603.15809, 2026. [2] Mert Cemri, Melissa Z. Pan, Shuyi Yang, Lakshya A. Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandran, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. Why do multi-agent LLM systems fail?, 2025. [3] Shuaihang Chen, Yuanxing Liu, Wei Han, Weinan Zhang, and Ting Liu. A survey on llmbased multi-agent system: Recent advances and new frontiers in application. arXiv preprint arXiv:2412.17481, 2024. [4] Zixiang Chen, Yihe Deng, Yue Wu, Quanquan Gu, and Yuanzhi Li. Towards understanding mixture-of-experts layer in deep learning. In Advances in Neural Information Processing Systems, volume 35, 2022. [5] Morris H. DeGroot. Reaching a consensus. Journal of the American Statistical Association, 69(345):118–121, 1974. [6] Thomas G. Dietterich. Ensemble methods in machine learning. In Multiple Classifier Systems, pages 1–15. Springer, 2000. [7] Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. Improving factuality and reasoning in language models through multiagent debate. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 11409–11431. PMLR, 2024. [8] Noah E. Friedkin and Eugene C. Johnsen. Social influence and opinions. Journal of Mathematical Sociology, 15(3–4):193–206, 1990. [9] Noah E. Friedkin and Eugene C. Johnsen. Social Influence Network Theory: A Sociological Examination of Small Group Dynamics. Cambridge University Press, 2011. [10] Yulong He, Dutao Zhang, Sergey Kovalchuk, Pengyi Li, and Artem Sedakov. Opinion dynamics and mutual influence with llm agents through dialog simulation. arXiv preprint arXiv:2602.12583, 2026. [11] Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. Metagpt: Meta programming for a multi-agent collaborative framework. In International Conference on Learning Representations, 2024. [12] Zizhao Hu, Mohammad Rostami, and Jesse Thomason. Expert personas improve llm alignment but damage accuracy: Bootstrapping intent-based persona routing with prism, 2026. [13] Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan, and Geoffrey E. Hinton. Adaptive mixtures of local experts. Neural Computation, 3(1):79–87, 1991. 10

[14] Wenxin Jiang and Martin A. Tanner. Hierarchical mixtures-of-experts for generalized linear models: Some results on denseness and consistency. In Proceedings of the Seventh International Workshop on Artificial Intelligence and Statistics, volume R2 of Proceedings of Machine Learning Research, 03–06 Jan 1999. [15] Michael I. Jordan and Robert A. Jacobs. Hierarchical mixtures of experts and the em algorithm. Neural Computation, 6(2):181–214, 1994. [16] Changgeon Ko, Jisu Shin, Hoyun Song, Huije Lee, Eui Jun Hwang, and Jong C. Park. Social dynamics as critical vulnerabilities that undermine objective decision-making in LLM collectives, 2026. [17] Aobo Kong, Shiwan Zhao, Hao Chen, Qicheng Li, Yong Qin, Ruiqi Sun, Xin Zhou, Enzhi Wang, and Xiaohang Dong. Better zero-shot reasoning with role-play prompting. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), NAACL 2024, Mexico City, Mexico, June 16-21, 2024, pages 4099–4113. Association for Computational Linguistics, 2024. [18] Jae Hee Lee, Anne Lauscher, and Stefano V Albrecht. Towards ethical multi-agent systems of large language models: A mechanistic interpretability perspective. arXiv preprint arXiv:2512.04691, 2025. [19] Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. CAMEL: Communicative agents for ”mind” exploration of large language model society. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. [20] Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel R. Bowman. BBQ: A hand-built bias benchmark for question answering. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 2086–2105. Association for Computational Linguistics, 2022. [21] Sergey E. Parsegov, Anton V. Proskurnikov, Roberto Tempo, and Noah E. Friedkin. Novel multidimensional models of opinion dynamics in social networks. IEEE Transactions on Automatic Control, 62(5):2270–2285, May 2017. [22] Anton V. Proskurnikov and Roberto Tempo. A tutorial on modeling and analysis of dynamic social networks. part i. Annual Reviews in Control, 43:65–79, 2017. [23] Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, et al. Chatdev: Communicative agents for software development. In Proceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: Long papers), pages 15174–15186, 2024. [24] Abbavaram Gowtham Reddy, Celia Rubio-Madrigal, Rebekka Burkholz, and Krikamol Muandet. When shift happens - confounding is to blame. In The Fourteenth International Conference on Learning Representations, 2026. [25] Abbavaram Gowtham Reddy, Rajeev Verma, Celia Rubio-Madrigal, Krikamol Muandet, and Rebekka Burkholz. Boosting for predictive sufficiency. In The Fourteenth International Conference on Learning Representations, 2026. [26] Avi Rosenfeld and Ariella Richardson. Explainability in human–agent systems. Autonomous agents and multi-agent systems, 33(6):673–705, 2019. [27] Andries Petrus Smit, Nathan Grinsztajn, Paul Duckworth, Thomas D Barrett, and Arnu Pretorius. Should we be going MAD? A look at multi-agent debate strategies for LLMs. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 45883–45905, 21–27 Jul 2024. 11

[28] Dong Sun, Rahul Nittala, and Rebekka Burkholz. Robustness of mixtures of experts to feature noise. In Forty-third International Conference on Machine Learning, 2026. [29] Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. Commonsenseqa: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pages 4149–4158. Association for Computational Linguistics, 2019. [30] Ye Tian and Long Wang. Opinion dynamics in social networks with stubborn agents: An issue-based perspective. Automatica, 96:213–223, 2018. [31] Dat Tran and Douwe Kiela. Single-agent llms outperform multi-agent systems on multi-hop reasoning under equal thinking token budgets. arXiv preprint arXiv:2604.02460, 2026. [32] Chenxi Wang, Zongfang Liu, Dequan Yang, and Xiuying Chen. Decoding echo chambers: LLM-powered simulations revealing polarization in social networks. In Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, and Steven Schockaert, editors, Proceedings of the 31st International Conference on Computational Linguistics, pages 3913–3923, Abu Dhabi, UAE, January 2025. Association for Computational Linguistics. [33] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, 2023. [34] Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. MMLU-pro: A more robust and challenging multi-task language understanding benchmark. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. [35] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, 2022. [36] Michael Wooldridge. An Introduction to MultiAgent Systems. John Wiley & Sons, 2 edition, 2009. [37] Jiduan Wu, Rediet Abebe, and Celestine Mendler-Dünner. Opinion dynamics in learning systems, 2026. [38] Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryen W White, Doug Burger, and Chi Wang. Autogen: Enabling next-gen LLM applications via multi-agent conversations. In First Conference on Language Modeling, 2024. [39] Hangfan Zhang, Zhiyao Cui, Jianhao Chen, Xinrun Wang, Qiaosheng Zhang, Zhen Wang, Dinghao Wu, and Shuyue Hu. Stop overvaluing multi-agent debate – we must rethink evaluation and embrace model heterogeneity, 2025.

Theory: Proofs of theorems and discussion

Equilibrium beliefs are convex ensembles. The Friedkin-Johnsen dynamics induce a graphdependent ensemble over the initial agent beliefs. Proposition A.1 (Graph-induced convex ensemble). Assume that ρ(H) < 1, where ρ(·) denotes the spectral radius. Then the FJ dynamics converge to the unique equilibrium B ⋆ = (I − H)−1 ΓS. M := (I − H)−1 Γ is nonnegative and row-stochastic. Pn Consequently, each equilibrium belief is a convex combination of the innate beliefs: b⋆i = j=1 Mij sj . If the agentic network belief Pn Pn b⋆out = i=1 ηi b⋆i = j=1 πj sj with π ⊤ = η ⊤ M is formed by a linear aggregation (typically ηi = 1/N ) of all agents’ beliefs, the agentic network is equivalent to a fixed ensemble with weights πj that depend on the FJ parameters. Sketch. Unrolling the FJ dynamics gives B(t) =

t−1 X

H τ ΓS + H t B(0).

τ =0 t

If ρ(H) < 1, then H → 0 and the Neumann series converges: ∞ X

H τ = (I − H)−1 .

τ =0

Nonnegativity follows because H and Γ are nonnegative. Row-stochasticity follows from conservation of mass on the simplex: if all innate beliefs are equal to a constant simplex vector s, then all beliefs remain equal to s under the dynamics, implying M 1 = 1. Thus, deliberation over a fixed graph does not produce an arbitrary new predictor. It produces a graph-induced convex ensemble of the agents’ initial beliefs. For a designated readout agent r, the final prediction is n X b⋆r = mrj sj . (6) j=1

More generally, if the system output is a linear readout η ∈ ∆n over the final agent beliefs (typically an average with ηi = 1/N ), then b⋆out =

n X

ηi b⋆i =

i=1

n X

π ⊤ = η ⊤ M.

πj sj ,

j=1

Hence, a fixed agentic deliberation graph is equivalent to a fixed convex ensemble over agent priors, with weights determined by the network topology and the agents’ stubbornness, memory, and susceptibility parameters. Static deliberation versus mixture-of-experts deliberation. fixed Friedkin–Johnsen network implements a static ensemble: fFJ (x) =

n X

The preceding result shows that a

πj sj (x),

(7)

j=1

where sj (x) is the prediction of agent j on input x, and πj is independent of x. If, however, the graph, influence weights, or deliberation parameters depend on the input, then the equilibrium weights also become input-dependent. Writing W = W (x),

Γ = Γ(x),

we obtain M (x) = I − H(x) and therefore fFJ−MoE (x) =

n X j=1

−1

A = A(x), Γ(x),

πj (x)sj (x).

(8)

This is a mixture-of-experts architecture: the agents are experts, while the graph-dependent deliberation mechanism acts as a router or gating function. The static ensemble in (7) can only assign global importance to agents. By contrast, the mixture form in (8) can assign local, input-dependent responsibility to agents. Lemma A.2 (Local ambiguity decomposition). For any belief-dependent weights a(S) ∈ ∆n , 2 Pn Pn E | S = j=1 aj (S)rj (S) − Da(S) (S). j=1 aj (S)sj (X) − eY 2

Proof. For any fixed S and any label y, 2

aj ∥sj − ey ∥22 =

aj sj − ey

aj sj −

X k

ak sk

. 2

Taking conditional expectation over Y | S gives the claim. Proposition A.3 (MoE advantage under mutually exclusive competence). Assume n ≥ 2, ρj > 0 for all j, and p > u. Then LMoE < inf a∈∆n Lens (a). Proof. For any fixed ensemble weight vector a ∈ ∆N , the true-class probability on region Xj is qj (w) = u + (p − u)aj . Since aj ≤ 1 and p > u, we have qj (w) ≤ p, with equality if and only if aj = 1. For the fixed ensemble to match the MoE loss on every region withP positive probability, it would need aj = 1 for every j ∈ [N ]. This is impossible when n ≥ 2 and j aj = 1. Therefore at least one positive-mass region satisfies qj (w) < a, which implies Lens (w) > − log a = LMoE .

Experimental evaluation

We evaluate how well the FJ model fits in detail and present the results in Table 2. Over all datasets, we see a good fit. We compare the performance of the agentic system to the baseline of taking the answer with maximum belief (averaging over the initial agent beliefs), and using a task independent ensemble by fitting constant FJ parameters over all samples, optimizing for predictive performance. Table 3 that agentic systems can perform better than both baseline and FJ ensemble, underlining the theory that the routing is input dependent, suggesting a MoE model. In some cases, the agentic system slightly underperforms the baseline when prompted with specific persona or communication style prompts. Similar behavior was described in [12], which shows personas can potentially damage the discriminative performance. Regardless, utilizing these prompts introduces diversity in the agentic system, enabling a better study of its dynamics.

Additional Plots

We present additional plots for our experiments. More results can be found in the supplementary material. We see the same variability of the FJ parameters and tendency towards consensus in the other datasets and when using Qwen2.5-14B-Instruct as the underlying model (Figs. 6 and 7). Figs. 8 and 9 show the (peer-)influence of agents in relation to their confidence. We see that confident agents tend to be more influential. Fig. 10 shows the probability of an agent changing their answer to the majority answer when the agent is confident/not confident. We see that agents are more likely to change their answer if they are not confident. Interestingly, between the two models, the Qwen2.5-14B-Instruct agents are less 14

Table 2: Model fit (KL divergence and MSE) across datasets, prompt types, and models. Dataset Prompt Model KL MSE neutral MMLU-Pro

experts communication styles roles neutral

BBQ

experts communication styles roles neutral

CSQA

experts communication styles roles

GPT-5.4 Mini Qwen2.5-14B-Instruct GPT-5.4 Mini Qwen2.5-14B-Instruct GPT-5.4 Mini Qwen2.5-14B-Instruct GPT-5.4 Mini Qwen2.5-14B-Instruct

0.0760 ± 0.0133 0.0483 ± 0.0042 0.0640 ± 0.0100 0.0454 ± 0.0046 0.0664 ± 0.0076 0.0479 ± 0.0163 0.0719 ± 0.0136 0.0558 ± 0.0064

0.00227 ± 0.00097 0.00191 ± 0.00012 0.00182 ± 0.00042 0.00175 ± 0.00034 0.00196 ± 0.00042 0.00187 ± 0.00078 0.00207 ± 0.00059 0.00217 ± 0.00034

GPT-5.4 Mini Qwen2.5-14B-Instruct GPT-5.4 Mini Qwen2.5-14B-Instruct GPT-5.4 Mini Qwen2.5-14B-Instruct GPT-5.4 Mini Qwen2.5-14B-Instruct

0.0407 ± 0.0091 0.0391 ± 0.0052 0.0413 ± 0.0072 0.0520 ± 0.0068 0.0446 ± 0.0056 0.0343 ± 0.0040 0.0469 ± 0.0037 0.0492 ± 0.0030

0.000342 ± 0.000096 0.00113 ± 0.00010 0.000287 ± 0.000095 0.00145 ± 0.00049 0.000403 ± 0.000059 0.00101 ± 0.00015 0.000381 ± 0.000098 0.00165 ± 0.00037

GPT-5.4 Mini Qwen2.5-14B-Instruct GPT-5.4 Mini Qwen2.5-14B-Instruct GPT-5.4 Mini Qwen2.5-14B-Instruct GPT-5.4 Mini Qwen2.5-14B-Instruct

0.0450 ± 0.0041 0.0721 ± 0.0147 0.0457 ± 0.0001 0.0576 ± 0.0099 0.0452 ± 0.0044 0.0608 ± 0.0083 0.0467 ± 0.0031 0.0722 ± 0.0118

0.00218 ± 0.00023 0.00543 ± 0.00187 0.00228 ± 0.00017 0.00398 ± 0.00103 0.00218 ± 0.00022 0.00413 ± 0.00047 0.00242 ± 0.00030 0.00553 ± 0.00121

confident, but also less likely to change their answer to the majority answer when initially disagreeing with it. Figs. 11 and 12 show the impact of different diversifying prompts on the peer influence of the agents, grouped by question (super-)category on dataset MMLU-Pro. For both models, the influence of an agent seems to be driven also by perceived confidence, and roles seem to have a relatively big effect. Fig. 13 shows coefficient for random forests regression influence and classifying the most influential agent in an MAS. We observe a good fit for regression and a near perfect fit on the classification task. This shows, the defined variables are predictive of FJ dynamics. We investigate the relation between confidence and competence, as well as between influence and stubbornness in Fig. 14. We can see that an agents influence is highly correlated with its stubbornness. The relation between actual competence and influence is less prominent (see Fig.15). Fig. 17 shows the number of cases where Eq. 3 of Theorem 2.4 holds, compared to when it does not. We can see, that in most cases, the equation holds, and routing wins. Figs. 18 and 19 show weight heatmaps for different samples of the MMLU-Pro dataset (with communication style prompts). We can see that the weight matrices are task dependent.

Table 3: Accuracy of baseline, FJ ensemble, and agent system, across datasets, prompts, and models. Dataset

MMLU-Pro

BBQ

CSQA

Prompt

Model

Baseline

FJ Ens.

MAS

neutral