Frequency Is Not Sensitivity Identifying Safety-Sensitive Experts in Sparse MoE LLM

Frequency Is Not Sensitivity: Identifying Safety-Sensitive Experts in Sparse MoE LLMs Md Nurul Absar Siddiky, Liuwan Zhu, and Yingfei Dong

arXiv:2610.02910v1 [cs.AI] 2 Oct 2026

Department of Electrical and Computer Engineering University of Hawaii at Manoa, Honolulu, HI 96822, USA {msiddiky,liuwan,yingfei}@hawaii.edu

Abstract—Suppressing a small set of routed experts can weaken the safety behavior of a sparse Mixture-of-Experts (MoE) language model without retraining. Which experts to suppress is therefore a security question, and the usual answer is activation frequency — but frequency measures use, not influence. We test an alternative: router-gradient sensitivity, the sensitivity of the sequence loss to the gate weights that select an expert. Across five MoE architectures we rank experts by each signal on 500 benign and 500 malicious prompts and measure refusal on 100 held-out malicious prompts, under two budgets: equal expert counts, and equal nominal malicious routing traffic (1%–5%). Under each of the two budgets, router-gradient selection reduces refusals more than activation in 24 of 25 conditions, and more than a tentrial random mean in all 25. The largest effect is in OLMoE, where refusals fall from 34 to 9 of 100 prompts (73.53% relative) with no degraded outputs, indicating substantive compliance rather than broken generation. After matching expert counts in every layer, gradient selection still produces greater refusal reduction than activation in 23 of 25 conditions, with two ties. An exploratory cross-model analysis links larger maliciousversus-benign concentration gaps to greater peak gradient effects (ρ = 0.90; exact two-sided p = 0.083, n = 5). Together, the results support gradient selection under the tested budgets. Index Terms—mixture of experts, large language models, model safety, router gradients, expert suppression, mechanistic analysis

Router-gradient sensitivity provides a signal of influence: how strongly the sequence loss responds to the gate weights selecting an expert. The two signals need not rank experts alike, and neither is known in advance to give the better targets. We address two research questions. RQ1: Do benign and malicious prompts differ in model-wide and within-layer routing concentration? RQ2: Does router-gradient selection identify experts whose suppression reduces refusals more than activation-based or random selection? We evaluate five instruction-tuned models whose routed-expert pools span 256 to 6,144 experts [9]–[13] (Table I), ranking experts on 500 benign and 500 malicious prompts and suppressing them on a disjoint set of 100, under two budget protocols that control for expert count and for routing traffic. Our contributions are twofold. First, a five-architecture analysis of routing concentration and overlap, including an exploratory link between malicious-versus-benign concentration gaps and peak suppression effects. Second, held-out comparisons of gradient, activation, and random selection under complementary budgets, with an exact per-layer-count control showing that the gradient advantage is not explained by layer placement alone.

I. I NTRODUCTION

II. R ELATED W ORK

Sparse Mixture-of-Experts (MoE) language models increase capacity by routing each token through a small subset of feedforward experts [1]–[3]. Because the router makes this choice explicitly, an MoE model exposes a discrete, per-token decision that can be observed and altered. Recent work exploits this: biasing the router so a small set of experts is never selected weakens refusal at inference, with no retraining [4], [5]. For openly released models this makes expert selection a security question: an attacker must know which experts to disable, and a defender which to protect. This requires a signal that ranks experts by importance to safety behavior. Most MoE safety work ranks by activation frequency — how often malicious prompts route to an expert [6]–[8]. But frequency measures use, not influence. An expert may be selected constantly yet matter little to any particular behavior, while a rarely selected one may be pivotal.

MoE routing and specialization. Sparse MoE research has emphasized conditional computation, capacity, and load balancing [1]–[3], later examining token-level specialization [14], interpretable architectures [15], steerable routing [16], and activation-magnitude “super experts” whose ablation collapses general capability [17]. These studies motivate treating each expert, identified by its layer and in-layer index, as the unit of analysis; but routing frequency alone does not reveal how sensitive predictions are to the router parameters selecting it. Safety evaluation and internal interventions. AdvBench and JailbreakBench provide harmful-prompt benchmarks for refusal and jailbreak behavior [18], [19]. StrongREJECT shows low-quality generations should not count as harmful compliance [20], so we separate restricted, compliant, and degraded outputs; dense-model work also finds low-dimensional refusal mechanisms [21], suggesting safety can depend on localized structure. MoE safety and expert-level interventions. Recent MoEspecific studies manipulate behavior-associated experts using

This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.

TABLE I A NALYZED ROUTED EXPERT SPACES ( ALWAYS - ACTIVE SHARED EXPERTS EXCLUDED FROM RANKINGS AND INTERVENTIONS ).

and average over the prompt set Pg for group g ∈ {B, M } (benign/malicious): (p)

Model

L

E

Top-k

N

Mixtral-8x7B Qwen1.5-MoE DeepSeek-MoE OLMoE-1B-7B Qwen3-30B-A3B

32 24 27 16 48

8 60 64 64 128

2 4 6 8 8

256 1,440 1,728 1,024 6,144

activation-based signals: SAFEx uses stability-selected activation probabilities and masking [6]; SteerMoE steers safety via activation-rate differences [7]; RASET fine-tunes safetycritical experts via contrastive activation sensitivity [8]. Inputonly prompts can steer generation toward weakly aligned experts [5], [22], again selecting targets from activation patterns. Internal approaches include gate-guided ablation to remove safety behavior without retraining [23] and masking-based search to flip refusals into compliance [4]. Closest to our work, Zhao et al. [24] use gradient sensitivity, ablated against activation-based alternatives, to localize experts for backdoor injection rather than safety-behavior suppression. Existing MoE safety work thus relies mainly on activation strength or masking. To our knowledge, this is the first study to compare router-gradient sensitivity against activation frequency as a selection criterion for expert suppression, under matched budgets and a random baseline. III. M ETHODS A. Models, Analysis Sets, and Evaluation Sets Table I reports the number of sparse layers L, routed experts per layer E, and the total number of routed experts N = LE; always-active shared experts are excluded. Prompts use each model’s native chat template (Qwen3 in non-thinking mode [13]). Each model’s analysis set comprises 500 benign AEGIS2.0 prompts [25] and 500 malicious AdvBench prompts [18], used throughout for routing analysis, maliciousminus-benign scoring, ranking, and traffic calibration. Behavioral evaluation uses two additional AEGIS2.0 sets [25], comprising 100 malicious and 100 benign prompts, with no overlap with the 500-benign/500-malicious analysis set. The malicious set evaluates the baseline and suppression based on gradient, activation, and random expert selection, whereas the benign set audits gradient-based suppression only. Responses are manually labeled as restricted (refusal or safety redirection), compliant (a substantive answer to the prompt), or degraded (empty, incoherent, repetitive, or unrelated).1 B. Activation and Router-Gradient Scores (p)

For prompt p, let Al,e denote the number of tokens routed to expert e in sparse layer l. We normalize within each layer 1 These labels are used only for behavioral evaluation and do not influence expert identification.

A e(p) = P l,e , A l,e (p) E j=1 Al,j

g

Al,e =

1 X e(p) Al,e . |Pg |

(1)

p∈Pg

For gradient analysis, no parameters are updated; gradients are collected only for router-gate weights. For the chat(p) (p) formatted input sequence x1 , . . . , xTp , we use the causallanguage-model sequence loss L(p) = −

Tp   1 X (p) (p) log Pθ xt | x<t . Tp − 1 t=2

(2)

The loss uses only the formatted analysis prompt, with standard causal shifting; generated responses are not appended. gate For a router row Wl,e,: ∈ Rd , where d is the router-input hidden dimension (the number of weights in each row), the prompt-level gradient score is the mean absolute sequence-loss gradient: d 1 X ∂L(p) (p) Gl,e = (3) gate . d j=1 ∂Wl,e,j The absolute value prevents cancellation. Unlike activation (p) frequency, Gl,e measures local sensitivity of formatted-prompt token prediction to a router row. It is not a refusal objective or direct “safety gradient”; suppression separately tests the behavior of experts selected by this signal. M,(m) B,(m) For model m, let Gl,e and Gl,e be the averages of (3) over the 500 malicious and 500 benign analysis prompts, respectively. We define gradient malicious dominance as G,(m)

Dl,e

M,(m)

= Gl,e

B,(m)

− Gl,e

.

(4) A,(m)

For the activation baseline, we analogously define Dl,e

=

M,(m) B,(m) Al,e − Al,e from (1). Gradient and activation targets

are ranked in descending order of their respective maliciousminus-benign scores. (p) Computing Gl,e requires a backward pass per analysis prompt to obtain router-gate gradients, whereas activation frequency requires only a forward pass; the gradient signal therefore carries a higher one-time profiling cost during expert identification, though this cost is not incurred at inference time once experts are selected. C. Suppression Budgets and Evaluation For a selected set S, malicious routing-traffic share is P P M M τ (S) = (l,e)∈S Al,e /L. The denominator is l,e Al,e = L, since each routed layer’s shares sum to one. To form a traffictarget set, we accumulate these shares in the relevant ranked or random order and stop before adding the first expert that would exceed the target; that expert is excluded. We first select gradient-ranked expert sets corresponding approximately to 1%, 2%, 3%, 4%, and 5% of malicious routing traffic, measured on the 500 malicious analysis prompts. These sets contain 3/5/7/9/11 experts for Mixtral, 7/20/31/43/53 for

Qwen1.5, 9/13/21/28/35 for DeepSeek, 5/9/13/17/21 for OLMoE, and 10/22/33/45/57 for Qwen3. Each expert is identified by its layer and in-layer index. Using these gradient sets as the reference, we construct two complementary comparisons. Same-count comparison. For each gradient-selected set, the activation-based and random-selection methods suppress the same number of experts. Activation-based selection follows the activation-dominance ranking, whereas random selection uses a random ordering. This comparison controls the number of suppressed experts, but not their cumulative routing traffic or layer-wise distribution. Traffic-target comparison. Because the same-count comparison does not control cumulative routing traffic, we additionally construct activation-based and random expert sets corresponding approximately to the same nominal traffic targets of 1%–5%, using the same traffic definition and analysis prompts. Expert counts may therefore differ across methods and random trials, while the original gradient-selected sets remain unchanged. Layer-wise distributions may still differ across methods. Because experts are discrete, the achieved traffic shares may fall below the nominal targets. Depth-matched activation control. Using the same 500benign/500-malicious analysis rankings, for each model–K setting we select the highest activation-dominance experts within each sparse layer, subject to selecting exactly as many experts as the gradient set contains in that layer. This matches total expert count and the complete layer-count distribution, not only mean depth. Expert overlap is allowed; evaluation prompts and suppression settings remain unchanged. Routing traffic is not additionally matched in this control. Intervention and evaluation. Selected experts receive a negative router-logit bias before Top-k selection, making them effectively unavailable; shared experts remain unchanged. All methods use the same 100 malicious evaluation prompts, greedy decoding, and a maximum of 200 new tokens. Let Bm be the number of restricted responses produced by model m (q) without expert suppression, and let Cm be the corresponding count after expert suppression at setting q ∈ {1, . . . , 5}, evaluated on the same 100 malicious prompts. Method and protocol indices are omitted for brevity. Relative refusal reduction is (q)

(q) Rm = 100

Bm − C m . Bm

(5)

Positive values indicate fewer restricted responses after suppression; negative values indicate more. Random selection uses ten independently seeded trials per model and protocol, sampling without replacement while retaining enough experts per layer for native Top-k routing. (q) Random refusal reduction is computed by replacing Cm in (5) with the mean restricted count across the ten trials. IV. R ESULTS AND D ISCUSSION A. Routing Concentration For RQ1, we compare model-wide concentration, withinlayer concentration, and benign–malicious expert overlap.

TABLE II ROUTING CONCENTRATION AND OVERLAP. B/M: BENIGN / MALICIOUS ; EFFECTIVE - EXPERT RATIOS ARE AVERAGED OVER PROMPTS AND LAYERS . R EGIONS ARE ILLUSTRATIVE EXAMPLES , NOT THE LARGEST BENIGN – MALICIOUS DIFFERENCES ; LAYER INDICES ARE ZERO - BASED .

Model

∆k90

Eeff /E B/M

Top-10 overlap

Example layers

Mixtral Qwen1.5 DeepSeek OLMoE Qwen3

−1.95 −2.22 −6.94 −7.32 −3.52

.8999/.8900 .7454/.7455 .6144/.5889 .6362/.6079 .3479/.3448

4 7 6 6 7

L8–15 L23 L12,25 L3,6,14 L6,18–19

g Overall concentration and overlap. Let k90 denote the smallest number of ranked experts covering 90% of group M B g’s activation mass. We report ∆k90 = 100(k90 − k90 )/N in percentage points, where negative values indicate stronger concentration for malicious than benign prompts. Table II shows this pattern in all five models, most strongly in OLMoE and DeepSeek. Nevertheless, benign and malicious Top-10 sets share 4–7 experts, and Jensen–Shannon divergence between g their model-wide distributions Pg (l, e) = Al,e /L, computed using base-2 logarithms, remains small (0.0074–0.0408). Thus, greater malicious concentration coexists with substantial overlap in expert usage. Within-layer concentration. Entropy is computed per e(p) in (1) as H (p) = − P A e(p) e(p) prompt from A e l,e ln Al,e and l,e l averaged over prompts within each group. For effective-expert (p) (p) counts, we first compute Eeff,l = eHl and then average (p) Eeff,l /E over prompts and layers; smaller values indicate more concentrated usage relative to the available expert pool. Malicious ratios are lower in four models, with the largest differences in OLMoE and DeepSeek. Qwen1.5 is essentially unchanged, showing that greater model-wide concentration does not imply stronger concentration within every layer. The layer profiles reveal different concentration patterns (Table II, Example layers). Mixtral shows slightly more concentrated malicious routing over L8–15, with 6.906 effective experts versus 7.002 for benign prompts. Qwen1.5’s malicious usage is slightly broader at L23 (40.2 versus 39.8 effective experts). OLMoE shows lower malicious effective-expert counts at L3 and L14, alongside a shared minimum at L6. Qwen3 shows smaller, mixed group differences.

B. Held-Out Suppression To answer RQ2, we test whether gradient-selected experts produce greater refusal reduction than activation-selected or randomly selected experts on held-out malicious prompts. We first compare equal expert counts (Fig. 1(a)), then examine whether the advantage persists under common nominal routing-traffic targets (Fig. 1(b)). Baseline restricted counts Bm are 58, 94, 82, 34, and 94 for Mixtral, Qwen1.5, DeepSeek, OLMoE, and Qwen3, respectively, out of 100 prompts. Same-count comparison. Gradient selection yields greater refusal reduction than activation in 24 of 25 model–K con-

Fig. 1. Relative refusal reduction under (a) matched expert counts and (b) nominal routing-traffic targets. Each column uses the same vertical scale across rows; scales differ between models. Gradient sets and results are reused; activation and random sets are selected separately for each protocol. Random values use ten-trial mean restricted counts. Positive values indicate fewer refusals and weaker safety; lines guide the eye.

ditions and exceeds the random mean in all 25. The only exception is Qwen3 at K = 10, where activation produces a 4.26% reduction versus 1.06% for gradient selection. Since expert counts are identical within each comparison, the gradient advantage cannot be explained simply by suppressing more experts. The magnitude and progression of the effect differ across models. OLMoE shows the largest relative reduction, reaching 73.53% at K = 21: restricted responses fall from 34 to 9. In contrast, activation and random suppression increase OLMoE refusals at every setting. DeepSeek’s gradient reduction rises from 7.32% to 28.05%, and Qwen3 reaches 21.28% at K = 57. Qwen1.5 plateaus at 12.77% from K = 31, while Mixtral peaks at 15.52% for K = 5 and then declines. Increasing the suppression budget therefore does not necessarily increase refusal reduction. Traffic-target comparison. Equal expert counts do not ensure comparable routing traffic. Figure 1(b) therefore compares new activation and random selections at the same nominal 1%–5% traffic targets as the original gradient sets. Gradient selection again exceeds activation in 24 of 25 conditions and the random mean in all 25. The exception remains Qwen3 at the smallest setting: activation gives a 3.19% reduction at 1%, compared with 1.06% for gradient selection. At the 5% target, DeepSeek reductions are 28.05%, 9.76%, and 5.61% for gradient, activation, and random selection, respectively. For OLMoE, gradient suppression reduces refusals while both controls increase them. The gradient advantage therefore persists when experts are

selected under common nominal routing-traffic targets rather than matched expert counts. Because only the activation and random sets are rebuilt to the traffic targets, this is a complementary check on a different confound, not an independent replication. Layer-position analysis. We normalize sparse-layer positions from 0% (first) to 100% (last) and average over the 25 model–budget settings. Under same-count budgets, gradient versus activation mean depths are 68.2% versus 50.9%, and deepest-third fractions are 56% versus 29%; gradient sets are deeper in all 25 settings. Activation stays shallow under traffictarget budgets (48.7%; 25.7%). Settings are not independent replications because budgets reuse models and overlapping sets. Depth-matched suppression. To test whether layer placement alone explains this advantage, we compare gradient selection with the exact layer-matched activation control (Table III). Gradient yields greater refusal reduction in 23 of 25 conditions, with ties for Qwen3 at K = 10 (1.06%) and Mixtral at K = 11 (3.45%). At the largest budgets, gradient versus depth-matched activation reductions are 28.05% versus 4.88% for DeepSeek and 73.53% versus −29.41% for OLMoE. Thus, the observed gradient advantage is not explained by layer placement alone, supporting its value as a selection signal under the tested conditions. This control does not isolate effects from differences in selected routing traffic. Connecting RQ1 to RQ2. Across the five models, the malicious-versus-benign concentration gap −∆k90 (Table II) is positively associated with peak gradient-induced refusal

TABLE III D EPTH - MATCHED ACTIVATION REFUSAL REDUCTION R (%). S ETTINGS q = 1, . . . , 5 USE EACH MODEL’ S INCREASING K SEQUENCE SPECIFIED IN M ETHODS , NOT MATCHED TRAFFIC TARGETS . Model Mixtral Qwen1.5 DeepSeek OLMoE Qwen3

q=1

q=2

q=3

q=4

q=5

−5.17 0.00 2.44 −20.59 1.06

6.90 1.06 7.32 −17.65 0.00

6.90 1.06 6.10 −29.41 1.06

1.72 1.06 7.32 −23.53 0.00

3.45 2.13 4.88 −29.41 1.06

reduction over the five tested budgets (Fig. 1): Spearman ρ = 0.90. An exact two-sided permutation test, enumerating all 5! = 120 assignments of the five peak effects to models, gives p = 10/120 = 0.083 by counting correlations with |ρ| ≥ 0.90. Thus, models with larger concentration gaps tend to exhibit larger peak effects. This post hoc association is suggestive, not evidence that concentration causes susceptibility. It involves only five architectures, uses maxima across tested budgets, and compares relative reductions normalized by differing baseline refusal counts. All three experiments produce no degraded outputs, and the benign gradient-suppression audit yields no restricted or degraded responses. These checks do not establish preserved accuracy, instruction following, or general capability, but they do indicate that the refusal reductions reflect substantive compliance rather than unusable generations, reinforcing the depth-matched result. V. C ONCLUSION This study shows that activation frequency is not a consistently reliable proxy for suppression-induced behavioral effects in sparse MoE language models. Across five architectures, malicious inputs exhibit greater model-wide routing concentration despite substantial expert overlap with benign inputs, and router-gradient sensitivity consistently identifies suppression targets that reduce refusals more effectively than activation-based or random selection, with the largest effect a 73.53% relative reduction in OLMoE. With exact per-layer counts matched, gradient selection exceeds activation in 23 of 25 conditions and ties in two, supporting an advantage beyond layer placement alone. The exploratory association between concentration gaps and peak gradient effects connects routing structure to behavioral susceptibility, but requires validation across more models. We intend these findings to support the identification and hardening of safety-sensitive experts in MoE models, not to enable misuse. The mechanism behind the non-monotonic effects observed at larger suppression budgets, along with general model capability preservation, remain topics for future study.

R EFERENCES [1] N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. V. Le, G. Hinton, and J. Dean, “Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,” in International Conference on Learning Representations (ICLR), 2017. [2] D. Lepikhin, H. Lee, Y. Xu, D. Chen, O. Firat, Y. Huang, M. Krikun, N. Shazeer, and Z. Chen, “GShard: Scaling giant models with conditional computation and automatic sharding,” in International Conference on Learning Representations (ICLR), 2021. [3] W. Fedus, J. Dean, and B. Zoph, “A review of sparse expert models in deep learning,” arXiv:2209.01667, 2022. [4] Y. Jiang et al., “Sparse models, sparse safety: Unsafe routes in mixtureof-experts LLMs,” in Proc. Int. Conf. Mach. Learn. (ICML), 2026. [5] Z. Xu et al., “RouteHijack: Routing-aware attack on mixture-of-experts LLMs,” arXiv:2605.02946, 2026. [6] Z. Lai et al., “SAFEx: Analyzing vulnerabilities of MoE-based LLMs via stable safety-critical expert identification,” in Advances in Neural Information Processing Systems, vol. 38, 2025. [7] M. Fayyaz et al., “Steering MoE LLMs via expert (de)activation,” in Proc. Int. Conf. Learning Representations (ICLR), 2026. [8] Z. Zhang et al., “RASET: Router-agnostic safety-critical expert tuning exposes localized safety enforcement failures in mixture-of-experts LLMs,” in Proc. Conf. Empirical Methods Natural Language Processing (EMNLP), 2026, forthcoming. [9] A. Q. Jiang, A. Sablayrolles, A. Roux et al., “Mixtral of experts,” arXiv:2401.04088, 2024. [10] Qwen Team, “Qwen1.5-MoE: Matching 7B model performance with 1/3 activated parameters,” Qwen Blog, 2024. [11] D. Dai, C. Deng, C. Zhao et al., “DeepSeekMoE: Towards ultimate expert specialization in mixture-of-experts language models,” in Proc. 62nd Annu. Meeting Assoc. Comput. Linguistics (ACL), vol. 1, 2024, pp. 1280–1297, doi: 10.18653/v1/2024.acl-long.70. [12] N. Muennighoff, L. Soldaini, D. Groeneveld et al., “OLMoE: Open mixture-of-experts language models,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2025. [13] A. Yang et al., “Qwen3 technical report,” arXiv:2505.09388, 2025. [14] F. Xue et al., “OpenMoE: An early effort on open mixture-of-experts language models,” in Proc. 41st Int. Conf. Machine Learning (ICML), 2024, pp. 55625–55655. [15] X. Yang et al., “Mixture of experts made intrinsically interpretable,” in Proc. 42nd Int. Conf. Machine Learning (ICML), 2025, pp. 71231– 71248. [16] L. Bandarkar et al., “Multilingual routing in mixture-of-experts,” in Proc. Int. Conf. Learning Representations (ICLR), 2026. [17] Z. Su et al., “Unveiling super experts in mixture-of-experts large language models,” in Proc. Int. Conf. Learning Representations (ICLR), 2026. [18] A. Zou et al., “Universal and transferable adversarial attacks on aligned language models,” arXiv:2307.15043, 2023. [19] P. Chao, E. Debenedetti, A. Robey et al., “JailbreakBench: An open robustness benchmark for jailbreaking large language models,” in Advances in Neural Information Processing Systems, vol. 37, 2024. [20] A. Souly, Q. Lu, D. Bowen et al., “A StrongREJECT for empty jailbreaks,” in Advances in Neural Information Processing Systems, vol. 37, 2024. [21] A. Arditi et al., “Refusal in language models is mediated by a single direction,” in Advances in Neural Information Processing Systems, vol. 37, 2024. [22] Z. Fei et al., “MisRouter: Exploiting routing mechanisms for input-only attacks on mixture-of-experts LLMs,” arXiv:2605.04446, 2026. [23] L. Wu et al., “GateBreaker: Gate-guided attacks on mixture-of-expert LLMs,” in Proc. 35th USENIX Security Symp., 2026. [24] X. Zhao et al., “Who speaks for the trigger? Dynamic expert routing in backdoored mixture-of-experts transformers,” in Advances in Neural Information Processing Systems, vol. 38, 2025. [25] S. Ghosh et al., “AEGIS2.0: A diverse safety dataset and risk taxonomy for LLM guardrails,” in Proc. NAACL-HLT, 2025, pp. 5992–6026.

Related documents

• Frequency Is Not Sensitivity Identifying Safety-Sensitive Experts in Sparse MoE LLM#1187588
• Strong yet Controllable LLM Abstention without Learning to Abstain#1327908
• Aggregating sets of judgments: An impossibility result#748730
• Aggregating Sets of Judgments: An Impossibility Result#465574
• Mixture-of-Parallelisms: Towards Memory-Efficient Training Stack for Mixture-of-Experts Models#332492