DT-Guard: Intent-Driven Reasoning-Active Training for Reasoning-Free LLM Safety Guardrail

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

DT-Guard: Intent-Driven Reasoning-Active Training for Reasoning-Free LLM Safety Guardrail

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

artificialintelligence knowledgerepresentation reasoning

artificial intelligence, reasoning, knowledge representation

DIGITAL TECHNOLOGIES

DT-Guard: Intent-Driven Reasoning-Active Training for Reasoning-Free LLM Safety Guardrail He Liu∗ , Changtao Miao∗ , Xinjie Yang∗ , Tianle Song, Yin Wu, Junchi Chen, Bintao He, Xinyuan Zhang, Bo Zhang† , Shi Yan, Wei Lu, Wei Wang, Danyang Xu, Jiansheng Cai, Zhe Li Ant Digital Technologies, Ant Group

arXiv:2607.06326v1 [cs.AI] 7 Jul 2026

Abstract Large language models deployed in open-world applications require safety guardrails that are both robust to complex risks and efficient enough for low-latency runtime moderation. Existing guardrails face a practical trade-off between lightweight classificationbased models, which are efficient but often struggle with concealed intent, ambiguous semantics, and borderline safety decisions, and reasoning-based guards, which improve judgment quality but introduce additional token generation and inference latency. We present DT-Guard, a content safety guardrail model based on a ReasoningActive Training, Reasoning-Free Inference paradigm. The key idea is to use reasoning supervision during training while emitting only structured safety labels at inference time. DT-Guard formulates safety judgment as a progressive decision process, Intent → Category → Safety, and constructs an intent-driven dataset with intent labels, risk categories, safety labels, and structured reasoning trajectories. To further improve hard-case robustness, we propose Rollout-Guided Progressive Hard-Case Optimization (RG-PHO), which uses multi-rollout consistency to identify stably mastered, persistently failed, and preference-unstable samples, and applies targeted supervised and preference optimization accordingly. At inference time, DT-Guard directly generates structured labels without explicit reasoning traces, preserving deployment efficiency. Experiments on prompt-side and response-side safety benchmarks show that DT-Guard achieves average F1 scores of 0.886 and 0.870, respectively. With only a 4B backbone, it reaches a dual-side average F1 of 0.878, outperforming strong 8B guardrail baselines. These results demonstrate that reasoning supervision can be effectively internalized into low-latency safety discrimination.

1. Introduction Large language models (LLMs) have achieved substantial progress in instruction following, knowledge-intensive question answering, complex reasoning, and multi-turn interaction, and are increasingly deployed in open-ended real-world applications [1, 2, 3]. As their deployment scope expands, LLMs must handle diverse user inputs and generate responses across safetysensitive scenarios. User inputs may contain concealed harmful intent, adversarial jailbreak * Equal contribution. † Corresponding to [email protected]

Sorry-Bench

0. n-

.8 -0

-R

-X

0.6583

-4

-G

-R rd

w Q

-X

3G en w

-8

B .8 -0

-G

-G B

-8 rd en

ua Q

en w

-G

-4

-R

ua G

-X

0.7423 0.7112

0.797 0.776

0.8456

0.8511 0.8493 0.8339

rd ua ng

-X

ea -R

-0

-G

-G ua

-4

-G

en w

-8

-R

G -X

0.8983 0.8865

0.6845 0.6844 0.6618

el ea -R rd ua Fe

-X

0.7467 0.7278

od m ur O

-R

-X G

en -0 .8 B -G -X en ea G so ua nrd 0. -R 6B ea so n8B

-G

-4 B

en w

-8 B

en w Q

0.7898

S-Eval

0.9427

0.87040.8692

OpenAIMod

Aegis 0.9152 0.9118 0.9063 0.9063

Figure 1: DT-Guard achieves the top F1 on representative safety benchmarks under reasoningfree inference, outperforming strong guardrail baselines. attempts, or ambiguous borderline requests, while model outputs may include unsafe, harmful, biased, or policy-violating content under specific contexts [4, 5]. Safety guardrail models [6, 7, 8, 9] have therefore become an important runtime safety layer for detecting and intercepting risks before user inputs are passed to the model or before model responses are returned to users. Existing guardrail models typically follow one of two inference paradigms. Classificationbased guardrails [6, 7, 10, 11, 12] directly predict safety labels according to predefined taxonomies and are efficient for real-time deployment. However, they often struggle with concealed intent, ambiguous semantics, and adversarially framed requests, because complex safety judgments are compressed into flat label prediction. Safety risk is not always determined by surface text alone. Requests involving the same sensitive topic may correspond to benign educational inquiry, defensive analysis, risk exploration, or malicious exploitation [13, 14]. Without explicit intent modeling, guardrails may suffer from over-refusal on benign requests or risk under-detection on malicious ones. Reasoning-enhanced guardrails provide another direction. By generating explicit chain-ofthought reasoning or explanatory traces, such methods can analyze context more carefully and improve robustness on difficult safety cases [15, 9]. Nevertheless, requiring complete reasoning traces or detailed explanations at inference time introduces additional token-generation overhead and response latency, which makes deployment challenging in high-throughput and low-latency industrial systems [8]. These limitations highlight the need for a guardrail training paradigm that can benefit from reasoning supervision while preserving reasoning-free inference. To address this challenge, we propose DT-Guard, a content safety guardrail model following the Reasoning-Active Training, Reasoning-Free Inference paradigm. Instead of treating explicit reasoning as a mandatory deployment-time output, DT-Guard uses reasoning trajectories as supervision signals during training and directly generates structured safety labels during inference. We formulate safety judgment as a progressive decision process, Intent → Category → Safety. The model first identifies the underlying interaction intent, then attributes the relevant risk categories, and finally predicts the safety level. Compared with directly mapping text to a safety label, this structure provides clearer intermediate supervision for difficult cases where surface-level cues alone are unreliable. Based on this formulation, we construct an intent-driven safety dataset containing prompts, responses, intent labels, risk categories, safety labels, and structured reasoning trajectories. We 2

further introduce Rollout-Guided Progressive Hard-Case Optimization (RG-PHO), which uses multi-rollout consistency to identify different types of residual errors. Persistently failed samples are optimized with stronger supervised correction, while preference-unstable samples are optimized through contrastive preference learning over correct and incorrect rollouts. In this way, explicit reasoning signals are transferred into more stable label-level safety judgment without requiring reasoning-chain generation at inference time. Experiments on multiple prompt-side and response-side safety benchmarks demonstrate the effectiveness of DT-Guard under Reasoning-Free inference. On 10 prompt-side benchmarks, DT-Guard achieves an average 𝐹1 score of 0.886, improving over Qwen3Guard-8B (0.852) by 3.4 points. On 7 response-side benchmarks, it achieves an average 𝐹1 score of 0.870. Overall, with only a 4B backbone, DT-Guard reaches a dual-side average 𝐹1 score of 0.878, outperforming strong 8B guardrail baselines. These results show that reasoning supervision can be internalized during training and converted into efficient safety discrimination at inference time. Our main contributions are summarized as follows: • We propose DT-Guard, a safety guardrail model following the Reasoning-Active Training, Reasoning-Free Inference paradigm, which internalizes complex safety reasoning without emitting explicit reasoning chains during deployment. • We construct an intent-driven safety judgment framework and dataset, and introduce RG-PHO, a progressive hard-case optimization strategy. It organizes safety discrimination as Intent → Category → Safety, stratifies hard cases via rollout consistency, and applies Hard-Case SFT and Hard-Case DPO for targeted optimization. • We conduct multiple experiments on input risk detection and response safety moderation, providing an effective training path that balances reasoning capability and deployment efficiency for low-latency safety guardrails.

2. Related Work 2.1. Safety Alignment The rapid development and deployment of large language models (LLMs) have made safety alignment a central research topic. Existing methods aim to align model behavior with human preferences and safety requirements during training. RLHF [16] optimizes LLMs with reward models trained from human preference data, while Constitutional AI [17] improves harmlessness through self-critique and revision based on predefined principles. DPO [18] further simplifies preference optimization by directly learning from preference pairs. Although these methods improve intrinsic model safety, alignment alone remains insufficient for reliable open-world deployment. Automated jailbreak methods such as GCG [4] and AutoDAN [5] show that aligned models can still be induced to generate unsafe or policy-violating responses. These limitations motivate runtime safety mechanisms that monitor and regulate LLM interactions beyond model-level alignment. 2.2. Guardrail Models Safety guardrails serve as runtime safety layers for moderating user inputs and model outputs. Existing guardrail models usually formulate safety detection as an instruction-following or classification task based on predefined taxonomies. Llama Guard [6] trains a dedicated LLM to classify user inputs and model responses according to safety policies. ShieldGemma [7] improves 3

Data Collection Open-source training dataset

Data Processing and Training Set Generation

Distillation

Data Format 3

Data Voting and Expert Judge

Data Format 2 Label == Safety

BBQ Ambig

•

CrowS-Pairs

Prompt

Expert Judge •

SG-Bench

•

AdvBench ……

•

Response

vote1

Label

Model is right

3:0

CoT Model is wrong

Data Format 1 •

Prompt

•

Response

•

Label

Binary classification

•

Intent

•

Category

•

Safety (Safe/Borderline/Unsafe)

Label != Safety

disagreement

( safe/unsafe)

vote3

811.9k 1:2

Safety -> Borderline

(safe/unsafe)

Figure 2: Intent-driven data construction pipeline: heterogeneous safety sources are distilled, filtered by voting, and expert-verified into structured CoT and label supervision. configurability through customizable policy instructions. Aegis [10] introduces fine-grained risk categories, while WildGuard [11] unifies safety risk detection, jailbreak detection, and refusal detection. PolyGuard [12] further extends guardrails to multilingual safety moderation. Despite their effectiveness, these methods mainly rely on direct label prediction, providing limited modeling of latent intent and complex decision processes in ambiguous or deceptive cases. 2.3. Guardrail Reasoning Recent studies have explored reasoning-enhanced guardrails to improve robustness and interpretability. GuardReasoner [15] explicitly models the reasoning process behind safety judgments, while YuFeng-XGuard [9] emphasizes reasoning-centric and interpretable risk perception. However, generating reasoning traces at inference time introduces additional token cost and response latency, making such methods less practical for latency-sensitive applications. To improve deployment efficiency, Qwen3Guard [8] enables token-level real-time monitoring through streaming detection. Nevertheless, existing guardrails still face a tension between reasoning-enhanced judgment and efficient inference. Our DT-Guard addresses this gap by using reasoning and intent supervision during training while preserving reasoning-free structured-label inference during deployment.

3. Intent-Driven Safety Data Construction We build an intent-driven safety corpus for training DT-Guard, as shown in Figure 2. Starting from heterogeneous safety resources, we normalize all samples into a unified prompt-responselabel format and enrich each instance with chain-of-thought (CoT), intent, risk-category, and safety annotations. This converts safety moderation from flat label prediction into a structured decision path, Intent → Category → Safety. To reduce annotation noise and expose boundary cases, we combine GLM-5.1 distillation, multi-round voting, and expert verification: agreement cases are retained, confirmed relabels are kept, erroneous cases are filtered, and inconsistent cases are assigned to Borderline. The final 811,897-sample corpus provides both structured-label supervision for reasoning-free inference and reasoning trajectories for subsequent RG-PHO training. 4

Table 1: Final corpus scale by task side after filtering and balancing. Data Type

Count

Percentage

Prompt-level samples Response-level samples

450,437 361,460

55.48% 44.52%

Total samples

811,897

100.00%

Table 2: Intent taxonomy and final distribution, separating normal use, risky content, and adversarial attack intent. Intent

Definition

Count

Percentage

Normal Risky Attack

Benign user intent or safe response. Unsafe content without attack techniques. Unsafe prompt with jailbreak or adversarial attacks.

471,612 329,313 10,972

58.08% 40.56% 1.35%

Total

–

811,897

100.00%

3.1. Data Sources We aggregate 1,918,565 raw samples from six safety domains, covering red–blue teaming, jailbreak attacks, alignment data, toxicity, bias, and domain-specific risks. The raw pool contains 949,050 prompt-level samples (49.46%) for input-side risk detection and 969,515 response-level samples (50.54%) for output-side safety assessment. After filtering and balancing, the final corpus remains dual-sided, with 450,437 prompt-level samples and 361,460 response-level samples (Table 1), enabling comparable supervision for both guardrail entry points. 3.2. Data Processing and Training Set Construction We use a two-stage pipeline to expand annotation dimensions and control label quality. LLM-based Distillation and Quality Filtering. Public safety datasets differ in label spaces, risk granularity, and annotation criteria. We therefore use GLM-5.1 as an automatic annotator and require a fixed output schema containing CoT, Intent, Category, and Safety. The intent label follows Table 2; the category label is selected from the nine risk types in Figure 3; and the safety label is one of Safe, Unsafe, or Borderline (Table 3). The annotation order explicitly follows Intent → Category → Safety, encouraging consistency between interaction motivation, risk attribution, and final adjudication. Multi-round Voting and Expert Verification. We next compare the distilled Safety label with the original label. Matched samples are retained directly. For mismatched samples, GLM-5.1 performs three additional independent rollouts. Unanimous relabeling results are sent to expert verification; confirmed corrections are kept, and rejected cases are removed. Non-unanimous results (2:1 or 1:2) are treated as annotation-unstable boundary cases and relabeled as Borderline. This procedure corrects both false positives that cause over-refusal and false negatives that cause risk under-detection. It retains 1,121,574 high-quality samples from the raw corpus, with a 58.46% retention rate. Training Set Construction. Each retained sample contains dialogue content, CoT reasoning, intent labels, risk categories, and safety labels, providing aligned supervision for intent recognition, risk localization, and safety classification.

Figure 3: Distribution of unsafe risk categories. Table 3: Final safety-label distribution, retaining safe, unsafe, and borderline cases for boundaryrisk learning. Safety Level

Definition

Count

Percentage

Safe Unsafe Borderline

Benign content Contains unsafe content Potentially unsafe or ambiguous

450,000 317,134 44,763

55.43% 39.06% 5.51%

Total

–

811,897

100.00%

3.3. Data Balancing Starting from the 1,121,574 retained samples, we balance the final training distribution along safety labels and risk categories. Safety Label Balancing. We target a Safe:Unsafe:Borderline ratio of approximately 5.5:4:0.5. This preserves sufficient unsafe supervision, keeps enough safe samples to reduce over-refusal, and explicitly maintains boundary cases for ambiguity modeling. The final corpus contains 450,000 Safe samples (55.43%), 317,134 Unsafe samples (39.06%), and 44,763 Borderline samples (5.51%), as shown in Table 3. Risk Category Balancing. We also perform category-aware sampling over the nine risk types to limit head-category dominance and preserve long-tail coverage. The final distribution covers frequent risks such as Prejudice and Discrimination and Illegal Activities, while retaining sparse but deployment-critical categories such as Politically Sensitive Content and Others (Figure 3). After applying both balancing strategies, the final dataset contains 811,897 samples, corresponding to an overall retention rate of 42.32% relative to the original corpus.

4. Rollout-Guided Progressive Hard-Case Optimization DT-Guard aims to internalize reasoning supervision while keeping inference as structured-label generation. As shown in Figure 4, directly fine-tuning on CoT-heavy targets can create a train– test format mismatch: the model learns to depend on explicit reasoning chains, but deployment 6

Stage1 General SFT baseline • •

Prompt / Response CoT Intent / Category / Safety

Prompt / Response CoT / noCoT = 1 : 1 Intent / Category / Safety

Hard-to-CoT Mixing • • •

Prompt / Response CoT( includes all borderline) / noCoT = 1 : 1 Intent / Category / Safety

✖ 2

0/3

811.9k

Basemodel : Qwen3-4B

Stage3 Rollout-Contrastive Hard-Case DPO

Reasoning Active

1/3

Stage1-SFT-v3

Stage3-DPO Training Set

25.1k

Different Wrong 25.1k

6.2k

Reasoning-Active Training

811.9k

CoT+ noCoT

Stage1-SFT-v2

Reasoning-Active Training

811.9k

CoT

＋

CoT

21.8k

Reasoning-Active Training

855.5k

811.9k

21.8k

Reasoning Active

Stage1-SFT-v1

Stage2-SFT Stage2 Training Set

Stage1 Training Set

Hard Cases OverSampling

Prompt / Response Intent / Category / Safety

Random Balanced Mixing • • •

Stage2-SFT Training Set

Stage1 Best Model 811.9k

+CoT • • •

Stage1-SFT-v0

Prompt / Response Category / Safety

+intent • •

811.9k

Stage2 Failure-Driven Hard-Case SFT

Reasoning Active Rollout

Stage1-SFT-v4

Stage1 Training Set

2/3 Reasoning Active 37.4k

Reasoning-Active Training

18.9k 37.4k

Stage3-DPO

(

) 6.2k

(

) 6.2k

(

)

(

) 37.4k

18.9k

Same Wrong

Stage3 Training Set 68.7k

Reasoning-Active Training

CoT

CoT Basemodel : Stage2-SFT-best

( Chosen, Rejected )

811.9k

Basemodel : Qwen3-4B

Figure 4: RG-PHO training pipeline: mixed-mode SFT is followed by rollout-based hard-case stratification, hard-case SFT, and rollout-contrastive DPO. requires Reasoning-Free outputs. We address this with Rollout-Guided Progressive Hard-Case Optimization (RG-PHO), which uses rollout consistency to estimate sample difficulty and assigns each difficulty pattern to a matching optimization objective. RG-PHO has three stages. Intent-Guided Mixed-Mode SFT first learns the Intent → Category → Safety decision structure from both structured-label and CoT outputs. The trained model is then rolled out multiple times on training samples to separate stably mastered, persistently failed, and preference-unstable cases. Failure-Driven Hard-Case SFT repairs persistently failed samples with stronger supervision, while Rollout-Contrastive Hard-Case DPO builds chosen– rejected pairs from correct and incorrect rollouts of preference-unstable samples. This pipeline converts explicit reasoning signals into stable label-level safety judgment. 4.1. Intent-Guided Mixed-Mode SFT The first stage establishes intent-aware safety discrimination. Using the structured taxonomy in Section 3, the model maps each input to Intent, Category, and Safety. Intent captures interaction motivation, Category localize risk types, and Safety gives the final decision. Unlike standard SFT, this stage mixes two output formats. Borderline and fine-grained cases retain CoT reasoning trajectories to expose intermediate decision logic, while clear safe or unsafe cases more often use compact structured-label outputs. The mixed format injects reasoning supervision without forcing all training samples into an inference-mismatched CoT style. The training objective is standard autoregressive supervised learning. Given input 𝑥 and target output 𝑦 , the model minimizes the negative log-likelihood: LSFT = −

|𝑦| ∑︁

log 𝑝𝜃 ( 𝑦𝑡 | 𝑥 , 𝑦<𝑡 ).

𝑡 =1

This stage yields the initial safety model used for rollout-based hard-case identification.

(1)

4.2. Rollout-Consistency-Based Sample Stratification After Stage 1, we estimate sample difficulty by prediction stability. For each sample 𝑥 𝑖 , the model generates 𝐾 independent outputs and produces safety-label predictions. The rollout consistency score is: 𝐾 h i ∑︁ 𝑠𝑖 = 1 𝑦ˆ𝑖( 𝑘 ) = 𝑦𝑖 . (2) 𝑘=1

We use three rollouts in all experiments. Samples with 3/3 correct predictions are stably mastered; samples with 0/3 correct predictions are persistently failed; and samples with 1/3 or 2/3 correct predictions are preference-unstable. These groups correspond to replay suppression, capability repair, and preference calibration, respectively. This behavior-driven stratification prevents all residual errors from being optimized identically, and determines whether a sample should be skipped, supervised, or converted into preference pairs. 4.3. Failure-Driven Hard-Case SFT The second stage applies Failure-Driven Hard-Case SFT to persistently failed samples. Since the model produces no correct rollout for these cases, DPO lacks a naturally generated chosen response and is poorly matched to the error type. We therefore use CoT-supervised correction to teach the missing path from text understanding to intent recognition, risk attribution, and safety adjudication. Unlike Stage 1, this stage is not designed to expand coverage; it targets systematic failures that cause risk under-detection or over-refusal. After Failure-Driven Hard-Case SFT, the model receives new supervision on previously failed samples, providing a more stable initialization for subsequent preference optimization. 4.4. Rollout-Contrastive Hard-Case DPO The third stage applies Rollout-Contrastive Hard-Case DPO to preference-unstable samples. These samples already contain both correct and incorrect model-generated outputs, making them natural candidates for preference learning. For each sample, rollouts consistent with the ground-truth label are chosen responses, and inconsistent rollouts are rejected responses. When multiple distinct incorrect outputs exist, we construct one-to-many preference pairs. Since the outputs retain CoT trajectories, DPO optimizes both the reasoning path and the final safety decision. The DPO objective is: 𝑝𝜃 ( 𝑦 + | 𝑥 ) 𝑝𝜃 ( 𝑦 − | 𝑥 ) LDPO = −E ( 𝑥 , 𝑦 + , 𝑦 − ) log 𝜎 𝛽 log − log 𝑝ref ( 𝑦 + | 𝑥 ) 𝑝ref ( 𝑦 − | 𝑥 )

(3)

where 𝑦 + denotes the chosen output, 𝑦 − denotes the rejected output, 𝑝ref is the reference model, and 𝛽 controls preference strength. The objective increases the relative probability of correct reasoning paths and safety labels over incorrect adjudications. Compared with further SFT, DPO is better suited to samples where the model can already generate a correct answer but selects it inconsistently. The contrastive rollout pairs calibrate this selection behavior. 8

Table 4: Training variants for isolating intent labels, CoT allocation, hard-case SFT, and rolloutcontrastive DPO. Variant

Model Configuration

Purpose

Stage1-SFT-v0

Qwen3-4B-base; without Intent labels or CoT training. Qwen3-4B-base; with Intent labels, without CoT training. Qwen3-4B-base; with Intent labels and CoT training; CoT disabled during inference. Qwen3-4B-base; with Intent labels; mixed noCoT/CoT training data (1:1); CoT disabled during inference. Qwen3-4B-base; with Intent labels; all Borderline samples trained with CoT, remaining samples mixed at a 1:1 noCoT/CoT ratio; CoT disabled during inference. Stage1-SFT-v4 + SFT with upsampled samples misclassified in all three rollouts. Stage2-SFT + DPO using synthesized preference pairs from partially misclassified rollout samples.

Baseline.

Stage1-SFT-v1 Stage1-SFT-v2 Stage1-SFT-v3

Stage1-SFT-v4

Stage2-SFT Stage3-DPO

Evaluate the effect of Intent supervision. Evaluate full CoT training. Evaluate mixed CoT/noCoT training. Evaluate selective CoT for Borderline samples.

Evaluate hard-sample learning. Evaluate DPO-based preference optimization.

4.5. Reasoning-Free Inference After training, DT-Guard defaults to Reasoning-Free inference. It does not output CoT traces, and directly generates Intent, Category, and Safety. This matches the structured-label format used in mixed-mode training. Reasoning-Free inference removes explicit reasoning generation, not the reasoning supervision learned during training. RG-PHO converts CoT samples, hard-case SFT, and rolloutcontrastive DPO into label-level discrimination capability without adding reasoning-chain generation overhead.

5. Experiments and Analysis This section evaluates DT-Guard on prompt-side risk detection and response-side safety auditing. We first define the evaluation protocol, then compare against strong guardrail baselines, and finally analyze the contribution of each training component under Reasoning-Free inference. 5.1. Experimental Setup We use Qwen3-4B as the backbone and compare with the Qwen3Guard and YuFeng-XGuard model families. Prompt-side evaluation covers 10 benchmarks, including ToxicChat, OpenAIModeration, AegisSafety, AegisSafety2.0, SimpleSafetyTests, HarmBench-Prompt, WildGuardPrompt, SafetyEval, Sorry-Bench, and XSTest. Response-side evaluation covers 7 benchmarks, including HarmBench-Response, SafeRLHF, BeaverTails, XSTest-Response, AegisSafety2.0Response, WildGuard-Response, and Think. All experiments report classification 𝐹1 . We summarize prompt-side average, response-side average, and dual-side average 𝐹1 . For baselines with multiple decision settings, we report the best setting on each benchmark. 9

Table 5: Prompt-side 𝐹1 comparison on 10 safety benchmarks. The best result in each column is highlighted in bold. Model

Params

Toxic

OpenAI

Aegis

Aegis2

SimpST

HarmB

WildG

S-Eval

Sorry

XSTest

Avg.

0.990 0.995 0.995 0.995 1.000

0.988 1.000 1.000 0.824 0.821

0.878 0.887 0.891 0.879 0.877

0.834 0.851 0.849 0.887 0.898

0.711 0.742 0.797 0.658 0.776

0.853 0.899 0.908 0.918 0.954

0.825 0.844 0.852 0.833 0.849

1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.947 0.995 0.805 0.866 0.998 1.000 1.000

0.870 0.870 0.829 0.864 0.888 0.884 0.888

0.907 0.909 0.889 0.902 0.842 0.901 0.943

0.816 0.798 0.757 0.772 0.702 0.818 0.846

0.871 0.874 0.847 0.860 0.936 0.867 0.929

0.855 0.863 0.831 0.840 0.859 0.865 0.886

Existing Guard Models Qwen3Guard-0.8B-Gen Qwen3Guard-4B-Gen Qwen3Guard-8B-Gen YuFeng-XGuard-Reason-0.6B YuFeng-XGuard-Reason-8B

0.8B 4B 8B 0.6B 8B

0.578 0.616 0.617 0.713 0.696

0.662 0.684 0.685 0.728 0.747

0.906 0.906 0.912 0.870 0.869

Stage1-SFT-v0 Stage1-SFT-v1 Stage1-SFT-v2 Stage1-SFT-v3 Stage1-SFT-v4 Stage2-SFT Stage3-DPO

4B 4B 4B 4B 4B 4B 4B

0.658 0.665 0.656 0.638 0.723 0.660 0.697

0.740 0.768 0.813 0.758 0.803 0.798 0.790

0.913 0.910 0.875 0.902 0.850 0.876 0.915

0.851 0.858 0.861 0.862 0.854

DT-Guard (Ours) 0.830 0.844 0.836 0.837 0.851 0.843 0.847

Table 6: Response-side 𝐹1 comparison on 7 safety benchmarks. The best result in each column is highlighted in bold. Model

Params

HarmB

Qwen3Guard-0.8B-Gen Qwen3Guard-4B-Gen Qwen3Guard-8B-Gen YuFeng-XGuard-Reason-0.6B YuFeng-XGuard-Reason-8B

0.8B 4B 8B 0.6B 8B

0.985 0.964 0.947 0.802 0.817

Stage1-SFT-v0 Stage1-SFT-v1 Stage1-SFT-v2 Stage1-SFT-v3 Stage1-SFT-v4 Stage2-SFT Stage3-DPO

4B 4B 4B 4B 4B 4B 4B

0.931 0.936 0.925 0.925 0.953 0.964 0.964

SafeRLHF

BeaverTails

XSTest

Aegis2

WildG

Think

Avg.

0.892 0.881 0.921 0.872 0.885

0.844 0.849 0.862 0.819 0.832

0.761 0.758 0.782 0.780 0.779

0.859 0.845 0.842 0.872 0.871

0.858 0.853 0.863 0.835 0.841

0.897 0.874 0.847 0.838 0.898 0.909 0.917

0.825 0.832 0.831 0.815 0.834 0.833 0.847

0.750 0.769 0.746 0.749 0.784 0.783 0.798

0.853 0.854 0.818 0.865 0.871 0.871 0.884

0.847 0.848 0.824 0.839 0.860 0.863 0.870

Existing Guard Models 0.806 0.812 0.820 0.848 0.857

0.862 0.860 0.865 0.849 0.849

DT-Guard (Ours) 0.819 0.818 0.771 0.830 0.826 0.825 0.826

0.853 0.856 0.832 0.850 0.854 0.855 0.856

5.2. Main Results Table 5 and Table 6 show that DT-Guard improves both input-side and output-side safety classification. On the prompt side, Stage3-DPO reaches an average 𝐹1 of 0.886, outperforming Qwen3Guard-8B-Gen (0.852) and YuFeng-XGuard-Reason-8B (0.849). On the response side, it reaches 0.870, exceeding Qwen3Guard-8B-Gen(0.863) and YuFeng-XGuard-Reason-8B (0.841). Across both task types, DT-Guard obtains a dual-side average 𝐹1 of 0.878 (Table 7), surpassing Qwen3Guard-8B-Gen (0.858) and YuFeng-XGuard-Reason-8B (0.845) with only a 4B backbone. The gain therefore comes primarily from intent-driven supervision and RG-PHO rather than model scale. 5.3. Ablation Study and Training Analysis We ablate intent labels, CoT allocation, hard-case SFT, and DPO to isolate the contribution of each RG-PHO component. The results show that gains come from matching supervision type to sample difficulty, rather than from adding CoT uniformly.

Table 7: Overall average 𝐹1 comparison across prompt-side and response-side benchmarks. Model

Params

Prompt

Response

Avg.

0.825 0.844 0.852 0.833 0.849

0.858 0.853 0.863 0.835 0.841

0.842 0.848 0.858 0.834 0.845

0.855 0.863 0.831 0.840 0.859 0.865 0.886

0.847 0.848 0.824 0.839 0.860 0.863 0.870

0.851 0.856 0.828 0.840 0.860 0.864 0.878

Existing Guard Models Qwen3Guard-0.8B-Gen Qwen3Guard-4B-Gen Qwen3Guard-8B-Gen YuFeng-XGuard-Reason-0.6B YuFeng-XGuard-Reason-8B

0.8B 4B 8B 0.6B 8B

DT-Guard (Ours) Stage1-SFT-v0 Stage1-SFT-v1 Stage1-SFT-v2 Stage1-SFT-v3 Stage1-SFT-v4 Stage2-SFT Stage3-DPO

4B 4B 4B 4B 4B 4B 4B

Table 8: CoT allocation ablation under Reasoning-Free inference. Selective CoT on Borderline samples gives the best average 𝐹1 . Training Strategy

Prompt

Response

Avg.

NoCoT only Full CoT Mixed CoT/NoCoT (1:1) Borderline CoT + Mixed (1:1)

0.863 0.831 0.840 0.859

0.848 0.824 0.839 0.860

0.856 0.828 0.840 0.860

5.3.1. Effect of Intent Labels Intent supervision models interaction motivation before the final safety decision. Compared with Stage1-SFT-v0, adding Intent improves prompt-side average 𝐹1 from 0.855 to 0.863, with a smaller but positive response-side gain. This gain is largest on prompt-side tasks, where semantically similar requests may reflect normal use, risky exploration, or attack intent. Intent labels provide an intermediate constraint before category attribution and safety prediction. 5.3.2. Effect of CoT Training Strategy Table 8 shows that CoT supervision is useful only when allocated selectively. Full CoT training reduces dual-side average 𝐹1 from 0.856 to 0.828, indicating a format mismatch between CoTheavy training and Reasoning-Free inference. Random CoT/noCoT mixing partially alleviates the mismatch but remains below the noCoT baseline. In contrast, assigning CoT to Borderline samples and mixing the remaining data improves the average to 0.860, supporting our selective reasoning-supervision design. 5.3.3. Rollout-Consistency-Based Sample Stratification Rollout consistency separates residual errors by model behavior. Stably mastered samples offer limited marginal value for replay; persistently failed samples lack a correct generated path and 11

Table 9: Progressive average 𝐹1 gains of DT-Guard across the three-stage training pipeline. Gain is measured over Stage1-SFT-v0. Stage

Variant

Prompt

Response

Avg.

Gain

Baseline

0.855

0.847

0.851

–

Stage 1

+Intent Labels +Full CoT +Mixed CoT/NoCoT (1:1) +Borderline CoT

0.863 0.831 0.840 0.859

0.848 0.824 0.839 0.860

0.856 0.828 0.840 0.860

+0.5 −2.3 −1.1 +0.9

Stage 2 Stage 3

+Hard-example SFT +Preference-aware DPO

0.865 0.886

0.863 0.870

0.864 0.878

+1.3 +2.7

require supervised repair; preference-unstable samples already contain correct and incorrect rollouts and are suitable for DPO. This stratification is the routing mechanism that decides whether a sample contributes to Stage2-SFT or Stage3-DPO. 5.3.4. Effect of Hard-Case SFT Failure-Driven Hard-Case SFT targets persistently failed samples instead of replaying the full dataset uniformly. As shown in Table 9, it improves the dual-side average 𝐹1 from 0.860 to 0.864. Although the gain is smaller than DPO, this stage creates a stronger initialization for preference optimization. For samples without any correct rollout, supervised correction is more appropriate than preference ranking because no model-generated chosen response exists. 5.3.5. Effect of Hard-Case DPO Rollout-Contrastive Hard-Case DPO provides the largest single-stage gain. It constructs chosen– rejected pairs from correct and incorrect rollouts of preference-unstable samples, encouraging the model to consistently select the correct adjudication path. As shown in Table 9, DPO improves the dual-side average 𝐹1 from 0.864 to 0.878. The prompt-side average increases from 0.865 to 0.886, indicating that preference optimization is especially effective for boundary requests, jailbreak-style inputs, and over-refusal control. 5.4. Analysis of Reasoning-Free Inference All main results are obtained under Reasoning-Free inference, where DT-Guard outputs only structured safety labels. Unlike explicit reasoning-based guardrails, our model uses reasoning primarily as training supervision through mixed SFT, hard-case SFT, and rollout-contrastive DPO. The ablations support this asymmetry. Full CoT training degrades when CoT inference is disabled, whereas selective CoT and RG-PHO improve performance. Reasoning enhancement therefore depends less on emitting reasoning chains at deployment, and more on aligning reasoning supervision with the Reasoning-Free output format.

6. Conclusion We present DT-Guard, a content safety guardrail model designed for low-latency deployment. Unlike guardrail methods that rely on explicit chain-of-thought or explanation generation during inference, DT-Guard follows a Reasoning-Active Training, Reasoning-Free Inference paradigm: explicit reasoning supervision is used during training, while inference outputs only structured safety labels. This enables the model to internalize intent recognition, risk attribution, and safety adjudication required for complex safety judgment while preserving an efficient inference format. To achieve this, we construct an intent-driven safety data framework that organizes safety discrimination as Intent → Category → Safety, and introduce Rollout-Guided Progressive Hard-Case Optimization. This training strategy uses multi-rollout consistency to identify different types of hard cases, and improves the model’s judgment on persistently failed and preference-unstable samples through Hard-Case SFT and Hard-Case DPO, respectively. Experiments show that DT-Guard achieves consistent gains on both prompt-side and response-side safety benchmarks, outperforming several 8B guardrail models with only 4B parameters. These results validate that reasoning supervision during training can be effectively converted into low-latency discrimination capability at inference time.

References [1] Qwen Team. Qwen3.5: Accelerating productivity with native multimodal agents, February 2026. URL https://qwen.ai/blog?id=qwen3.5. [2] OpenAI. Openai gpt-5 system card, 2026. URL https://arxiv.org/abs/2601.03267. [3] Ling Team. Ling and ring 2.6 technical report: Efficient and instant agentic intelligence at trillion-parameter scale, 2026. URL https://arxiv.org/abs/2606.15079. [4] Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models, 2023. URL https://arxiv.org/abs/2307.15043. [5] Sicheng Zhu, Ruiyi Zhang, Bang An, Gang Wu, Joe Barrow, Zichao Wang, Furong Huang, Ani Nenkova, and Tong Sun. Autodan: Interpretable gradient-based adversarial attacks on large language models, 2023. URL https://arxiv.org/abs/2310.15140. [6] Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa. Llama guard: Llm-based input-output safeguard for human-ai conversations, 2023. URL https: //arxiv.org/abs/2312.06674. [7] Wenjun Zeng, Yuchi Liu, Ryan Mullins, Ludovic Peran, Joe Fernandez, Hamza Harkous, Karthik Narasimhan, Drew Proud, Piyush Kumar, Bhaktipriya Radharapu, Olivia Sturman, and Oscar Wahltinez. Shieldgemma: Generative ai content moderation based on gemma, 2024. URL https://arxiv.org/abs/2407.21772. [8] Haiquan Zhao, Chenhan Yuan, Fei Huang, Xiaomeng Hu, Yichang Zhang, An Yang, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin, et al. Qwen3guard technical report, 2025. URL https://arxiv.org/abs/2510.14276. [9] Junyu Lin, Meizhen Liu, Xiufeng Huang, Jinfeng Li, Haiwen Hong, Xiaohan Yuan, Yuefeng Chen, Longtao Huang, Hui Xue, Ranjie Duan, Zhikai Chen, Yuchuan Fu, Defeng Li, Lingyao Gao, and Yitong Yang. Yufeng-xguard: A reasoning-centric, interpretable, and flexible guardrail model for large language models, 2026. URL https://arxiv.org/abs/2601 .15588. [10] Shaona Ghosh, Prasoon Varshney, Erick Galinkin, and Christopher Parisien. Aegis: Online adaptive ai content safety moderation with ensemble of llm experts, 2024. URL https: //arxiv.org/abs/2404.05993. [11] Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, Bill Yuchen Lin, Nathan Lambert, Yejin Choi, and Nouha Dziri. Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of llms. Advances in neural information processing systems, 37: 8093–8131, 2024. [12] Priyanshu Kumar, Devansh Jain, Akhila Yerukola, Liwei Jiang, Himanshu Beniwal, Thomas Hartvigsen, and Maarten Sap. Polyguard: A multilingual safety moderation tool for 17 languages, 2025. URL https://arxiv.org/abs/2504.04377. [13] Wei Jie Yeo, Ranjan Satapathy, and Erik Cambria. Mitigating jailbreaks with intent-aware llms, 2025. URL https://arxiv.org/abs/2508.12072.

[14] Jun Zhuang, Haibo Jin, Ye Zhang, Zhengjian Kang, Wenbin Zhang, Gaby G. Dagher, and Haohan Wang. Exploring the vulnerability of the content moderation guardrail in large language models via intent manipulation, 2025. URL https://arxiv.org/abs/2505.1 8556. [15] Yue Liu, Hongcheng Gao, Shengfang Zhai, Yufei He, Jun Xia, Zhengyu Hu, Yulin Chen, Xihong Yang, Jiaheng Zhang, Stan Z. Li, Hui Xiong, and Bryan Hooi. Guardreasoner: Towards reasoning-based llm safeguards, 2025. URL https://arxiv.org/abs/2501.1 8492. [16] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744, 2022. [17] Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback, 2022. URL https://arxiv.org/abs/2212.0 8073. [18] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728–53741, 2023.

Record · ID 346561 · SHA-256 349163d71f52cf40
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.