Qwen3-8B-TRACE

TRACE is a trajectory-aware defense model for multi-turn jailbreaks. Instead of judging each user message in isolation, it commits to an explicit safety assessment of the whole conversation so far — in a <STATE> block — and only then writes its reply in an <ANSWER> block. The assessment is part of the generation, so the reply is conditioned on it.

This is the Qwen3-8B member of the TRACE family; see Dipto084/Llama3.1-8B-TRACE for the Llama-3.1-8B counterpart.

• Base model: Qwen/Qwen3-8B
• Release form: full merged weights, bf16, single model.safetensors
• Paper / code: TRACE

This is the model-agnostic transfer experiment of the TRACE paper (§6.3): the full recipe — SFT followed by GRPO, every component of the pipeline held fixed — applied to a model from a different architecture family, with a distinct base safety-alignment profile and native thinking behavior. It lowers Qwen3-8B's average ASR across seven multi-turn attacks from 77.0% to 13.8%, closely matching the 14.5% obtained on Llama-3.1-8B-Instruct.

Paper

TRACE: Trajectory Aware Reasoning for Multi-Turn Adversarial Conversation Evaluation — arXiv:2608.15594 · Code: github.com/Dipto084/TRACE

Output format

<STATE>
{
  "cues": {"<cue_name>": "<turn reference and reasoning>"},
  "benign_interpretation":     {"score": 1-10, "explanation": "..."},
  "adversarial_interpretation":{"score": 1-10, "explanation": "..."},
  "jailbreak_score": 1-5,
  "action": "ALLOW | CAUTION | DECLINE",
  "justification": "..."
}
</STATE>
<ANSWER>
<the reply to the current user turn>
</ANSWER>

Only the <ANSWER> block should be shown to an end user. The <STATE> block is an internal reasoning commitment; it is useful for logging, auditing, and routing (e.g. escalate on action == "DECLINE"), but it is not user-facing text.

Usage

The model requires the TRACE system prompt — it defines the cue taxonomy, the scoring rubric, and the output contract above. Without it the model will not emit well-formed <STATE> blocks. The prompt ships with the model as system_prompt.txt; it is the file the attack evaluations load (agents/state_answer_action_prompt.txt in the code repo).

A second variant, system_prompt_or.txt, adds an explicit "harmful vs. harmful-looking" distinction and requires the first sentence of the <ANSWER> to be substantive. It is the prompt used for the PHTest over-refusal measurement; use it when over-refusal on sensitive-but-benign requests matters more than anything else.

The conversation is passed as a single user message, not as a list of chat turns. The model was trained and evaluated on the whole trajectory collapsed into one message with numbered turns; the last turn holds only the user message being answered:

[Turn 1]
USER: ...
ASSISTANT: ...
[Turn 2]
USER: ...

The model replies with <STATE>…</STATE><ANSWER>…</ANSWER>. Show the user only the ANSWER, and append only the ANSWER to the history for the next turn — the STATE block never re-enters the context.

The easiest way to get all of this right is the reference package at github.com/Dipto084/TRACE, which provides the formatting and parsing helpers plus an OpenAI-compatible proxy that lets any client (or attack framework) talk to the model with ordinary chat messages. Doing it by hand:

import re
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Dipto084/Qwen3-8B-TRACE"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
system_prompt = open("system_prompt.txt").read()

def format_trajectory(history, user_message):
    # history: list of (user, answer) pairs already exchanged; answers are ANSWER text only
    lines = []
    for i, (u, a) in enumerate(history, start=1):
        lines += [f"[Turn {i}]", f"USER: {u}", f"ASSISTANT: {a}"]
    lines += [f"[Turn {len(history) + 1}]", f"USER: {user_message}"]
    return "\n".join(lines)

def parse(raw):
    m = re.search(r"<ANSWER>(.*?)</ANSWER>", raw, re.S)
    return m.group(1).strip() if m else re.sub(r"<STATE>.*?</STATE>", "", raw, flags=re.S).strip()

history = []
for user_message in ["first user turn", "second user turn"]:
    messages = [
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": format_trajectory(history, user_message)},
    ]
    ids = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False, return_tensors="pt").to(model.device)
    out = model.generate(ids, max_new_tokens=4096, do_sample=False)
    raw = tok.decode(out[0][ids.shape[-1]:], skip_special_tokens=True)
    answer = parse(raw)
    history.append((user_message, answer))
    print(answer)

The safety assessment lives in the <STATE> block, not in Qwen's <think> block — training used non-thinking mode, so pass enable_thinking=False (the default in the shipped chat template).

Serving with vLLM (the evaluations used greedy decoding, temperature 0):

vllm serve Dipto084/Qwen3-8B-TRACE --max-model-len 65536 --dtype bfloat16

Budget generously for the completion: the <STATE> block is generated before the answer.

Training

Stage 1 — SFT. State-answer-action fine-tune of Qwen3-8B on multi-turn red-teaming conversations (actor, crescendo, ICON and other strategies) plus benign dialogues, teaching the model to emit a grounded <STATE> before every answer. Benign and topic-pivoting conversations are included deliberately, so the assessment habit does not collapse into blanket refusal.

Stage 2 — GRPO with group-decoupled advantages (GDPO). A fresh LoRA policy over the merged SFT base, trained against a co-located Qwen3-8B-AWQ judge. Each reward component is normalized independently within the group before aggregation into a per-token advantage, rather than normalizing the summed reward as standard GRPO does.

Reward components:

Component	Grounded in	Signal
R_jb	<STATE>	jailbreak-score accuracy
R_con	<ANSWER>	behavioral consistency: the generated response checked against the ground-truth action
R_cue	<STATE>	cue-set agreement

Hyperparameter	Value
Advantage estimator	GDPO, norm_adv_by_std_in_grpo=False
Reward weights (R_jb, R_con, R_cue)	0.3, 0.5, 0.2
LoRA rank / alpha	32 / 64 (q,k,v,o,gate,up,down)
Learning rate / schedule	3e-5, constant, 10 warmup steps
KL loss	low-variance KL, coefficient 5e-3
Entropy coefficient	1e-3
Train batch / PPO mini / micro per GPU	64 / 32 / 2
Rollouts per prompt, temperature	8, 0.9
Max prompt / response length	16384 / 3072
Training data	5,234 curated + 500 harm-adjacent, stratified sampler (56/8 per batch)
Judge	Qwen3-8B-AWQ (4-bit), vLLM, temperature 0
Hardware	4x H100 80GB, single node

Evaluation

From the TRACE paper, Table 4 (multi-turn) and Table 5 (single-turn).

Behavior-level attack success rate (ASR, %) across seven multi-turn attack frameworks; lower is better. FITD and AMA are held-out attacks, not represented in the training corpus.

Model	X-Teaming	Crescendo	ActorAttack	CoA	ICON	FITD	AMA	Avg
Qwen3-8B (base)	97.5	89.2	37.5	93.3	100.0	89.9	31.7	77.0
Qwen3-8B TRACE-GRPO (this model)	19.2	14.2	2.5	25.0	1.7	15.0	19.2	13.8

ASR drops on all seven attacks. The Llama member of the family reaches 14.5% average on the same suite, so the recipe's effect is near-identical on a model with entirely different internal representations.

Single-turn robustness under AutoDAN-Turbo, a strong single-turn attacker (a length-1 trajectory under the TRACE formulation). ASR@k is over k independent attempts — lower is better; Avg. Attempts to jailbreak per behavior — higher is better.

Target	ASR@3	ASR@5	ASR@10	Avg. attempts
Qwen3-8B	68.3	80.8	95.8	3.4
+ TRACE-GRPO (this model)	5.8	10.0	16.7	9.2

Citation

@article{miah2026trace,
  title   = {TRACE: Trajectory Aware Reasoning for Multi-Turn Adversarial Conversation Evaluation},
  author  = {Miah, Md Messal Monem and Anika, Adrita and Yu, Zhiyuan and Huang, Ruihong},
  journal = {arXiv preprint arXiv:2608.15594},
  year    = {2026}
}