EuroSafeGuard v1: probe, 5 input-classifier seeds, exchange classifier, thresholds, loader, model card

.gitattributes

@@ -33,3 +33,9 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+exchange/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+gate/seed0/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+gate/seed1/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+gate/seed2/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+gate/seed3/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+gate/seed4/tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,114 @@
+---
+license: apache-2.0
+language:
+- en
+base_model:
+- Qwen/Qwen3.5-9B
+- meta-llama/Llama-3.1-8B-Instruct
+tags:
+- safety
+- guardrail
+- constitutional-classifiers
+- jailbreak-detection
+- content-moderation
+library_name: peft
+pipeline_tag: text-classification
+---
+
+# EuroSafeGuard
+
+**EuroSafeGuard** is an open constitutional-classifier guardrail from EuroSafeAI. It screens the requests sent to a
+language model and the replies the model produces, and it is trained on data generated from a written constitution of
+eleven harm categories.
+
+Paper: *EuroSafeAI Open Constitutional Classifiers* (arXiv: TODO) · Code: TODO · Data:
+[EuroSafeGuard-prompts](https://huggingface.co/datasets/EuroSafeAI/EuroSafeGuard-prompts),
+[EuroSafeGuard-replies](https://huggingface.co/datasets/EuroSafeAI/EuroSafeGuard-replies) (gated)
+
+## Components
+
+| Folder | What it is | Base model |
+|---|---|---|
+| `probe/` | MLP probe (hidden size 256) on the activations of all 32 blocks of the protected model, reading the chat-formatted prompt; per-token logits are smoothed with an exponential moving average and the prompt score is their maximum | Llama-3.1-8B-Instruct |
+| `gate/seed{0..4}/` | Input classifier: LoRA adapter + one sigmoid head per harm category (11 heads); score = highest head | Qwen3.5-9B |
+| `exchange/` | Exchange classifier: reads `User: <prompt>\n\nAssistant: <reply>`; LoRA + 11 heads | Qwen3.5-9B |
+| `thresholds/seed{k}.json` | Score normalisation and thresholds for each input-classifier seed | |
+| `eurosafeguard.py` | Single-file loader | |
+
+The five `gate/seed*` folders are the five training runs (same data, different seeds) that the paper reports as a mean.
+They vary, especially against search attacks such as PAIR, so evaluate the seed you deploy.
+
+**Harm categories:** CBRN, Cyber, Harm to Minors, Harmful Manipulation, Hate Speech, Illegal Activities, Information
+Integrity, Physical Harm, Privacy, Self-Harm, Sexual Content. `taxonomy_11heads_short.json` lists what each category
+allows and disallows.
+
+## How it decides (alert-unless-CBRN)
+
+1. **Input stage.** The input score is the mean of the probe and input-classifier scores, each z-scored on
+   WildChat-clean. Its threshold (`alert_input_thr`) flags 5% of WildChat-clean prompts.
+2. A flagged request whose top category is **CBRN** is **refused**. Any other flagged request is **answered under an
+   alert system note** (`system_note(category)`), which tells the protected model what it must not provide.
+3. **Output stage.** The exchange classifier checks the reply. The reply is withheld if its score exceeds
+   `exch_thr_20pct` for alerted requests (a stricter threshold, flagging 20% of WildChat-clean replies), or `exch_thr_1pct`
+   for unflagged requests (flagging 1%).
+
+```python
+from eurosafeguard import EuroSafeGuard          # download eurosafeguard.py from this repo
+g = EuroSafeGuard.from_pretrained("EuroSafeAI/EuroSafeGuard", seed=0, devices=("cuda:0", "cuda:0", "cuda:1"))
+decision, category, score = g.route(prompt)      # "pass" | "alert" | "refuse"
+messages = [{"role": "user", "content": prompt}]
+if decision == "alert":
+    messages.insert(0, {"role": "system", "content": g.system_note(category)})
+reply = ...                                        # your protected model
+if decision == "refuse" or g.check_reply(prompt, reply, decision):
+    reply = "Sorry, I can't help with that."
+```
+
+**Requirements:** `torch`, `transformers` with Qwen3.5 support, `peft`, `safetensors`, `huggingface_hub`. About 16 GB of
+GPU memory for Llama-3.1-8B (the probe) plus about 19 GB for each Qwen3.5-9B classifier (input and exchange). Use
+`load_exchange=False` to run the input stage only. The probe reads Llama-3.1-8B-Instruct activations. If you protect
+Llama-3.1-8B-Instruct itself, you can reuse the activations of its forward pass.
+
+## Results (from the paper; every guard calibrated to the same false-positive budget on real chat traffic)
+
+- **BELLS-O:** our input classifiers detect more harmful prompts than any supervisor on the leaderboard at a
+  false-positive rate near 1%, and have the highest recall on its held-out jailbreak sets.
+- **PAIR** (iterative black-box attack), input stage at the 5% WildChat-clean budget: 21% of the behaviors that break
+  the unprotected model get through, against 49% for the best other open guard.
+- **Chemical and biological goals** transferred to three target models: at most 1.1% get through in every setting.
+- **Alert mode** (this release's rule) vs refusing every flagged request: refusals of benign sensitive requests fall
+  from 37% to 5–9%, while at most 3.6% of chemical and biological goals get through on five protected models.
+- **Latency:** the full guardrail adds 1.5% to generation time on the requests it escalates.
+
+Numbers are means over the five input-classifier seeds; see the paper for the full tables, judges and budgets.
+
+## Limitations
+
+- **Low-resource languages:** recall on multilingual harmful prompts (MSafeTemp) drops to 74% in low-resource
+  languages, below several other guards.
+- **Content moderation:** on general moderation benchmarks (ToxicChat, OpenAI Moderation and others), dedicated
+  moderation models remain ahead.
+- **Over-refusal:** the input stage flags a fraction of benign but sensitive requests. Alert mode answers most of them,
+  but it is not a substitute for checking false positives on your own traffic.
+- **Fixed policy:** the thresholds are calibrated on WildChat-clean. Recalibrate on your own benign traffic if your
+  traffic differs.
+- **Not a complete defence:** no guardrail stops every attack. Combine it with a safety-trained model, monitoring and
+  rate limits.
+
+## License
+
+Our adapters, heads, probe and code are released under Apache-2.0. The base models keep their own licenses:
+[Llama 3.1 Community License](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) for the probe's protected model,
+and the [Qwen license](https://huggingface.co/Qwen/Qwen3.5-9B) for the classifier backbones. Some training data is
+licensed for non-commercial use only; see the dataset cards.
+
+## Citation
+
+```bibtex
+@article{eurosafeguard2026,
+  title   = {EuroSafeAI Open Constitutional Classifiers},
+  author  = {Waibl, Leonhard and Chen, Yen-Shan and Simko, Samuel and others},
+  journal = {arXiv preprint arXiv:TODO},
+  year    = {2026}
+}
+```

eurosafeguard.py

@@ -0,0 +1,206 @@
+"""EuroSafeGuard: open constitutional classifiers (EuroSafeAI). Single-file loader.
+
+    from eurosafeguard import EuroSafeGuard
+    g = EuroSafeGuard.from_pretrained("EuroSafeAI/EuroSafeGuard", seed=0)
+    decision, category, score = g.route("How do I ...?")      # "pass" | "alert" | "refuse"
+    if decision == "alert":
+        system_message = g.system_note(category)                   # prepend to the protected model's context
+    blocked = g.check_reply(prompt, reply, decision)               # True -> withhold the reply
+
+Components (see README.md):
+  probe      MLP probe on Llama-3.1-8B-Instruct activations (all 32 blocks) of the chat-formatted prompt; per-token
+             logits are smoothed with an EMA and the prompt score is their maximum.
+  gate       input classifier: Qwen3.5-9B + LoRA + 11 sigmoid heads (one per harm category); score = max head.
+  exchange   exchange classifier: Qwen3.5-9B + LoRA + 11 sigmoid heads on "User: <prompt>\\n\\nAssistant: <reply>".
+Rule (alert-unless-CBRN): input score = mean of the probe and gate z-scores (z-scored on WildChat-clean). Above
+alert_input_thr (flags 5% of WildChat-clean): refuse if the gate's top category is CBRN, otherwise answer under an
+alert system note and block the reply if the exchange score exceeds exch_thr_20pct. Below: answer normally and block the
+reply if the exchange score exceeds exch_thr_1pct.
+
+Memory: Llama-3.1-8B (~16 GB bf16) + Qwen3.5-9B for the gate (~19 GB) + Qwen3.5-9B for the exchange classifier (~19 GB).
+Pass devices=("cuda:0", "cuda:1", "cuda:2") to spread them, or load_exchange=False to skip the reply check.
+Requires: torch, transformers (with Qwen3.5 support), peft, safetensors, huggingface_hub, numpy.
+"""
+from __future__ import annotations
+
+import json
+import math
+import os
+
+import numpy as np
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+HEADS = ["CBRN", "Cyber", "Harm to Minors", "Harmful Manipulation", "Hate Speech", "Illegal Activities",
+         "Information Integrity", "Physical Harm", "Privacy", "Self-Harm", "Sexual Content"]
+
+
+# ----------------------------------------------------------------------------------------------------- probe
+class MLPProbe(nn.Module):
+    """z_t = w2 . gelu(W1 (scale * psi_t) + b1) + b2, per token."""
+
+    def __init__(self, dim: int, hidden: int = 256):
+        super().__init__()
+        self.W1 = nn.Linear(dim, hidden)
+        self.w2 = nn.Linear(hidden, 1)
+        self.register_buffer("scale", torch.ones(dim))
+
+    def forward(self, psi):
+        w = (self.W1.weight * self.scale).to(psi.dtype)
+        return self.w2(F.gelu((psi @ w.T).float() + self.W1.bias)).squeeze(-1)
+
+
+def ema_max(z: torch.Tensor, mask: torch.Tensor, M: int = 16) -> torch.Tensor:
+    """Streaming score: EMA of per-token logits (alpha = 2/(M+1)) over scored positions; returns the max per row."""
+    a = 2.0 / (M + 1)
+    state = torch.zeros(z.shape[0], dtype=z.dtype, device=z.device)
+    started = torch.zeros(z.shape[0], dtype=torch.bool, device=z.device)
+    best = torch.full((z.shape[0],), -1e9, dtype=z.dtype, device=z.device)
+    for t in range(z.shape[1]):
+        m = mask[:, t]
+        new = torch.where(started, (1 - a) * state + a * z[:, t], z[:, t])
+        state = torch.where(m, new, state)
+        started = started | m
+        best = torch.where(m, torch.maximum(best, state), best)
+    return best
+
+
+# ----------------------------------------------------------------------------------------------------- classifiers
+class LoRAClassifier:
+    """Causal LM + LoRA adapter + linear head on the last real token (sigmoid per category)."""
+
+    def __init__(self, path: str, device: str, base_model: str | None = None):
+        from peft import LoraConfig, get_peft_model, set_peft_model_state_dict
+        from safetensors.torch import load_file
+        from transformers import AutoModelForCausalLM, AutoTokenizer
+
+        cfg = json.load(open(os.path.join(path, "train_config.json")))
+        base_model = base_model or cfg["model_name"]
+        self.device, self.max_length = device, cfg.get("max_length", 1024)
+        base = AutoModelForCausalLM.from_pretrained(base_model, dtype=torch.bfloat16)
+        lcfg = LoraConfig(r=cfg.get("lora_r", 16), lora_alpha=cfg.get("lora_alpha", 32),
+                          lora_dropout=cfg.get("lora_dropout", 0.05),
+                          target_modules=cfg.get("lora_target_modules") or ["q_proj", "v_proj"], bias="none",
+                          task_type="CAUSAL_LM")
+        self.backbone = get_peft_model(base, lcfg)
+        set_peft_model_state_dict(self.backbone, load_file(os.path.join(path, "adapter_model.safetensors")))
+        self.backbone = self.backbone.to(device).eval()
+        self.head = nn.Linear(base.config.hidden_size, cfg["num_labels"])
+        self.head.load_state_dict(torch.load(os.path.join(path, "classification_head.pt"), map_location="cpu",
+                                             weights_only=True))
+        self.head = self.head.to(device).eval()
+        self.tok = AutoTokenizer.from_pretrained(path)
+        if self.tok.pad_token is None:
+            self.tok.pad_token = self.tok.eos_token
+        self.tok.padding_side = "right"
+
+    @torch.no_grad()
+    def probs(self, texts: list[str], batch_size: int = 8) -> np.ndarray:
+        out = []
+        for i in range(0, len(texts), batch_size):
+            enc = self.tok(texts[i:i + batch_size], return_tensors="pt", truncation=True, max_length=self.max_length,
+                           padding=True).to(self.device)
+            hs = self.backbone(**enc, output_hidden_states=True).hidden_states[-1]
+            last = enc["attention_mask"].shape[1] - 1 - enc["attention_mask"].flip(1).long().argmax(1)
+            pooled = hs[torch.arange(hs.shape[0], device=hs.device), last]
+            out.append(torch.sigmoid(self.head(pooled.to(self.head.weight.dtype)).float()).cpu().numpy())
+        return np.concatenate(out, 0)
+
+
+# ----------------------------------------------------------------------------------------------------- guard
+class EuroSafeGuard:
+    def __init__(self, root: str, seed: int = 0, devices=("cuda", "cuda", "cuda"), load_exchange: bool = True):
+        from safetensors.torch import load_file
+        from transformers import AutoModelForCausalLM, AutoTokenizer
+
+        self.c = json.load(open(os.path.join(root, "thresholds", f"seed{seed}.json")))
+        self.tax = json.load(open(os.path.join(root, "taxonomy_11heads_short.json")))
+        pc = json.load(open(os.path.join(root, "probe", "probe_config.json")))
+        self.layers, self.M, self.max_len = pc["layers"], pc["M"], pc["max_len"]
+        self.dev_lm, self.dev_gate, self.dev_x = devices
+        self.lm_tok = AutoTokenizer.from_pretrained(pc["base_model"])
+        self.lm = AutoModelForCausalLM.from_pretrained(pc["base_model"], dtype=torch.bfloat16).to(self.dev_lm).eval()
+        self.probe = MLPProbe(pc["feature_dim"], pc["hidden"])
+        self.probe.load_state_dict(load_file(os.path.join(root, "probe", "probe.safetensors")))
+        self.probe = self.probe.to(self.dev_lm).eval()
+        self.gate = LoRAClassifier(os.path.join(root, "gate", f"seed{seed}"), self.dev_gate)
+        self.exchange = LoRAClassifier(os.path.join(root, "exchange"), self.dev_x) if load_exchange else None
+
+    @classmethod
+    def from_pretrained(cls, repo_or_path: str = "EuroSafeAI/EuroSafeGuard", seed: int = 0, **kw):
+        if os.path.isdir(repo_or_path):
+            return cls(repo_or_path, seed, **kw)
+        from huggingface_hub import snapshot_download
+        root = snapshot_download(repo_or_path, allow_patterns=["probe/*", f"gate/seed{seed}/*", "exchange/*",
+                                                               f"thresholds/seed{seed}.json", "*.json", "*.py"])
+        return cls(root, seed, **kw)
+
+    # -- input stage
+    @torch.no_grad()
+    def probe_scores(self, prompts: list[str], batch_size: int = 8) -> np.ndarray:
+        out = np.zeros(len(prompts), dtype=np.float32)
+        pad = self.lm_tok.pad_token_id if self.lm_tok.pad_token_id is not None else self.lm_tok.eos_token_id
+        for i in range(0, len(prompts), batch_size):
+            ids = [self.lm_tok(self.lm_tok.apply_chat_template([{"role": "user", "content": str(p)}], tokenize=False,
+                                                               add_generation_prompt=True),
+                               add_special_tokens=False)["input_ids"][-self.max_len:] for p in prompts[i:i + batch_size]]
+            T = max(map(len, ids))
+            inp = torch.full((len(ids), T), pad, dtype=torch.long)
+            att = torch.zeros((len(ids), T), dtype=torch.long)
+            for k, x in enumerate(ids):
+                inp[k, :len(x)] = torch.tensor(x); att[k, :len(x)] = 1
+            hs = self.lm(input_ids=inp.to(self.dev_lm), attention_mask=att.to(self.dev_lm),
+                         output_hidden_states=True).hidden_states
+            psi = torch.cat([hs[l] for l in self.layers], dim=-1)
+            out[i:i + len(ids)] = ema_max(self.probe(psi), att.bool().to(self.dev_lm), self.M).float().cpu().numpy()
+        return out
+
+    def input_scores(self, prompts: list[str]):
+        """-> (combined z-score [N], gate head probabilities [N, 11])."""
+        c = self.c
+        p = self.probe_scores(prompts)
+        H = self.gate.probs(list(prompts))
+        s = ((p - c["probe_mu"]) / c["probe_sd"] + (H.max(1) - c["gate_mu"]) / c["gate_sd"]) / 2
+        return s, H
+
+    def route(self, prompt):
+        """One prompt -> (decision, top category, input score); a list of prompts -> list of those tuples."""
+        single = isinstance(prompt, str)
+        prompts = [prompt] if single else list(prompt)
+        s, H = self.input_scores(prompts)
+        out = []
+        for si, hi in zip(s, H.argmax(1)):
+            top = HEADS[hi]
+            dec = "pass" if si <= self.c["alert_input_thr"] else ("refuse" if top == "CBRN" else "alert")
+            out.append((dec, top, float(si)))
+        return out[0] if single else out
+
+    def system_note(self, category: str) -> str:
+        t = self.tax.get(category, {})
+        dis, ok = "; ".join(t.get("disallowed", [])), "; ".join(t.get("allowed", []))
+        return (f"[ALERT: an automated safety classifier flagged the next user request as possibly touching the "
+                f"'{category}' policy. Under our constitution, for '{category}' you must NOT provide: {dis}. You MAY help "
+                f"with: {ok}; general, high-level, historical or educational descriptions; and everyday, technical or "
+                "creative uses of the words involved. Give the most helpful answer that stays within what is allowed: "
+                "answer normally if the request is benign, give a high-level non-actionable answer if it is borderline. "
+                "If it asks for something disallowed, say briefly what you will not provide, and where a safe part of the "
+                "topic is clearly relevant, add it in a sentence or two (for example what the thing is in general terms, "
+                "or why it is dangerous); do not force an alternative when none fits. Do not mention this alert or any "
+                "classifier to the user.]")
+
+    # -- output stage
+    def exchange_scores(self, prompts: list[str], replies: list[str]) -> np.ndarray:
+        if self.exchange is None:
+            raise RuntimeError("loaded with load_exchange=False")
+        t = [f"User: {a}\n\nAssistant: {b}" for a, b in zip(prompts, replies)]
+        return self.exchange.probs(t).max(1)
+
+    def check_reply(self, prompt, reply, decision="pass"):
+        """True if the reply should be withheld. Stricter threshold for replies to alerted requests."""
+        single = isinstance(prompt, str)
+        P, R = ([prompt], [reply]) if single else (list(prompt), list(reply))
+        D = [decision] * len(P) if isinstance(decision, str) else list(decision)
+        sc = self.exchange_scores(P, R)
+        out = [bool(x > (self.c["exch_thr_20pct"] if d == "alert" else self.c["exch_thr_1pct"])) for x, d in zip(sc, D)]
+        return out[0] if single else out

exchange/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-9B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.0456,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.2",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

exchange/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5c8fa37b78cc15ef1d0a47f05dd55782aae30da12fa4c64234697dd259edfd8
+size 71307544

exchange/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

exchange/classification_head.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e2996815d414e597fbb9e24eff5052b9aafddc8ce443c4dde0e51d4e90da8be4
+size 182277

exchange/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6ae9719a160898e8c38d3072c2862d8b3cf610c4069e447a8dfd457cff2a6c0f
+size 19989592

exchange/tokenizer_config.json

@@ -0,0 +1,32 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

exchange/train_config.json

@@ -0,0 +1,34 @@
+{
+ "model_name": "Qwen/Qwen3.5-9B",
+ "objective": "multilabel",
+ "balanced_sampling": false,
+ "num_labels": 11,
+ "borderline_benign_target": 0.5,
+ "dual_use_harmful_target": 0.8,
+ "lora_r": 128,
+ "lora_alpha": 256,
+ "lora_dropout": 0.0456,
+ "lora_target_modules": null,
+ "lr": 0.0002,
+ "lr_backbone": 9.16254877203936e-05,
+ "lr_head": 0.00020784609690826527,
+ "max_steps": 8000,
+ "batch_size": 4,
+ "max_grad_norm": 1.0,
+ "compile": false,
+ "seed": 0,
+ "max_length": 2048,
+ "dataset_loop": true,
+ "dataset_text_column": "prompt_response",
+ "dataset_category_column": "category",
+ "dataset_entry_type_column": "entry_type",
+ "dataset_method_column": null,
+ "dataset_split": "train",
+ "sample_categories": null,
+ "sample_complexity": 1,
+ "eval_frac": 0.02,
+ "eval_every": 500,
+ "table_every": 500,
+ "moving_avg_window": 100,
+ "save_every": 2000
+}

exchange/weight_frame.json

@@ -0,0 +1,25 @@
+{
+ "version": "0.1.0",
+ "classifier": {
+  "base_model": "Qwen/Qwen3.5-9B",
+  "taxonomy": [
+   "Benign",
+   "CBRN",
+   "Cyber",
+   "Harm to Minors",
+   "Harmful Manipulation",
+   "Hate Speech",
+   "Illegal Activities",
+   "Information Integrity",
+   "Physical Harm",
+   "Privacy",
+   "Self-Harm",
+   "Sexual Content"
+  ],
+  "objective": "multilabel",
+  "adapter_path": null,
+  "adapter_repo": null,
+  "head_filename": "classification_head.pt",
+  "thresholds": null
+ }
+}

gate/seed0/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-9B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.0456,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.2",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

gate/seed0/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a67631789a29127f67e5929822b57189ae346236967d4b755681ee8597afe102
+size 71307544

gate/seed0/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

gate/seed0/classification_head.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:21ade7483684505a19961da0df0841397bbb656194be7c0ac0c16b86d51daa04
+size 182277

gate/seed0/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fee140e1e6ed13c7070c891349ce980eb030618221dde35dd00e44f8cfbdacf
+size 19989592

gate/seed0/tokenizer_config.json

@@ -0,0 +1,32 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

gate/seed0/train_config.json

@@ -0,0 +1,34 @@
+{
+ "model_name": "Qwen/Qwen3.5-9B",
+ "objective": "multilabel",
+ "balanced_sampling": false,
+ "num_labels": 11,
+ "borderline_benign_target": 0.5,
+ "dual_use_harmful_target": 0.8,
+ "lora_r": 128,
+ "lora_alpha": 256,
+ "lora_dropout": 0.0456,
+ "lora_target_modules": null,
+ "lr": 0.0002,
+ "lr_backbone": 5.29e-05,
+ "lr_head": 0.00012,
+ "max_steps": 24000,
+ "batch_size": 4,
+ "max_grad_norm": 1.0,
+ "compile": false,
+ "seed": 0,
+ "max_length": 1024,
+ "dataset_loop": true,
+ "dataset_text_column": "prompt",
+ "dataset_category_column": "category",
+ "dataset_entry_type_column": "entry_type",
+ "dataset_method_column": null,
+ "dataset_split": "train",
+ "sample_categories": null,
+ "sample_complexity": 1,
+ "eval_frac": 0.02,
+ "eval_every": 2000,
+ "table_every": 500,
+ "moving_avg_window": 100,
+ "save_every": 12000
+}

gate/seed0/weight_frame.json

@@ -0,0 +1,25 @@
+{
+ "version": "0.1.0",
+ "classifier": {
+  "base_model": "Qwen/Qwen3.5-9B",
+  "taxonomy": [
+   "Benign",
+   "CBRN",
+   "Cyber",
+   "Harm to Minors",
+   "Harmful Manipulation",
+   "Hate Speech",
+   "Illegal Activities",
+   "Information Integrity",
+   "Physical Harm",
+   "Privacy",
+   "Self-Harm",
+   "Sexual Content"
+  ],
+  "objective": "multilabel",
+  "adapter_path": null,
+  "adapter_repo": null,
+  "head_filename": "classification_head.pt",
+  "thresholds": null
+ }
+}

gate/seed1/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-9B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.0456,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.2",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

gate/seed1/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:41777884895fcd0b84ce51198fa9244997b9240fc9429dcfff4e260f6fd8fc37
+size 71307544

gate/seed1/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

gate/seed1/classification_head.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f16ad52e4c7249b092ef0d1c691bd6f83c7ee8bfbd3139fbac225321c1f294d
+size 182277

gate/seed1/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fee140e1e6ed13c7070c891349ce980eb030618221dde35dd00e44f8cfbdacf
+size 19989592

gate/seed1/tokenizer_config.json

@@ -0,0 +1,32 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

gate/seed1/train_config.json

@@ -0,0 +1,34 @@
+{
+ "model_name": "Qwen/Qwen3.5-9B",
+ "objective": "multilabel",
+ "balanced_sampling": false,
+ "num_labels": 11,
+ "borderline_benign_target": 0.5,
+ "dual_use_harmful_target": 0.8,
+ "lora_r": 128,
+ "lora_alpha": 256,
+ "lora_dropout": 0.0456,
+ "lora_target_modules": null,
+ "lr": 0.0002,
+ "lr_backbone": 5.29e-05,
+ "lr_head": 0.00012,
+ "max_steps": 24000,
+ "batch_size": 4,
+ "max_grad_norm": 1.0,
+ "compile": false,
+ "seed": 1,
+ "max_length": 1024,
+ "dataset_loop": true,
+ "dataset_text_column": "prompt",
+ "dataset_category_column": "category",
+ "dataset_entry_type_column": "entry_type",
+ "dataset_method_column": null,
+ "dataset_split": "train",
+ "sample_categories": null,
+ "sample_complexity": 1,
+ "eval_frac": 0.02,
+ "eval_every": 2000,
+ "table_every": 500,
+ "moving_avg_window": 100,
+ "save_every": 12000
+}

gate/seed1/weight_frame.json

@@ -0,0 +1,25 @@
+{
+ "version": "0.1.0",
+ "classifier": {
+  "base_model": "Qwen/Qwen3.5-9B",
+  "taxonomy": [
+   "Benign",
+   "CBRN",
+   "Cyber",
+   "Harm to Minors",
+   "Harmful Manipulation",
+   "Hate Speech",
+   "Illegal Activities",
+   "Information Integrity",
+   "Physical Harm",
+   "Privacy",
+   "Self-Harm",
+   "Sexual Content"
+  ],
+  "objective": "multilabel",
+  "adapter_path": null,
+  "adapter_repo": null,
+  "head_filename": "classification_head.pt",
+  "thresholds": null
+ }
+}

gate/seed2/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-9B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.0456,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.2",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

gate/seed2/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd2fe684dc53236bc2a19fe8f374b6f23814007178dad97dc163a2aa95291337
+size 71307544

gate/seed2/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

gate/seed2/classification_head.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f90ed1d53d3232be7d08540140143b68bc65f23a730de09ad8219217a02ef883
+size 182277

gate/seed2/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fee140e1e6ed13c7070c891349ce980eb030618221dde35dd00e44f8cfbdacf
+size 19989592

gate/seed2/tokenizer_config.json

@@ -0,0 +1,32 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

gate/seed2/train_config.json

@@ -0,0 +1,34 @@
+{
+ "model_name": "Qwen/Qwen3.5-9B",
+ "objective": "multilabel",
+ "balanced_sampling": false,
+ "num_labels": 11,
+ "borderline_benign_target": 0.5,
+ "dual_use_harmful_target": 0.8,
+ "lora_r": 128,
+ "lora_alpha": 256,
+ "lora_dropout": 0.0456,
+ "lora_target_modules": null,
+ "lr": 0.0002,
+ "lr_backbone": 5.29e-05,
+ "lr_head": 0.00012,
+ "max_steps": 24000,
+ "batch_size": 4,
+ "max_grad_norm": 1.0,
+ "compile": false,
+ "seed": 2,
+ "max_length": 1024,
+ "dataset_loop": true,
+ "dataset_text_column": "prompt",
+ "dataset_category_column": "category",
+ "dataset_entry_type_column": "entry_type",
+ "dataset_method_column": null,
+ "dataset_split": "train",
+ "sample_categories": null,
+ "sample_complexity": 1,
+ "eval_frac": 0.02,
+ "eval_every": 2000,
+ "table_every": 500,
+ "moving_avg_window": 100,
+ "save_every": 12000
+}

gate/seed2/weight_frame.json

@@ -0,0 +1,25 @@
+{
+ "version": "0.1.0",
+ "classifier": {
+  "base_model": "Qwen/Qwen3.5-9B",
+  "taxonomy": [
+   "Benign",
+   "CBRN",
+   "Cyber",
+   "Harm to Minors",
+   "Harmful Manipulation",
+   "Hate Speech",
+   "Illegal Activities",
+   "Information Integrity",
+   "Physical Harm",
+   "Privacy",
+   "Self-Harm",
+   "Sexual Content"
+  ],
+  "objective": "multilabel",
+  "adapter_path": null,
+  "adapter_repo": null,
+  "head_filename": "classification_head.pt",
+  "thresholds": null
+ }
+}

gate/seed3/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-9B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.0456,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.2",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

gate/seed3/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:683814b7dcc72874c8ce1d1c36410f5eba2cb56757e90527ce95ea43ccea41c7
+size 71307544

gate/seed3/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

gate/seed3/classification_head.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2941d125f051024dc2c902376f70c829f93505007cda4f986c3446a33c0349fe
+size 182277

gate/seed3/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fee140e1e6ed13c7070c891349ce980eb030618221dde35dd00e44f8cfbdacf
+size 19989592

gate/seed3/tokenizer_config.json

@@ -0,0 +1,32 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

gate/seed3/train_config.json

@@ -0,0 +1,34 @@
+{
+ "model_name": "Qwen/Qwen3.5-9B",
+ "objective": "multilabel",
+ "balanced_sampling": false,
+ "num_labels": 11,
+ "borderline_benign_target": 0.5,
+ "dual_use_harmful_target": 0.8,
+ "lora_r": 128,
+ "lora_alpha": 256,
+ "lora_dropout": 0.0456,
+ "lora_target_modules": null,
+ "lr": 0.0002,
+ "lr_backbone": 5.29e-05,
+ "lr_head": 0.00012,
+ "max_steps": 24000,
+ "batch_size": 4,
+ "max_grad_norm": 1.0,
+ "compile": false,
+ "seed": 3,
+ "max_length": 1024,
+ "dataset_loop": true,
+ "dataset_text_column": "prompt",
+ "dataset_category_column": "category",
+ "dataset_entry_type_column": "entry_type",
+ "dataset_method_column": null,
+ "dataset_split": "train",
+ "sample_categories": null,
+ "sample_complexity": 1,
+ "eval_frac": 0.02,
+ "eval_every": 2000,
+ "table_every": 500,
+ "moving_avg_window": 100,
+ "save_every": 12000
+}

gate/seed3/weight_frame.json

@@ -0,0 +1,25 @@
+{
+ "version": "0.1.0",
+ "classifier": {
+  "base_model": "Qwen/Qwen3.5-9B",
+  "taxonomy": [
+   "Benign",
+   "CBRN",
+   "Cyber",
+   "Harm to Minors",
+   "Harmful Manipulation",
+   "Hate Speech",
+   "Illegal Activities",
+   "Information Integrity",
+   "Physical Harm",
+   "Privacy",
+   "Self-Harm",
+   "Sexual Content"
+  ],
+  "objective": "multilabel",
+  "adapter_path": null,
+  "adapter_repo": null,
+  "head_filename": "classification_head.pt",
+  "thresholds": null
+ }
+}

gate/seed4/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-9B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.0456,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.2",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

gate/seed4/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5960949b06a641b1acf833d10655a280a18d6d4a419585e04a980633ca21b356
+size 71307544

gate/seed4/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

gate/seed4/classification_head.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c51b919d71b24e1db8ba339c50a7152ece34b4f6a0237da117da5b8a7bbd91f
+size 182277

gate/seed4/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fee140e1e6ed13c7070c891349ce980eb030618221dde35dd00e44f8cfbdacf
+size 19989592

gate/seed4/tokenizer_config.json

@@ -0,0 +1,32 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}