Release KUTUP-1-Decision-2B with AutoModel & AutoTokenizer support

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,133 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen3.5-2B-Base
+tags:
+- clef
+- systemone
+- decision-model
+- zero-token
+- latent-reasoning
+- turkish
+- classification
+datasets:
+- bezir/MMLU-pro-TR
+metrics:
+- accuracy
+model-index:
+- name: KUTUP-1-Decision-2B
+  results:
+  - task:
+      type: multiple-choice
+      name: Multiple Choice Question Answering
+    dataset:
+      name: MMLU-Pro-TR
+      type: bezir/MMLU-pro-TR
+    metrics:
+    - name: Accuracy
+      type: accuracy
+      value: 34.12
+---
+
+# 🚀 KUTUP-1-Decision-2B: Türkçe SystemOne Karar Motoru (0-Token Latent Decision)
+
+**KUTUP-1-Decision-2B**, Cloudflare'ın öncülük ettiği **CLEF / SystemOne (Classification via Latent Evidence Fusion)** mimarisi referans alınarak, `Qwen/Qwen3.5-2B-Base` omurgası üzerinde sıfırdan inşa edilmiş ve eğitilmiş **ilk yerli Türkçe Ayrımcı Karar Motorudur.**
+
+Geleneksel üretken LLM'lerin (Autoregressive) aksine, bu model yeni kelimeler üretmez (0-Token Inference). Girdi metnini tek bir ileri geçişte (Forward Pass) işleyerek sadece **43 milisaniyede** doğrudan yapılandırılmış kararlar, olasılık dağılımları ve kalibre edilmiş güven skorları üretir.
+
+---
+
+## 🏆 Benchmark Karşılaştırması: MMLU-Pro-TR (bezir/MMLU-pro-TR)
+
+10 seçenekli (A-J), üniversite ve doktora seviyesindeki 11.842 adet zorlu Türkçe soru üzerinde yapılan genel bilgi ve akıl yürütme değerlendirmesi:
+
+| Model | Mimari / Türü | Parametre Boyutu | MMLU-Pro-TR Doğruluk | Ortalama Gecikme |
+| :--- | :--- | :---: | :---: | :---: |
+| **Cloudflare CLEF-Flash** | Ayrımcı SystemOne (Global) | **9B** | **%41.88** | 79.67 ms |
+| **KUTUP-1-Decision-2B (Bu Model)** | **Ayrımcı SystemOne (Türkçe)** | **2B** | **%34.12** | **43.12 ms** |
+| **LiquidAI/d1-3B** | Multimodal / Autoregressive | **3B** | **%27.24** | 17.11 ms |
+| **Laya-Multilingual** | mmBERT Karar Modeli | **~300M** | **%11.76** | 10.48 ms |
+
+### 💡 Kritik Başarılar:
+1. **+1 Milyar Parametre Üstünlüğü:** Kendisinden 1 milyar daha büyük parametreye sahip olan `LiquidAI/d1-3B` modeline **neredeyse 7 puan net fark** atmıştır (%34.12 vs %27.24).
+2. **4.5 Kat Büyük 9B Modelin Ensesinde:**
+   - **Biyoloji:** %59.52 (LiquidAI: %49.58)
+   - **Psikoloji:** %49.87 (LiquidAI: %35.13)
+   - **İktisat:** %46.27 (LiquidAI: %34.46)
+   - **Sağlık:** %39.00 (LiquidAI: %20.25)
+   - **Bilgisayar Bilimi:** %34.76 (LiquidAI: %24.18)
+   - **Mühendislik:** %37.41 (9B modelin %40.21'lik skoruna yakın rekabette)
+
+---
+
+## 🛠️ Mimari ve Katman Tasarımı
+
+1. **Omurga (Backbone):** `Qwen/Qwen3.5-2B-Base` (LoRA adaptörleri ile r=32, α=64).
+2. **Context Memory Projeksiyonu:** 2048 boyutlu omurga uzayını 1024 boyutlu karar uzayına indirgeyen darboğaz katmanı.
+3. **EvidenceRoutingBlock (Cross-Attention):** Seçeneklerin metindeki kanıtları kendi içine çekmesini sağlayan delil toplama bloğu.
+4. **Joint Transformer Decoder (x4):** Seçeneklerin birbirini eleme mantığıyla yarıştığı derin karar katmanı.
+5. **İkili Skorlayıcı (Prior + Gated Residual MLP):** Sözcüksel benzerlik ile derin mantıksal çelişkileri birleştiren nihai kapılı skorlama modülü.
+
+---
+
+## 📚 Nasıl Eğitildi?
+
+Model 2 aşamalı disiplinli bir eğitimden geçmiştir:
+- **Aşama 1 (Head Warmup):** Qwen omurgası dondurularak 100.000 örnekle sadece `JointSchemaHead` katmanları ısıtıldı.
+- **Aşama 2 (Knowledge & Decision LoRA):** 38.3 milyon tokenlık "3'lü Altın Harman":
+  - 35.000 adet çok alanlı kurumsal sentetik sözleşme (max 1.500 token),
+  - 13.985 adet saf akademik MMLU-TR sorusu (57 disiplin),
+  - 25.000 adet NLI mantıksal çelişki verisi.
+  - Parametre başına düşen token oranı **1.004 token / parametre** ideal seviyesine oturtuldu. Brier Kalibrasyonu ile güven skorları hizalandı.
+
+---
+
+## 💻 Hızlı Kullanım (Quickstart)
+
+Model, standart Hugging Face `transformers` kütüphanesi üzerinden doğrudan `AutoModel` ve `AutoTokenizer` ile kullanılabilir:
+
+```python
+import torch
+from transformers import AutoModel, AutoTokenizer
+
+model_id = "TurkishCodeMan/KUTUP-1-Decision-2B"
+
+# 1. Tokenizer ve Modeli Yükle (GPU üzerinde ~6-8 GB VRAM)
+tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
+model = AutoModel.from_pretrained(
+    model_id,
+    trust_remote_code=True,
+    device_map="cuda:0",
+    torch_dtype=torch.bfloat16
+)
+
+# 2. Durum ve Karar Sorularını Tanımla
+durum = "Hasta 48 yaşında erkek. Göğüs sol tarafında sıkıştırıcı ağrı ve EKG'de ST yükselmesi mevcut."
+sorular = {
+    "klinik_teshis": {
+        "type": "choice",
+        "instructions": "Bu klinik tablo için en olası acil tanı nedir?",
+        "criteria": {
+            "A": "Akut Miyokard Enfarktüsü (Kalp Krizi)",
+            "B": "Gastroözofageal Reflü",
+            "C": "Panik Atak",
+            "D": "Kaburga Kas Zedelenmesi"
+        }
+    }
+}
+
+# 3. Tek İleri Geçişte (43 ms) Kararı Al
+sonuc = model.decide(tokenizer=tokenizer, state=durum, questions=sorular)
+print(sonuc["answers"]["klinik_teshis"])
+# Çıktı: {'choice': 'A', 'confidence': 0.9517, 'probabilities': {'A': 0.9517, 'B': 0.0154, 'C': 0.0147, 'D': 0.0183}}
+```
+
+---
+
+## 🔮 Gelecek Planları (Roadmap)
+- **Stage 3: RLCD (Reinforcement Learning from Contrastive Decisions):** Daha geniş ve filtrelenmiş Türkçe karar senaryoları üzerinde çeldirici eleme politikasını pekiştirerek 2B boyutuyla 9B modelin genel skorunu geride bırakmak.
+
+---
+
+## 📜 Lisans & Atıf
+Bu model açık kaynak topluluğu için Apache 2.0 lisansı altında paylaşılmıştır.
+Geliştirici: **TurkishCodeMan**

config.json

@@ -0,0 +1,17 @@
+{
+  "architectures": [
+    "KutupForDecision"
+  ],
+  "model_type": "kutup_decision",
+  "auto_map": {
+    "AutoConfig": "configuration_kutup.KutupConfig",
+    "AutoModel": "modeling_kutup.KutupForDecision"
+  },
+  "backbone_model_id": "Qwen/Qwen3.5-2B-Base",
+  "hidden_size": 2048,
+  "decision_width": 1024,
+  "max_length": 4096,
+  "torch_dtype": "bfloat16",
+  "temperature": 1.0,
+  "transformers_version": "4.48.0"
+}

configuration_kutup.py

@@ -0,0 +1,23 @@
+from transformers import PretrainedConfig
+
+class KutupConfig(PretrainedConfig):
+    model_type = "kutup_decision"
+    keys_to_ignore_at_inference = ["past_key_values"]
+
+    def __init__(
+        self,
+        backbone_model_id="Qwen/Qwen3.5-2B-Base",
+        hidden_size=2048,
+        decision_width=1024,
+        max_length=4096,
+        torch_dtype="bfloat16",
+        temperature=1.0,
+        **kwargs
+    ):
+        super().__init__(**kwargs)
+        self.backbone_model_id = backbone_model_id
+        self.hidden_size = hidden_size
+        self.decision_width = decision_width
+        self.max_length = max_length
+        self.torch_dtype = torch_dtype
+        self.temperature = temperature

joint_head.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e98ad549ad3f45b0d79f93b245ce5e72953840b8748537bac0e3a644d8ecc321
+size 218363968

lora_adapter/README.md

@@ -0,0 +1,206 @@
+---
+base_model: Qwen/Qwen3.5-2B-Base
+library_name: peft
+tags:
+- base_model:adapter:Qwen/Qwen3.5-2B-Base
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.21.2

lora_adapter/adapter_config.json

@@ -0,0 +1,51 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-2B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.2",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "k_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "down_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "FEATURE_EXTRACTION",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

lora_adapter/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cca1ea27664bc4a5cac35a8a9232fd41fd6348f75734eecf5a64e34f0a6ce108
+size 87322136

modeling_kutup.py

@@ -0,0 +1,150 @@
+import os
+import json
+import time
+from typing import Any, Dict, Optional, Union
+import torch
+import torch.nn as nn
+from transformers import PreTrainedModel, AutoModelForCausalLM, AutoTokenizer
+from safetensors.torch import load_file
+from peft import PeftModel
+from .configuration_kutup import KutupConfig
+
+class EvidenceRoutingBlock(nn.Module):
+    def __init__(self, d_model: int, n_heads: int = 8, dropout: float = 0.05):
+        super().__init__()
+        self.cross_attn = nn.MultiheadAttention(d_model, n_heads, dropout=dropout, batch_first=True)
+        self.norm = nn.LayerNorm(d_model)
+        self.dropout = nn.Dropout(dropout)
+
+    def forward(self, query: torch.Tensor, key_value: torch.Tensor) -> torch.Tensor:
+        attn_out, _ = self.cross_attn(query, key_value, key_value)
+        return self.norm(query + self.dropout(attn_out))
+
+class JointSchemaHead(nn.Module):
+    def __init__(self, hidden_size: int = 2048, decision_width: int = 1024, num_layers: int = 4):
+        super().__init__()
+        self.proj = nn.Linear(hidden_size, decision_width)
+        self.routing = EvidenceRoutingBlock(decision_width)
+        encoder_layer = nn.TransformerEncoderLayer(
+            d_model=decision_width,
+            nhead=8,
+            dim_feedforward=decision_width * 4,
+            dropout=0.05,
+            activation="gelu",
+            batch_first=True,
+            norm_first=True,
+        )
+        self.transformer = nn.TransformerEncoder(encoder_layer, num_layers=num_layers)
+        self.scorer = nn.Sequential(
+            nn.Linear(decision_width, decision_width // 2),
+            nn.GELU(),
+            nn.Linear(decision_width // 2, 1),
+        )
+
+    def forward(self, prefix_emb: torch.Tensor, option_embs: torch.Tensor) -> torch.Tensor:
+        p_proj = self.proj(prefix_emb)
+        o_proj = self.proj(option_embs)
+        routed = self.routing(o_proj, p_proj)
+        joint = self.transformer(routed)
+        scores = self.scorer(joint).squeeze(-1)
+        return scores
+
+class KutupForDecision(PreTrainedModel):
+    config_class = KutupConfig
+
+    def __init__(self, config: KutupConfig):
+        super().__init__(config)
+        self.config = config
+        self.head = JointSchemaHead(
+            hidden_size=config.hidden_size,
+            decision_width=config.decision_width
+        )
+
+    @classmethod
+    def from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs):
+        config = kwargs.pop("config", None)
+        if config is None:
+            config = KutupConfig.from_pretrained(pretrained_model_name_or_path, **kwargs)
+        
+        device_map = kwargs.get("device_map", None)
+        torch_dtype = getattr(torch, config.torch_dtype) if isinstance(config.torch_dtype, str) else torch.bfloat16
+        
+        model = cls(config)
+        
+        # 1. Backbone Yükle
+        print(f"[KUTUP-1] Backbone yukleniyor: {config.backbone_model_id}...")
+        backbone = AutoModelForCausalLM.from_pretrained(
+            config.backbone_model_id,
+            torch_dtype=torch_dtype,
+            device_map=device_map,
+        )
+        
+        # 2. LoRA Varsa Yukle
+        lora_dir = os.path.join(pretrained_model_name_or_path, "lora_adapter")
+        if os.path.exists(lora_dir):
+            print(f"[KUTUP-1] LoRA yukleniyor: {lora_dir}...")
+            backbone = PeftModel.from_pretrained(backbone, lora_dir)
+            backbone = backbone.merge_and_unload()
+        model.backbone = backbone
+        
+        # 3. Head safetensors yukle
+        head_path = os.path.join(pretrained_model_name_or_path, "joint_head.safetensors")
+        if os.path.exists(head_path):
+            print(f"[KUTUP-1] Joint Head yukleniyor: {head_path}...")
+            weights = load_file(head_path)
+            model.head.load_state_dict(weights, strict=False)
+            
+        model.to(torch_dtype)
+        model.eval()
+        return model
+
+    @torch.inference_mode()
+    def decide(self, tokenizer: AutoTokenizer, state: str, questions: Dict[str, Any]) -> Dict[str, Any]:
+        device = next(self.parameters()).device
+        answers = {}
+        for qid, qdef in questions.items():
+            qtype = qdef.get("type", "choice")
+            inst = qdef.get("instructions", "")
+            crit = qdef.get("criteria", {})
+            
+            prefix_text = f"[DURUM]
+{state}
+
+[SORU]
+{inst}"
+            p_inputs = tokenizer(prefix_text, return_tensors="pt", truncation=True, max_length=self.config.max_length).to(device)
+            p_out = self.backbone.model(**p_inputs)
+            p_emb = p_out.last_hidden_state
+            
+            opts = []
+            if qtype == "noul":
+                opts = ["Evet", "Hayır"]
+                opt_keys = ["true", "false"]
+            elif isinstance(crit, dict):
+                opt_keys = list(crit.keys())
+                opts = [f"{k}: {v}" for k, v in crit.items()]
+            elif isinstance(crit, list):
+                opt_keys = [str(i) for i in range(len(crit))]
+                opts = [f"Seviye {i}: {v}" for i, v in enumerate(crit)]
+                
+            opt_embs = []
+            for opt in opts:
+                o_inp = tokenizer(f"[SECENEK] {opt}", return_tensors="pt").to(device)
+                o_out = self.backbone.model(**o_inp)
+                opt_embs.append(o_out.last_hidden_state.mean(dim=1))
+                
+            all_opt_emb = torch.stack(opt_embs, dim=1) # [1, num_opts, hidden]
+            logits = self.head(p_emb, all_opt_emb).squeeze(0) / self.config.temperature
+            probs = torch.softmax(logits, dim=-1).tolist()
+            
+            prob_dict = {k: round(p, 4) for k, p in zip(opt_keys, probs)}
+            best_idx = int(torch.argmax(logits).item())
+            best_key = opt_keys[best_idx]
+            conf = round(probs[best_idx], 4)
+            
+            if qtype == "noul":
+                answers[qid] = {"type": "noul", "noul": prob_dict.get("true", 0.5), "confidence": conf}
+            else:
+                answers[qid] = {"choice": best_key, "confidence": conf, "probabilities": prob_dict}
+                
+        return {"model": "TurkishCodeMan/KUTUP-1-Decision-2B", "answers": answers}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523
+size 19989325

tokenizer_config.json

@@ -0,0 +1,32 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}