Convert model to bfloat16 and fix total_parameters metadata

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,214 @@
+---
+base_model: Qwen/Qwen2.5-Math-1.5B-Instruct
+library_name: transformers
+model_name: Qwen2.5-Math-1.5B-Instruct-SHARP-Math-PRM
+tags:
+- generated_from_trainer
+- prm
+- trl
+- math
+- process-reward-model
+- qwen2.5
+- sharp
+---
+
+# Model Card for Qwen2.5-Math-1.5B-Instruct-SHARP-Math-PRM
+
+## Introduction
+
+**Qwen2.5-Math-1.5B-Instruct-SHARP-Math-PRM** is a Process Reward Model (PRM) fine-tuned from [Qwen2.5-Math-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Math-1.5B-Instruct). This model is specifically designed to evaluate the correctness of intermediate reasoning steps in mathematical problem-solving processes, enabling more reliable and interpretable mathematical reasoning.
+
+The model has been trained on the **SHARP-Math** dataset using the Process Reward Model methodology, which provides step-by-step feedback on mathematical reasoning chains.
+
+This model is part of the SHARP-PRM series, trained using advanced Process Reward Model techniques.
+
+## Model Information
+
+### Base Model
+- **Base Model**: [Qwen/Qwen2.5-Math-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Math-1.5B-Instruct)
+- **Architecture**: Qwen2ForTokenClassification
+- **Parameters**: 1.5B
+
+### Training Details
+- **Training Dataset**: SHARP-Math (Process Reward Model dataset)
+- **Training Method**: Process Reward Model (PRM) as introduced in [Uesato et al., 2022](https://huggingface.co/papers/2211.14275)
+- **Training Framework**: [TRL (Transformer Reinforcement Learning)](https://github.com/huggingface/trl) v0.24.0
+- **Task Type**: Token Classification (binary classification: error/correct for each reasoning step)
+
+## PRM Evaluation
+
+This model is designed to evaluate mathematical reasoning processes by:
+1. **Step-level Evaluation**: Classifying each step in a reasoning chain as either "correct" or "error"
+2. **Process Feedback**: Providing feedback on the reasoning process, not just the final answer
+3. **Error Detection**: Identifying where mistakes occur in multi-step mathematical solutions
+
+### Evaluation Metrics
+The model is evaluated on the [ProcessBench](https://huggingface.co/datasets/Qwen/ProcessBench) benchmark.
+
+Key metrics include:
+- **Error Accuracy**: Ability to correctly identify incorrect steps
+- **Correct Accuracy**: Ability to correctly identify correct steps
+- **F1 Score**: Balanced measure of error and correct step classification
+
+## Quick Start
+
+### Installation
+
+```bash
+pip install transformers torch
+```
+
+### Basic Usage
+
+#### Using the Model for Step Classification
+
+```python
+from transformers import AutoModelForTokenClassification, AutoTokenizer
+import torch
+import torch.nn.functional as F
+
+model_name = "path/to/Qwen2.5-Math-1.5B-Instruct-SHARP-Math-PRM"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForTokenClassification.from_pretrained(model_name)
+model.eval()
+
+# Example: Evaluate a mathematical reasoning chain
+# Problem with steps (one correct, one incorrect)
+problem = "Solve: 2x + 5 = 13"
+steps = [
+    "Subtract 5 from both sides: 2x = 8",  # Correct step
+    "Divide by 2: x = 5"  # Incorrect step (should be x = 4)
+]
+
+# Format input with step separator
+input_text = problem + "\n\n" + "\n\n".join(steps)
+inputs = tokenizer(input_text, return_tensors="pt", truncation=True, max_length=8192)
+
+# Get model predictions
+with torch.no_grad():
+    outputs = model(**inputs)
+    logits = outputs.logits  # Shape: [batch_size, sequence_length, num_labels]
+    probabilities = F.softmax(logits, dim=-1)  # Convert to probabilities
+    predictions = torch.argmax(logits, dim=-1)  # Get predicted class indices
+
+# Aggregate predictions per step
+# In practice, you would map tokens to steps based on your step separator
+labels = ["error", "correct"]
+for i, step in enumerate(steps):
+    # Get average probability for step tokens (simplified)
+    # In real usage, you'd need to map token positions to step boundaries
+    step_start = len(tokenizer(problem + "\n\n", return_tensors="pt")["input_ids"][0])
+    step_tokens = predictions[0, step_start:step_start+len(tokenizer(step)["input_ids"])]
+    step_label = labels[step_tokens.mode().values.item()] if len(step_tokens) > 0 else "unknown"
+    print(f"\nStep {i+1}: {step}")
+    print(f"  Prediction: {step_label}")
+    print(f"  Confidence: {probabilities[0, step_start, 1].item():.2%}")
+
+# Expected output:
+# Step 1: Subtract 5 from both sides: 2x = 8
+#   Prediction: correct
+#   Confidence: 0.95
+#
+# Step 2: Divide by 2: x = 5
+#   Prediction: error
+#   Confidence: 0.87
+```
+
+**Output Interpretation:**
+
+- **Logits**: Raw scores from the model (before softmax). Higher values indicate stronger confidence.
+- **Probabilities**: Softmax-normalized scores between 0 and 1. Sum to 1 for each token.
+- **Predictions**: Class indices (0 = "error", 1 = "correct") for each token.
+
+#### Using with Pipeline
+
+```python
+from transformers import pipeline
+
+classifier = pipeline(
+    "token-classification",
+    model="path/to/Qwen2.5-Math-1.5B-Instruct-SHARP-Math-PRM",
+    tokenizer="path/to/Qwen2.5-Math-1.5B-Instruct-SHARP-Math-PRM",
+    device=0 if torch.cuda.is_available() else -1
+)
+
+# Classify reasoning steps
+result = classifier(problem + "\n\n" + "\n\n".join(steps))
+```
+
+### Integration with Mathematical Reasoning
+
+This PRM model can be used to:
+1. **Filter incorrect reasoning paths** in tree-of-thought or chain-of-thought generation
+2. **Provide feedback** during step-by-step problem solving
+3. **Evaluate solution quality** before final answer generation
+4. **Improve training** by identifying problematic reasoning patterns
+
+## Training Procedure
+
+### Training Configuration
+
+- **Learning Rate**: 2e-5
+- **Batch Size**: Per-device batch size (with gradient accumulation)
+- **Epochs**: Multiple epochs with early stopping
+- **Optimizer**: AdamW with cosine learning rate schedule
+- **Warmup Ratio**: 3%
+- **Gradient Clipping**: 5.0
+- **Precision**: bfloat16
+- **Gradient Checkpointing**: Enabled for memory efficiency
+
+### Training Framework Versions
+
+- **TRL**: 0.24.0
+- **Transformers**: 4.56.2
+- **PyTorch**: 2.9.1
+- **Datasets**: 4.4.1
+- **Tokenizers**: 0.22.1
+
+### Training Data
+
+The model was trained on the **SHARP-Math** dataset, which contains:
+- Mathematical problems with step-by-step solutions
+- Labeled reasoning steps (correct/error)
+- Diverse mathematical domains and difficulty levels
+
+## Use Cases
+
+### 1. Mathematical Reasoning Evaluation
+- Evaluate intermediate steps in mathematical problem-solving
+- Identify errors in multi-step calculations
+- Provide feedback on reasoning quality
+
+### 2. Educational Applications
+- Automated grading of mathematical solutions
+- Step-by-step feedback for students
+- Identification of common error patterns
+
+### 3. Research Applications
+- Training better mathematical reasoning models
+- Analyzing reasoning patterns
+- Improving chain-of-thought generation
+
+## Limitations and Considerations
+
+1. **Domain Specificity**: This model is specifically trained for mathematical reasoning and may not generalize well to other domains
+2. **Step Length**: The model is optimized for step-level evaluation with a 256-token context per step
+3. **Language**: The model is primarily trained on English mathematical content
+4. **False Positives/Negatives**: Like all classification models, it may misclassify some steps
+
+## Citation
+
+If you use this model in your research, please cite:
+
+```bibtex
+@misc{qwen2.5-math-1.5b-instruct-sharp-math-prm,
+  title={Qwen2.5-Math-1.5B-Instruct-SHARP-Math-PRM: A Process Reward Model for Mathematical Reasoning},
+  author={Your Name/Organization},
+  year={2025},
+  howpublished={\url{https://huggingface.co/path/to/Qwen2.5-Math-1.5B-Instruct-SHARP-Math-PRM}}
+}
+```
+
+**Model Card Version**: 1.0  
+**Last Updated**: 2025-12-30
+

added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'Please reason step by step, and put your final answer within \\boxed{}.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nPlease reason step by step, and put your final answer within \\boxed{}.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

config.json

@@ -0,0 +1,66 @@
+{
+  "architectures": [
+    "Qwen2ForTokenClassification"
+  ],
+  "attention_dropout": 0.0,
+  "dtype": "bfloat16",
+  "eos_token_id": 151645,
+  "hidden_act": "silu",
+  "hidden_size": 1536,
+  "id2label": {
+    "0": "error",
+    "1": "correct"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 8960,
+  "label2id": {
+    "correct": 1,
+    "error": 0
+  },
+  "layer_types": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 4096,
+  "max_window_layers": 21,
+  "model_type": "qwen2",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 28,
+  "num_key_value_heads": 2,
+  "pad_token_id": 151643,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 10000.0,
+  "sliding_window": null,
+  "tie_word_embeddings": true,
+  "transformers_version": "4.56.2",
+  "use_cache": true,
+  "use_sliding_window": false,
+  "vocab_size": 151936
+}

model-00001-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:28ef99fff5c03d1d23d7f87769fe76f0f5bf0c029d6599ebfd5c93e235686091
+size 2498350192

model-00002-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5c994132ac490579cd24770fa36952ce63920c7aeb1767720692ce0b198390f
+size 589123068

model.safetensors.index.json

@@ -0,0 +1,348 @@
+{
+  "metadata": {
+    "total_parameters": 1543736630,
+    "total_size": 3087473260
+  },
+  "weight_map": {
+    "model.embed_tokens.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
+    "model.layers.24.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
+    "model.layers.24.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
+    "model.layers.24.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.k_proj.bias": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.q_proj.bias": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.v_proj.bias": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.norm.weight": "model-00002-of-00002.safetensors",
+    "score.bias": "model-00002-of-00002.safetensors",
+    "score.weight": "model-00002-of-00002.safetensors"
+  }
+}

special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c5ae00e602b8860cbd784ba82a8aa14e8feecec692e7076590d014d7b7fdafa
+size 11421896

tokenizer_config.json

@@ -0,0 +1,207 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

trainer_state.json

@@ -0,0 +1,373 @@
+{
+  "best_global_step": null,
+  "best_metric": 0.9054671281010055,
+  "best_model_checkpoint": null,
+  "epoch": 1.1578947368421053,
+  "eval_steps": 16,
+  "global_step": 88,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.05263157894736842,
+      "grad_norm": 291.3879089355469,
+      "learning_rate": 5e-06,
+      "loss": 4.5409,
+      "step": 4
+    },
+    {
+      "epoch": 0.10526315789473684,
+      "grad_norm": 203.68991088867188,
+      "learning_rate": 1.1666666666666668e-05,
+      "loss": 3.7189,
+      "step": 8
+    },
+    {
+      "epoch": 0.10526315789473684,
+      "eval_F1_err_corr": 0.6871469086338724,
+      "eval_accuracy": 0.7513151602104257,
+      "eval_correct_accuracy": 0.9983222883986244,
+      "eval_error_accuracy": 0.523860477930856,
+      "eval_f1": 0.0057361376673040155,
+      "eval_loss": 1.093567132949829,
+      "eval_pr_auc": 0.3045333732522034,
+      "eval_precision": 0.3,
+      "eval_recall": 0.0028957528957528956,
+      "eval_runtime": 7.2032,
+      "eval_samples_per_second": 162.011,
+      "eval_steps_per_second": 1.388,
+      "step": 8
+    },
+    {
+      "epoch": 0.15789473684210525,
+      "grad_norm": 187.05014038085938,
+      "learning_rate": 1.8333333333333333e-05,
+      "loss": 4.2318,
+      "step": 12
+    },
+    {
+      "epoch": 0.21052631578947367,
+      "grad_norm": 146.46641540527344,
+      "learning_rate": 1.99967206113942e-05,
+      "loss": 2.7212,
+      "step": 16
+    },
+    {
+      "epoch": 0.21052631578947367,
+      "eval_F1_err_corr": 0.6332040138458732,
+      "eval_accuracy": 0.6743185078909613,
+      "eval_correct_accuracy": 0.5579058876929509,
+      "eval_error_accuracy": 0.7319986862791484,
+      "eval_f1": 0.5659655831739961,
+      "eval_loss": 0.7970147132873535,
+      "eval_pr_auc": 0.5909991957581218,
+      "eval_precision": 0.4224548049476689,
+      "eval_recall": 0.8571428571428571,
+      "eval_runtime": 7.138,
+      "eval_samples_per_second": 163.491,
+      "eval_steps_per_second": 1.401,
+      "step": 16
+    },
+    {
+      "epoch": 0.2631578947368421,
+      "grad_norm": 28.303598403930664,
+      "learning_rate": 1.9982149887948264e-05,
+      "loss": 2.6877,
+      "step": 20
+    },
+    {
+      "epoch": 0.3157894736842105,
+      "grad_norm": 63.692806243896484,
+      "learning_rate": 1.995594042425798e-05,
+      "loss": 1.8805,
+      "step": 24
+    },
+    {
+      "epoch": 0.3157894736842105,
+      "eval_F1_err_corr": 0.8195852671712573,
+      "eval_accuracy": 0.8283118125298901,
+      "eval_correct_accuracy": 0.9559562730173418,
+      "eval_error_accuracy": 0.7172645896385392,
+      "eval_f1": 0.5815850815850816,
+      "eval_loss": 0.39416325092315674,
+      "eval_pr_auc": 0.693011369968824,
+      "eval_precision": 0.7338235294117647,
+      "eval_recall": 0.48166023166023164,
+      "eval_runtime": 7.1406,
+      "eval_samples_per_second": 163.433,
+      "eval_steps_per_second": 1.4,
+      "step": 24
+    },
+    {
+      "epoch": 0.3684210526315789,
+      "grad_norm": 53.82140350341797,
+      "learning_rate": 1.99181227793856e-05,
+      "loss": 1.6956,
+      "step": 28
+    },
+    {
+      "epoch": 0.42105263157894735,
+      "grad_norm": 38.888492584228516,
+      "learning_rate": 1.9868741047013382e-05,
+      "loss": 1.5156,
+      "step": 32
+    },
+    {
+      "epoch": 0.42105263157894735,
+      "eval_F1_err_corr": 0.8118976385967145,
+      "eval_accuracy": 0.8354854136776662,
+      "eval_correct_accuracy": 0.9372372100616376,
+      "eval_error_accuracy": 0.71612771037666,
+      "eval_f1": 0.5651074589127687,
+      "eval_loss": 0.36986085772514343,
+      "eval_pr_auc": 0.7616394776557907,
+      "eval_precision": 0.8186813186813187,
+      "eval_recall": 0.4314671814671815,
+      "eval_runtime": 7.1716,
+      "eval_samples_per_second": 162.725,
+      "eval_steps_per_second": 1.394,
+      "step": 32
+    },
+    {
+      "epoch": 0.47368421052631576,
+      "grad_norm": 22.607288360595703,
+      "learning_rate": 1.9807852804032306e-05,
+      "loss": 1.4892,
+      "step": 36
+    },
+    {
+      "epoch": 0.5263157894736842,
+      "grad_norm": 13.400656700134277,
+      "learning_rate": 1.9735529043410012e-05,
+      "loss": 1.2424,
+      "step": 40
+    },
+    {
+      "epoch": 0.5263157894736842,
+      "eval_F1_err_corr": 0.8711215240506508,
+      "eval_accuracy": 0.8627450980392157,
+      "eval_correct_accuracy": 0.9620440825784337,
+      "eval_error_accuracy": 0.7959010995250491,
+      "eval_f1": 0.6835722160970231,
+      "eval_loss": 0.3136674463748932,
+      "eval_pr_auc": 0.7987692721100079,
+      "eval_precision": 0.7969151670951157,
+      "eval_recall": 0.5984555984555985,
+      "eval_runtime": 7.1521,
+      "eval_samples_per_second": 163.17,
+      "eval_steps_per_second": 1.398,
+      "step": 40
+    },
+    {
+      "epoch": 0.5789473684210527,
+      "grad_norm": 16.472789764404297,
+      "learning_rate": 1.9651854091416175e-05,
+      "loss": 1.2995,
+      "step": 44
+    },
+    {
+      "epoch": 0.631578947368421,
+      "grad_norm": 7.745436668395996,
+      "learning_rate": 1.9556925509301844e-05,
+      "loss": 1.1904,
+      "step": 48
+    },
+    {
+      "epoch": 0.631578947368421,
+      "eval_F1_err_corr": 0.8839638940186375,
+      "eval_accuracy": 0.8842659014825442,
+      "eval_correct_accuracy": 0.9484455013462648,
+      "eval_error_accuracy": 0.8276919152969574,
+      "eval_f1": 0.7520491803278688,
+      "eval_loss": 0.2730002999305725,
+      "eval_pr_auc": 0.8327722297335601,
+      "eval_precision": 0.8013100436681223,
+      "eval_recall": 0.7084942084942085,
+      "eval_runtime": 7.1492,
+      "eval_samples_per_second": 163.234,
+      "eval_steps_per_second": 1.399,
+      "step": 48
+    },
+    {
+      "epoch": 0.6842105263157895,
+      "grad_norm": 22.183454513549805,
+      "learning_rate": 1.9450853979547384e-05,
+      "loss": 1.0703,
+      "step": 52
+    },
+    {
+      "epoch": 0.7368421052631579,
+      "grad_norm": 10.418429374694824,
+      "learning_rate": 1.9333763176811663e-05,
+      "loss": 1.1166,
+      "step": 56
+    },
+    {
+      "epoch": 0.7368421052631579,
+      "eval_F1_err_corr": 0.8963518458095878,
+      "eval_accuracy": 0.8854615016738403,
+      "eval_correct_accuracy": 0.9502566249131135,
+      "eval_error_accuracy": 0.8482344335548118,
+      "eval_f1": 0.7603801900950475,
+      "eval_loss": 0.26140278577804565,
+      "eval_pr_auc": 0.8394726873391715,
+      "eval_precision": 0.7892004153686397,
+      "eval_recall": 0.7335907335907336,
+      "eval_runtime": 7.1393,
+      "eval_samples_per_second": 163.461,
+      "eval_steps_per_second": 1.401,
+      "step": 56
+    },
+    {
+      "epoch": 0.7894736842105263,
+      "grad_norm": 24.996362686157227,
+      "learning_rate": 1.9205789623732923e-05,
+      "loss": 1.107,
+      "step": 60
+    },
+    {
+      "epoch": 0.8421052631578947,
+      "grad_norm": 10.622379302978516,
+      "learning_rate": 1.9067082531749496e-05,
+      "loss": 1.0344,
+      "step": 64
+    },
+    {
+      "epoch": 0.8421052631578947,
+      "eval_F1_err_corr": 0.8971597384167123,
+      "eval_accuracy": 0.8928742228598756,
+      "eval_correct_accuracy": 0.9511252134916257,
+      "eval_error_accuracy": 0.8489893075764926,
+      "eval_f1": 0.7764471057884231,
+      "eval_loss": 0.25267964601516724,
+      "eval_pr_auc": 0.8508672146171146,
+      "eval_precision": 0.8037190082644629,
+      "eval_recall": 0.750965250965251,
+      "eval_runtime": 7.1659,
+      "eval_samples_per_second": 162.854,
+      "eval_steps_per_second": 1.395,
+      "step": 64
+    },
+    {
+      "epoch": 0.8947368421052632,
+      "grad_norm": 4.826870441436768,
+      "learning_rate": 1.891780362712594e-05,
+      "loss": 0.984,
+      "step": 68
+    },
+    {
+      "epoch": 0.9473684210526315,
+      "grad_norm": 8.766145706176758,
+      "learning_rate": 1.875812696238745e-05,
+      "loss": 0.9791,
+      "step": 72
+    },
+    {
+      "epoch": 0.9473684210526315,
+      "eval_F1_err_corr": 0.89776260256973,
+      "eval_accuracy": 0.8940698230511717,
+      "eval_correct_accuracy": 0.9191008849787475,
+      "eval_error_accuracy": 0.8773926386058739,
+      "eval_f1": 0.7932804479701353,
+      "eval_loss": 0.25162529945373535,
+      "eval_pr_auc": 0.8580493064471059,
+      "eval_precision": 0.7678410117434508,
+      "eval_recall": 0.8204633204633205,
+      "eval_runtime": 7.1451,
+      "eval_samples_per_second": 163.329,
+      "eval_steps_per_second": 1.4,
+      "step": 72
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 15.486649513244629,
+      "learning_rate": 1.85882387133824e-05,
+      "loss": 1.0899,
+      "step": 76
+    },
+    {
+      "epoch": 1.0526315789473684,
+      "grad_norm": 9.650471687316895,
+      "learning_rate": 1.840833696220963e-05,
+      "loss": 0.61,
+      "step": 80
+    },
+    {
+      "epoch": 1.0526315789473684,
+      "eval_F1_err_corr": 0.8951485557028362,
+      "eval_accuracy": 0.8904830224772836,
+      "eval_correct_accuracy": 0.9052260320962611,
+      "eval_error_accuracy": 0.8852929850041195,
+      "eval_f1": 0.7973451327433628,
+      "eval_loss": 0.2587549686431885,
+      "eval_pr_auc": 0.8644362216080673,
+      "eval_precision": 0.7361111111111112,
+      "eval_recall": 0.8696911196911197,
+      "eval_runtime": 7.1493,
+      "eval_samples_per_second": 163.233,
+      "eval_steps_per_second": 1.399,
+      "step": 80
+    },
+    {
+      "epoch": 1.1052631578947367,
+      "grad_norm": 7.722207546234131,
+      "learning_rate": 1.8218631466263584e-05,
+      "loss": 0.6206,
+      "step": 84
+    },
+    {
+      "epoch": 1.1578947368421053,
+      "grad_norm": 23.517024993896484,
+      "learning_rate": 1.801934341366655e-05,
+      "loss": 0.6597,
+      "step": 88
+    },
+    {
+      "epoch": 1.1578947368421053,
+      "eval_F1_err_corr": 0.9054671281010055,
+      "eval_accuracy": 0.9033955045432808,
+      "eval_correct_accuracy": 0.9527881095438347,
+      "eval_error_accuracy": 0.8626242087113936,
+      "eval_f1": 0.7973921765295887,
+      "eval_loss": 0.25677669048309326,
+      "eval_pr_auc": 0.8667069010917992,
+      "eval_precision": 0.8298538622129437,
+      "eval_recall": 0.7673745173745173,
+      "eval_runtime": 7.1469,
+      "eval_samples_per_second": 163.288,
+      "eval_steps_per_second": 1.399,
+      "step": 88
+    }
+  ],
+  "logging_steps": 4,
+  "max_steps": 380,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 5,
+  "save_steps": 16,
+  "stateful_callbacks": {
+    "MinEpochEarlyStoppingCallback": {
+      "args": {
+        "early_stopping_patience": 5,
+        "early_stopping_threshold": 0.001
+      },
+      "attributes": {
+        "early_stopping_patience_counter": 0
+      }
+    },
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.358268710518784e+16,
+  "train_batch_size": 32,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1d4afbd41965d045713a70f37b0ad353967d5f7aa355c47c384f1ee2c0d6b31e
+size 6097