Upload 10 files

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,107 @@
+---
+license: apache-2.0
+language:
+- en
+- multilingual
+base_model:
+- Qwen/Qwen3-0.6B-Base
+pipeline_tag: text-generation
+tags:
+- reasoning
+- code
+- math
+- syngen
+---
+
+# 👋 SYNGEN REASONING 0.6B
+This is a 0.6B parameter LLM designed for synthetic reasoning generation between final model outputs, specifically for dataset modifications, but can be used for multiple use cases which require reasoning.
+
+For example, this model allows you to turn any chat dataset into a reasoning dataset as if it was generated by DeepSeek R1 or Openai's GPT OSS!
+
+# 🤔 HOW TO USE
+
+You should set the following:
+1) ``MIN P = 0.05``
+2) ```TOP P = 0.85```
+3) ```TEMPERATURE = 0.55```
+4) ```MAX_NEW_TOKENS = 16384``` (can be higher or lower, up to 32k)
+
+Syngen uses the CHATML format, but it is not a chat model. This is how you do interference with it:
+```jinja2
+<|im_start|>system
+
+
+THINKING GENERATION ONLY = TRUE
+
+THINKING GENERATION STYLE = (DEEPSEEK_R1 OR GPT_OSS)<|im_end|>
+<|im_start|>user
+Generate thinking for the following:
+
+The user message was: {human_msg}
+
+The assistant response was: {assistant_msg}
+
+Original system prompt was:{system_msg} (for GPT OSS please put "Reasoning effort: high/medium/low")<|im_end|>
+<|im_start|>assistant
+```
+
+
+here are some examples:
+
+## Deepseek r1 possible reasoning
+
+```jinja2
+<|im_start|>system
+
+
+THINKING GENERATION ONLY = TRUE
+
+THINKING GENERATION STYLE = DEEPSEEK_R1<|im_end|>
+<|im_start|>user
+Generate thinking for the following:
+
+The user message was: Your response should contain at least 3 sentences. Your answer must contain exactly 3 bullet points. Use the markdown bullet points such as: * This is point 1. Include keywords [food, water, shelter] in the response.
+
+The assistant response was:* To ensure survival, one must have access to adequate food which provides the necessary nutrients and energy.
+* Water is essential for hydration and maintaining bodily functions, and it is crucial to have a clean and reliable source.
+* Shelter protects individuals from the elements and provides a safe environment, which is fundamental for overall well-being.
+
+Original system prompt was:You are an Ai<|im_end|>
+<|im_start|>assistant
+<generated_thinking_dpsk>
+Okay, the user wants me to generate a response with three bullet points using markdown. The keywords they provided are food, water, and shelter. Let's think about how to structure each bullet point.
+
+First, maybe start with something related to food. Need to make sure it includes all three words. Maybe mention needing food to sustain life or staying hydrated? That covers both water and food indirectly but directly links them together. Then talk about shelter because without protection, you can't eat properly or stay alive. 
+
+Wait, do I need to include water again here? If shelter means avoiding dehydration, then yes. But let's check if the first bullet starts with "To ensure..." that ties everything together. Also, keep it concise but cover all required keywords. Let me draft:
+
+* To ensure survival, one must have access to adequate food which provides the necessary nutrients and energy.
+* Water is essential for hydration and maintaining bodily functions; it’s critical to have a clean and reliable source.
+* Shelter protects individuals from the elements and offers a safe environment—key factors for overall well-being.
+
+That works. Each bullet has two key points (food and water) and shelters. Check keyword usage: food, water, shelter. Yes. Three bullets. Good. Make sure each sentence ends with period. Alright.
+</generated_thinking_dpsk>
+```
+
+## GPT OSS possible reasoning
+```jinja2
+<|im_start|>system
+
+
+THINKING GENERATION ONLY = TRUE
+
+THINKING GENERATION STYLE = GPT_OSS<|im_end|>
+<|im_start|>user
+Generate thinking for the following:
+
+The user message was: Hello!
+
+The assistant response was: Hi! How may I assist you today?
+
+Original system prompt was:You are an Ai
+Reasoning effort: low<|im_end|>
+<|im_start|>assistant
+<generated_thinking_gpt>
+Must produce a simple greeting.
+</generated_thinking_gpt>
+```

added_tokens.json

@@ -0,0 +1,34 @@
+{
+  "</generated_thinking_dpsk>": 151672,
+  "</generated_thinking_gpt>": 151670,
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<generated_thinking_dpsk>": 151671,
+  "<generated_thinking_gpt>": 151669,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652,
+  "DEEPSEEK_R1": 151674,
+  "GPT_OSS": 151673
+}

config.json

@@ -0,0 +1,62 @@
+{
+  "architectures": [
+    "Qwen3ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "eos_token_id": 151643,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 1024,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_types": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 32768,
+  "max_window_layers": 28,
+  "model_type": "qwen3",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 28,
+  "num_key_value_heads": 8,
+  "pad_token_id": 151654,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 1000000,
+  "sliding_window": null,
+  "tie_word_embeddings": true,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.51.3",
+  "unsloth_fixed": true,
+  "unsloth_version": "2025.8.10",
+  "use_cache": true,
+  "use_sliding_window": false,
+  "vocab_size": 151675
+}

generation_config.json

@@ -0,0 +1,8 @@
+{
+  "bos_token_id": 151643,
+  "eos_token_id": 151643,
+  "max_length": 32768,
+  "max_new_tokens": 32768,
+  "pad_token_id": 151654,
+  "transformers_version": "4.51.3"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:216403be9fc9d94faee5fd100d75dc08ee92b0ff2ba8860552bb3f08e7dca13d
+size 1191600568

special_tokens_map.json

@@ -0,0 +1,60 @@
+{
+  "additional_special_tokens": [
+    {
+      "content": "<generated_thinking_gpt>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "</generated_thinking_gpt>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<generated_thinking_dpsk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "</generated_thinking_dpsk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "GPT_OSS",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "DEEPSEEK_R1",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    }
+  ],
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|vision_pad|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a5ad4633a6717f72f3454ce5b54123e3ae64bc6994b1852c0769ec0f8b59cb9
+size 11423834

tokenizer_config.json

@@ -0,0 +1,282 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151669": {
+      "content": "<generated_thinking_gpt>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151670": {
+      "content": "</generated_thinking_gpt>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151671": {
+      "content": "<generated_thinking_dpsk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151672": {
+      "content": "</generated_thinking_dpsk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151673": {
+      "content": "GPT_OSS",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151674": {
+      "content": "DEEPSEEK_R1",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<generated_thinking_gpt>",
+    "</generated_thinking_gpt>",
+    "<generated_thinking_dpsk>",
+    "</generated_thinking_dpsk>",
+    "GPT_OSS",
+    "DEEPSEEK_R1"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "chat_template": "{% for message in messages %}{% if message['role'] == 'system' %}<|im_start|>system\n{{ message['content'] }}<|im_end|>\n{% elif message['role'] == 'user' %}<|im_start|>user\n{{ message['content'] }}<|im_end|>\n{% elif message['role'] == 'assistant' %}<|im_start|>assistant\n{{ message['content'] }}<|im_end|>\n{% endif %}{% endfor %}",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 32768,
+  "pad_token": "<|vision_pad|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}