| --- |
| base_model: LiquidAI/LFM2.5-8B-A1B |
| library_name: transformers |
| model_name: Tini-8B-A1B |
| tags: |
| - generated_from_trainer |
| - sft |
| - unsloth |
| - trl |
| - reasoning |
| - agentic |
| - function-calling |
| licence: mit |
| pipeline_tag: text-generation |
| dataset: |
| - nohurry/Opus-4.6-Reasoning-3000x-filtered |
| - Jackrong/DeepSeek-V4-Distill-8000x |
| - Jackrong/Qwen3.5-reasoning-700x |
| - NousResearch/hermes-function-calling-v1 |
| --- |
| |
| # Tini-8B-A1B |
|
|
| <p align="center"> |
| <img src="thumbnail.png" width="600" alt="Tini-8B-A1B Logo"> |
| </p> |
|
|
| Tini-8B-A1B is a fine-tuned version of the hybrid model architecture LiquidAI/LFM2.5-8B-A1B. This model is optimized for Agentic Reasoning, seamlessly combining deep chain-of-thought (CoT), native system function calling capabilities. |
|
|
| --- |
|
|
| ## 📊 Dataset Mixture |
|
|
| The model was Supervised Fine-Tuned (SFT) on a curated mixture of samples balancing deep reasoning and function-calling actions: |
|
|
| | Dataset | Category | |
| | :--- | :--- | |
| | `nohurry/Opus-4.6-Reasoning-3000x-filtered` | Advanced Reasoning | |
| | `Jackrong/DeepSeek-V4-Distill-8000x` | Reasoning / Math / Code | |
| | `Jackrong/Qwen3.5-reasoning-700x` | Logic / Hard Math | |
| | `NousResearch/hermes-function-calling-v1` | Tool Use / Agentic | |
|
|
| --- |
|
|
| ## 🛠️ Training Techniques |
|
|
| To preserve the model's core capabilities while focusing gradient updates entirely on reasoning tracks, the following configurations were applied: |
|
|
| - **Train on Response Only** |
| - **LoRA Target Modules** |
|
|
| --- |
|
|
| ## 🏃♂️ Quick Start & Inference Parameters Guide |
|
|
| ### 💡 Recommended Decoding Parameters |
| * **General & Contextual Reasoning (Riddles, Nuances, Analysis):** |
| `temperature: 0.6` | `top_p: 0.95` | `top_k: 50` | `repetition_penalty: 1.10` |
|
|
| * **Mathematics & Technical Coding Tasks:** |
| `temperature: 0.35` | `top_p: 0.90` | `top_k: 40` | `repetition_penalty: 1.08` |
|
|
| ### 🚀 Python Example Script |
| ```python |
| import torch |
| from unsloth import FastLanguageModel |
| from transformers import TextStreamer |
| |
| MODEL_PATH = "./Tini-8B-A1B" |
| |
| # 1. Load model with 4-bit quantization for VRAM efficiency |
| model, tokenizer = FastLanguageModel.from_pretrained( |
| model_name = MODEL_PATH, |
| max_seq_length = 2048, |
| dtype = torch.bfloat16, |
| load_in_4bit = True, |
| trust_remote_code = True |
| ) |
| FastLanguageModel.for_inference(model) |
| |
| # 2. Set system prompt forcing Vietnamese internal monologue |
| messages = [ |
| { |
| "role": "system", |
| "content": "Bạn là một trợ lý AI thông minh. BẮT BUỘC phải thực hiện toàn bộ chuỗi suy luận trong thẻ <think> bằng TIẾNG VIỆT để bảo toàn ngữ cảnh văn hóa và tiết kiệm token." |
| }, |
| { |
| "role": "user", |
| "content": "Một bể nước đang cạn hoàn toàn. Nếu mở riêng vòi A đầy sau 4 giờ. Mở riêng vòi B (vòi xả) cạn sau 6 giờ. Hỏi nếu mở cả hai vòi cùng lúc thì sau bao lâu đầy được 75% bể?" |
| } |
| ] |
| |
| inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda") |
| text_streamer = TextStreamer(tokenizer, skip_prompt=True) |
| |
| # 3. Generate with streaming output and a 2048 max token limit |
| with torch.no_grad(): |
| _ = model.generate( |
| input_ids = inputs, |
| streamer = text_streamer, |
| max_new_tokens = 2048, |
| use_cache = True, |
| temperature = 0.6, |
| top_p = 0.95, |
| top_k = 50, |
| repetition_penalty = 1.10 |
| ) |
| ``` |
