File size: 3,161 Bytes
10bd2da 3ffb61f 1d17394 10bd2da 1d17394 676b8cf 1d17394 676b8cf 1d17394 676b8cf 1d17394 676b8cf 1d17394 676b8cf 1d17394 676b8cf 1d17394 676b8cf 1d17394 676b8cf 1d17394 676b8cf 1d17394 676b8cf 0d2c2bf 676b8cf 1d17394 676b8cf 1d17394 676b8cf 1d17394 676b8cf 6a00cf7 676b8cf |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 |
---
library_name: transformers
license: apache-2.0
language:
- pt
- en
base_model:
- unsloth/Qwen3-4B-Base
pipeline_tag: text-generation
datasets:
- nvidia/OpenMathReasoning
---
# 🧠 DogeAI-v2.0-4B-Reasoning
**"The Small Model That Thinks Big."**
DogeAI-v2.0-4B-Reasoning is a high-efficiency model optimized for **Chain-of-Thought (CoT)**. Built by [AxionLab-Co](https://huggingface.co), it merges a specialized reasoning LoRA onto the powerful **Qwen3-4B-Base** architecture, delivering structured, step-by-step analytical capabilities in a compact 4B footprint.
### 🚀 Key Highlights
- **Architecture:** Decoder-only Transformer (Qwen3 Base).
- **Core Strength:** Multi-step logical reasoning and structured problem solving.
- **Hardware Friendly:** Optimized for local inference (Low VRAM usage).
- **Final Merge:** No LoRA dependency; ready for production or GGUF conversion.
---
## 🎯 Use Cases
- **Complex Problem Solving:** Math, logic, and analytical tasks.
- **Detailed Explanations:** When you need the "why" and "how", not just the "what".
- **Local Agents:** High-performance reasoning for edge devices and local LLM setups.
---
## 🛠️ Quick Start
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "AxionLab-Co/DogeAI-v2.0-4B-Reasoning"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto",
torch_dtype=torch.bfloat16 # Recommended for Qwen3
)
prompt = "Solve this step-by-step: If a train leaves at 2 PM at 60mph, and another..."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=512,
temperature=0.3, # Lower temp recommended for reasoning
do_sample=True
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
🏋️ Training & Methodology
Our goal at AxionLab was to prioritize Depth of Thought over mere textual fluency.
Dataset: A curated mix of synthetic CoT datasets and manually pre-processed logical reasoning prompts.
Fine-tuning: Performed on Kaggle GPUs using PEFT (LoRA) with a focus on preserving the base model's knowledge while injecting structured logic.
Optimization: Mixed precision (fp16) with a final merge_and_unload for seamless deployment.
📊 Evaluation Results
In qualitative testing, DogeAI-v2.0-4B shows:
Higher Logical Consistency compared to the stock Qwen3-4B-Base.
Reduced Hallucination in multi-step word problems.
Structured Verbosity: It "thinks" before it answers.
⚠️ Limitations & Bias
Reasoning Loops: The model might occasionally over-explain simple tasks.
Safety: No specific safety RLHF has been applied. Use with external safety guardrails in production.
Factuality: While logic is improved, it can still hallucinate complex facts.
🤝 Contact & Collaboration
Developed with ❤️ by AxionLab-Co.
We are an independent, community-driven lab focused on efficient AI.
Organization: AxionLab-official
Feedback: Open a Discussion on this repo!
Language Support: Primarily English. Portuguese support is available but may vary in reasoning depth.
|