AxionLab-Co
/

DogeAI-v2.0-4B-Reasoning

@@ -10,422 +10,73 @@ pipeline_tag: text-generation
 datasets:
 - nvidia/OpenMathReasoning
 ---
-🧠 DogeAI-v2.0-4B-Reasoning
-📌 Model Details
-Model Description
-DogeAI-v2.0-4B-Reasoning is a language model focused on reasoning, structured thinking, and analytical responses, created from merging a reasoning LoRA onto the Qwen3-4B-Base model.
-The main objective of this model is to improve logical coherence, the ability to solve problems in multiple steps, and explanatory clarity, without drastically altering the overall behavior of the base model.
-This model represents the merged and final version, and can be used without dependence on external LoRA.
-Developed by: AxionLab-Co
-Funded by: Independent / Community-driven
-Shared by: AxionLab-Co
-Model type: Decoder-only Transformer (Causal Language Model)
-Language(s) (NLP): Primarily English
-License: Apache 2.0 (inherits from base model)
-Finetuned from model: Qwen3-4B-Base
-🔗 Model Sources
-Repository: Hugging Face – AxionLab-Co/DogeAI-v2.0-4B-Reasoning
-Base Model: Qwen/Qwen3-4B-Base
-Training Platform: Kaggle
-Frameworks: PyTorch, Transformers, PEFT
-🎯 Uses
-Direct Use
-This model can be used directly for:
-Logical and analytical reasoning
-Multi-step problem solving
-Detailed explanations (“Thinking-Style Responses”)
-AI Research, Experimentation, and Learning
-Downstream Use
-Conversational agents focused on reasoning
-Additional fine-tuning in specific domains
-Conversion to GGUF and use in engines like llama.cpp
-Academic or experimental research
-Out-of-Scope Use
-This model is not recommended for:
-Medical, legal, or financial decisions
-Critical safety applications
-Use where absolute factuality is mandatory
-⚠️ Bias, Risks, and Limitations
-May generate excessive reasoning chains, even when unnecessary
-Inherited potential biases from the base model and training data
-Has not undergone specific alignment or safety fine-tuning
-Generated reasoning is not guaranteed to be correct
-Recommendations
-Users should:
-Critically evaluate responses
-Use additional layers of security in production
-Avoid blindly trusting chains of reasoning
-🚀 How to Get Started with the Model
-'' from transformers import AutoModelForCausalLM, AutoTokenizer
-model = AutoModelForCausalLM.from_pretrained( "AxionLab-Co/DogeAI-v2.0-4B-Reasoning", device_map="auto", torch_dtype="auto" )
-tokenizer = AutoTokenizer.from_pretrained( "AxionLab-Co/DogeAI-v2.0-4B-Reasoning" )
-inputs = tokenizer("Solve this step by step:", return_tensors="pt").to(model.device) outputs = model.generate(**inputs, max_new_tokens=256)
-print(tokenizer.decode(outputs[0], skip_special_tokens=True)) ''
-🏋️Training Details
-Training Data
-The model was fitted using datasets focused on reasoning and chain-of-thought, containing:
-Step-by-step problem solving
-Structured explanatory responses
-Synthetic and curated analytical prompts
-The data were manually pre-processed to improve quality and consistency.
-Training Procedure Preprocessing
-Tokenization with Qwen's original tokenizer
-Filtering of inconsistent or low-quality examples
-Training Hyperparameters
-Training regime: fp16 mixed precision
-Fine-tuning method: LoRA (PEFT)
-Optimizer: AdamW
-Framework: Transformers + PEFT
-Speeds, Sizes, Times
-Training performed on Kaggle GPU
-LoRA intentionally kept lightweight
-Final merge performed via PEFT (merge_and_unload)
-📊 Evaluation
-Testing Data, Factors & Metrics Testing Data
-Manual reasoning prompts
-Direct comparison with the base model
-Factors
-Clarity of reasoning
-Logical coherence
-Tendency to hallucination
-Metrics
-Qualitative human evaluation
-Subjective comparison of responses
-Results
-The model demonstrates better logical organization and more concise explanations Consistent in direct comparison with Qwen3-4B-Base.
-Summary
-DogeAI-v2.0-4B-Reasoning prioritizes quality of thought, not just textual fluency.
-🌱 Environmental Impact
-Hardware Type: NVIDIA GPU (Kaggle)
-Hours used: Few hours (single-session fine-tuning + merge)
-Cloud Provider: Kaggle
-Compute Region: Unknown
-Carbon Emitted: Not measured
-⚙️ Technical Specifications
-Model Architecture and Objective
-Decoder-only Transformer
-Objective: Improve reasoning via efficient fine-tuning
-Compute Infrastructure Hardware
-NVIDIA GPU (Kaggle environment)
-Software
-PyTorch
-Transformers
-PEFT 0.18.1
-📚 Citation
-If you use this model in research or derivative projects, please cite the base model and this repository.
-👥 Model Card Authors
-AxionLab-Co
-📬 Model Card Contact
-For questions, feedback, or collaboration: AxionLab-Co – Hugging Face
-# --FOR PORTUGUESE READERS --
 # 🧠 DogeAI-v2.0-4B-Reasoning
-# 📌 Model Details
-**Model Description**
-DogeAI-v2.0-4B-Reasoning é um modelo de linguagem focado em raciocínio, pensamento estruturado e respostas analíticas, criado a partir do merge de uma LoRA de reasoning sobre o modelo base Qwen3-4B-Base.
-O objetivo principal deste modelo é melhorar a coerência lógica, a capacidade de resolver problemas em múltiplos passos e a clareza explicativa, sem alterar drasticamente o comportamento geral do modelo base.
-Este modelo representa a versão merged e final, podendo ser utilizado sem dependência de LoRA externa.
-Developed by: AxionLab-Co
-Funded by: Independent / Community-driven
-Shared by: AxionLab-Co
-Model type: Decoder-only Transformer (Causal Language Model)
-Language(s) (NLP): Primarily English
-License: Apache 2.0 (inherits from base model)
-Finetuned from model: Qwen3-4B-Base
-# 🔗 Model Sources
-Repository: Hugging Face – AxionLab-Co/DogeAI-v2.0-4B-Reasoning
-Base Model: Qwen/Qwen3-4B-Base
-Training Platform: Kaggle
-Frameworks: PyTorch, Transformers, PEFT
-# 🎯 Uses
-# Direct Use
-Este modelo pode ser utilizado diretamente para:
-Raciocínio lógico e analítico
-Resolução de problemas em múltiplos passos
-Explicações detalhadas (“thinking-style responses”)
-Pesquisa, experimentação e aprendizado em IA
-Downstream Use
-Conversational agents focados em reasoning
-Fine-tuning adicional em domínios específicos
-Conversão para GGUF e uso em engines como llama.cpp
-Pesquisa acadêmica ou experimental
-Out-of-Scope Use
-Este modelo não é recomendado para:
-Decisões médicas, legais ou financeiras
-Aplicações críticas de segurança
-Uso onde factualidade absoluta é obrigatória
-# ⚠️ Bias, Risks, and Limitations
-Pode gerar cadeias de raciocínio excessivas, mesmo quando não necessárias
-Herdou possíveis vieses do modelo base e dos dados de treino
-Não passou por fine-tuning específico de alinhamento ou safety
-Raciocínios gerados não são garantidamente corretos
-Recommendations
-Usuários devem:
-Avaliar criticamente as respostas
-Utilizar camadas adicionais de segurança em produção
-Evitar confiar cegamente em cadeias de raciocínio
-# 🚀 How to Get Started with the Model
-'' from transformers import AutoModelForCausalLM, AutoTokenizer
 model = AutoModelForCausalLM.from_pretrained(
-    "AxionLab-Co/DogeAI-v2.0-4B-Reasoning",
-    device_map="auto",
-    torch_dtype="auto"
 )
-tokenizer = AutoTokenizer.from_pretrained(
-    "AxionLab-Co/DogeAI-v2.0-4B-Reasoning"
 )
-inputs = tokenizer("Solve this step by step:", return_tensors="pt").to(model.device)
-outputs = model.generate(**inputs, max_new_tokens=256)
-print(tokenizer.decode(outputs[0], skip_special_tokens=True)) ''
-# 🏋️ Training Details
-Training Data
-O modelo foi ajustado utilizando datasets focados em reasoning e chain-of-thought, contendo:
-Resolução passo a passo de problemas
-Respostas explicativas estruturadas
-Prompts analíticos sintéticos e curados
-Os dados foram pré-processados manualmente para melhorar qualidade e consistência.
-Training Procedure
-Preprocessing
-Tokenização com tokenizer original do Qwen
-Filtragem de exemplos inconsistentes ou de baixa qualidade
-Training Hyperparameters
-Training regime: fp16 mixed precision
-Fine-tuning method: LoRA (PEFT)
-Optimizer: AdamW
-Framework: Transformers + PEFT
-Speeds, Sizes, Times
-Treinamento realizado em GPU do Kaggle
-LoRA mantida propositalmente leve
-Merge final realizado via PEFT (merge_and_unload)
-# 📊 Evaluation
-Testing Data, Factors & Metrics
-Testing Data
-Prompts manuais de reasoning
-Comparação direta com o modelo base
-Factors
-Clareza do raciocínio
-Coerência lógica
-Tendência a alucinação
-Metrics
-Avaliação qualitativa humana
-Comparação subjetiva de respostas
-Results
-O modelo demonstra melhor organização lógica e explicações mais consistentes em comparação direta com o Qwen3-4B-Base.
-Summary
-DogeAI-v2.0-4B-Reasoning prioriza qualidade de pensamento, não apenas fluência textual.
-# 🌱 Environmental Impact
-Hardware Type: NVIDIA GPU (Kaggle)
-Hours used: Few hours (single-session fine-tuning + merge)
-Cloud Provider: Kaggle
-Compute Region: Unknown
-Carbon Emitted: Not measured
-# ⚙️ Technical Specifications
-# Model Architecture and Objective
-Decoder-only Transformer
-Objetivo: melhorar raciocínio via fine-tuning eficiente
-Compute Infrastructure
-Hardware
-NVIDIA GPU (Kaggle environment)
-Software
-PyTorch
-Transformers
-PEFT 0.18.1
-# 📚 Citation
-Se você utilizar este modelo em pesquisas ou projetos derivados, cite o modelo base e este repositório.
-# 👥 Model Card Authors
-AxionLab-Co
-# 📬 Model Card Contact
-Para dúvidas, feedback ou colaboração:
-AxionLab-Co – Hugging Face

 datasets:
 - nvidia/OpenMathReasoning
 ---
 # 🧠 DogeAI-v2.0-4B-Reasoning
+**"The Small Model That Thinks Big."**
+DogeAI-v2.0-4B-Reasoning is a high-efficiency model optimized for **Chain-of-Thought (CoT)**. Built by [AxionLab-Co](https://huggingface.co), it merges a specialized reasoning LoRA onto the powerful **Qwen3-4B-Base** architecture, delivering structured, step-by-step analytical capabilities in a compact 4B footprint.
+### 🚀 Key Highlights
+- **Architecture:** Decoder-only Transformer (Qwen3 Base).
+- **Core Strength:** Multi-step logical reasoning and structured problem solving.
+- **Hardware Friendly:** Optimized for local inference (Low VRAM usage).
+- **Final Merge:** No LoRA dependency; ready for production or GGUF conversion.
+---
+## 🎯 Use Cases
+- **Complex Problem Solving:** Math, logic, and analytical tasks.
+- **Detailed Explanations:** When you need the "why" and "how", not just the "what".
+- **Local Agents:** High-performance reasoning for edge devices and local LLM setups.
+---
+## 🛠️ Quick Start
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+model_id = "AxionLab-Co/DogeAI-v2.0-4B-Reasoning"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
 model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    device_map="auto",
+    torch_dtype=torch.bfloat16 # Recommended for Qwen3
 )
+prompt = "Solve this step-by-step: If a train leaves at 2 PM at 60mph, and another..."
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=512,
+    temperature=0.3, # Lower temp recommended for reasoning
+    do_sample=True
 )
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+🏋️ Training & Methodology
+Our goal at AxionLab was to prioritize Depth of Thought over mere textual fluency.
+Dataset: A curated mix of synthetic CoT datasets and manually pre-processed logical reasoning prompts.
+Fine-tuning: Performed on Kaggle GPUs using PEFT (LoRA) with a focus on preserving the base model's knowledge while injecting structured logic.
+Optimization: Mixed precision (fp16) with a final merge_and_unload for seamless deployment.
+📊 Evaluation Results
+In qualitative testing, DogeAI-v2.0-4B shows:
+Higher Logical Consistency compared to the stock Qwen3-4B-Base.
+Reduced Hallucination in multi-step word problems.
+Structured Verbosity: It "thinks" before it answers.
+⚠️ Limitations & Bias
+Reasoning Loops: The model might occasionally over-explain simple tasks.
+Safety: No specific safety RLHF has been applied. Use with external safety guardrails in production.
+Factuality: While logic is improved, it can still hallucinate complex facts.
+🤝 Contact & Collaboration
+Developed with ❤️ by AxionLab-Co.
+We are an independent, community-driven lab focused on efficient AI.
+Organization: AxionLab-official
+Feedback: Open a Discussion on this repo!
+Language Support: Primarily English. Portuguese support is available but may vary in reasoning depth.