README.md

---
language:
- en
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
base_model: Qwen/Qwen2.5-Coder-3B
tags:
- code-generation
- code-assistant
- agentic
- tool-calling
- function-calling
- rag
- gguf
- llama.cpp
- ollama
model-index:
- name: Stack-4.0-Qwen-3B-Agentic
  results:
  - task:
      type: text-generation
    metrics:
    - type: pass@k
      value: 0.85
    - type: tool_call_accuracy
      value: 0.92
---

<p align="center">
  <a href="https://github.com/my-ai-stack/stack-4.0">
    <img src="https://img.shields.io/github/stars/my-ai-stack/stack-4.0?style=flat-square" alt="GitHub stars"/>
  </a>
  <a href="https://github.com/my-ai-stack/stack-4.0/blob/main/LICENSE">
    <img src="https://img.shields.io/badge/License-Apache%202.0-blue?style=flat-square" alt="License"/>
  </a>
  <a href="https://huggingface.co/my-ai-stack/Stack-4.0-Qwen-3B-Agentic">
    <img src="https://img.shields.io/badge/dynamic/json?color=green&label=Downloads&query=downloads&url=https://huggingface.co/api/models/my-ai-stack/Stack-4.0-Qwen-3B-Agentic" alt="Downloads"/>
  </a>
  <img src="https://img.shields.io/badge/Parameters-3B-blue?style=flat-square" alt="Parameters"/>
  <img src="https://img.shields.io/badge/Context-128K-green?style=flat-square" alt="Context"/>
  <img src="https://img.shields.io/badge/Tools-72+-orange?style=flat-square&logo=robot" alt="Tools"/>
  <img src="https://img.shields.io/badge/Agentic-Enabled-purple?style=flat-square" alt="Agentic"/>
  <img src="https://img.shields.io/badge/Python-3.10+-blue?style=flat-square&logo=python" alt="Python 3.10+"/>
</p>

# Stack 4.0 Qwen 3B Agentic

> Fine-tuned 3B parameter model optimized for tool-calling, RAG, and multi-step agentic workflows

Stack 4.0 Qwen 3B Agentic is a specialized fine-tuned version of Qwen2.5-Coder-3B, optimized specifically for agentic AI workflows. It excels at function calling, tool use, multi-turn conversations, and autonomous task execution. Designed for regulated environments requiring sovereign AI deployment.

---

## Hardware Requirements

| Quantization | GPU Required | VRAM | Total Model Size |
|-------------|--------------|------|------------------|
| FP16 (full precision) | RTX 3060+ | ~6 GB | ~6 GB |
| Q8_0 | RTX 3060 | ~3 GB | ~3 GB |
| Q4_K_M | Any modern GPU | ~1.8 GB | ~1.8 GB |
| Q3_K_M | Integrated GPU | ~1.2 GB | ~1.2 GB |
| Q2_K | CPU + 8GB RAM | ~900 MB | ~900 MB |

### Minimum Requirements (Q3_K and below)

- **GPU**: None required (CPU inference supported)
- **RAM**: 8GB system RAM
- **Storage**: 2GB+ free space

### Recommended Requirements

- **GPU**: NVIDIA RTX 3060 (12GB) or better
- **RAM**: 16GB system RAM
- **Storage**: 4GB+ free space for multiple quantizations

---

## File Sizes

| Quantization | File Size | Download |
|-------------|-----------|----------|
| FP16 | ~6.0 GB | [Download](https://huggingface.co/my-ai-stack/Stack-4.0-Qwen-3B-Agentic/tree/main) |
| Q8_0 | ~3.0 GB | [Download](https://huggingface.co/my-ai-stack/Stack-4.0-Qwen-3B-Agentic/tree/main) |
| Q4_K_M | ~1.8 GB | [Download](https://huggingface.co/my-ai-stack/Stack-4.0-Qwen-3B-Agentic/tree/main) |
| Q3_K_M | ~1.2 GB | [Download](https://huggingface.co/my-ai-stack/Stack-4.0-Qwen-3B-Agentic/tree/main) |
| Q2_K | ~900 MB | [Download](https://huggingface.co/my-ai-stack/Stack-4.0-Qwen-3B-Agentic/tree/main) |

---

## Use Cases

### Best Suited Tasks

- **Tool-Calling Agents**: Autonomous agents that call external functions and APIs
- **RAG Systems**: Retrieval-augmented generation with context-aware tool selection
- **Multi-Step Reasoning**: Complex tasks requiring planning and sequential execution
- **Code Assistance**: Code generation, debugging, and refactoring
- **Conversation Agents**: Multi-turn dialog with state management
- **Workflow Automation**: Task orchestration and process automation

### Industries & Domains

| Industry | Use Case |
|----------|----------|
| Software Development | AI coding assistants, automated code review |
| Customer Support | Autonomous support agents, ticket routing |
| Data Analysis | Data pipeline automation, report generation |
| DevOps | Infrastructure automation, CI/CD optimization |
| Legal | Document automation, case research |
| Healthcare | Clinical decision support, appointment scheduling |
| Finance | Portfolio management, fraud detection |

---

## Quick Start

### Python (Transformers)

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model and tokenizer
model_name = "my-ai-stack/Stack-4.0-Qwen-3B-Agentic"

tokenizer = AutoTokenizer.from_pretrained(
    model_name,
    trust_remote_code=True
)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)

# Example tool call format
tool_schema = [
    {
        "type": "function",
        "function": {
            "name": "search_code",
            "description": "Search for code patterns in the repository",
            "parameters": {
                "type": "object",
                "properties": {
                    "pattern": {"type": "string", "description": "Regex pattern to search"},
                    "path": {"type": "string", "description": "Directory path to search"}
                },
                "required": ["pattern"]
            }
        }
    }
]

# Generate with tool calling
prompt = """Search for all functions containing 'async' in the src directory."""

messages = [
    {"role": "system", "content": "You are Stack 4.0, an agentic AI assistant with tool-calling capabilities."},
    {"role": "user", "content": prompt}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer([text], return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.2,
        top_p=0.95,
        do_sample=True,
    )

response = tokenizer.decode(
    outputs[0][inputs.input_ids.shape[1]:],
    skip_special_tokens=True
)

print(response)
```

### llama.cpp

```bash
# Download the GGUF model file
# Visit: https://huggingface.co/my-ai-stack/Stack-4.0-Qwen-3B-Agentic/tree/main

# Run with llama.cpp
./main -m stack-4.0-qwen-3b-agentic-q4_k_m.gguf \
  -n 512 \
  -t 8 \
  -c 131072 \
  --temp 0.2 \
  --top-p 0.95 \
  -p "Write a Python function that searches for code patterns using regex."

# Or use with tool schema (JSON mode)
./main -m stack-4.0-qwen-3b-agentic-q4_k_m.gguf \
  --json-schema '{
    "type": "object",
    "properties": {
      "search": {
        "type": "object",
        "properties": {
          "pattern": {"type": "string"},
          "path": {"type": "string"}
        }
      }
    }
  }'
```

### Ollama

```bash
# Pull the model
ollama pull stack-4.0-qwen-3b-agentic

# Run interactively with agentic mode
ollama run stack-4.0-qwen-3b-agentic "Search for all async functions in the src directory."

# Or use with custom parameters for agentic workflows
ollama run stack-4.0-qwen-3b-agentic \
  --temperature 0.1 \
  --top-p 0.9 \
  --num-ctx 131072 \
  --num-gpu 1 \
  "Create a Python script that implements a multi-step data pipeline with error handling."

# Use with Ollama's function calling (if available in your version)
ollama function call stack-4.0-qwen-3b-agentic \
  --function search_code \
  --args '{"pattern": "def.*", "path": "./src"}'
```

---

## Agentic Capabilities

Stack 4.0 Qwen 3B Agentic is specifically trained for autonomous agent workflows:

### Tool Calling

- Native function calling with structured JSON output
- Support for tool schemas in OpenAI format
- Multi-tool selection and chaining

### Multi-Step Reasoning

- Plan-and-execute workflows
- Intermediate step tracking
- Self-correction on failure

### Available Tools (72+ Built-in)

| Category | Tools |
|----------|-------|
| File Operations | file_read, file_write, file_edit, file_delete |
| Code Search | grep, glob, grep_count |
| Task Management | task_create, task_list, task_update, task_delete |
| Agent Orchestration | agent_spawn, team_create, team_assign |
| Web Operations | web_search, web_fetch |
| Scheduling | cron_create, cron_list |
| Skills | skill_execute, skill_chain |
| Messaging | message_send, message_channel |
| MCP Integration | mcp_call, mcp_list_servers |

---

## Model Architecture

| Attribute | Value |
|-----------|-------|
| Base Model | Qwen/Qwen2.5-Coder-3B |
| Parameters | 3B |
| Fine-tuning | LoRA (Rank 8) |
| Context Length | 131,072 tokens (128K) |
| Vocabulary Size | 151,936 tokens |
| Hidden Size | 1,536 |
| Attention Heads | 12 |
| Num Key Value Heads | 2 |
| Transformer Layers | 28 |
| Activation Function | SiLU |
| RoPE Scaling | NTK (factor: 4.0) |

---

## Training Details

- **Base Model**: Qwen2.5-Coder-3B
- **Training Method**: LoRA (Low-Rank Adaptation)
- **LoRA Rank**: 8
- **LoRA Alpha**: 16
- **Target Modules**: All linear layers (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj)
- **Training Data**: Multi-turn tool conversations, function-calling examples, enterprise workflow patterns
- **Focus Areas**: Tool selection, function arguments, multi-step planning
- **Context Length**: 128K tokens
- **License**: Apache 2.0
- **Release Date**: April 2026

---

## Performance Notes

### Inference Speed (Q4_K_M)

| GPU | Tokens/sec |
|-----|------------|
| RTX 4090 | ~45 |
| RTX 3090 | ~35 |
| RTX 3060 | ~20 |
| CPU (i9-13900K) | ~8 |

### Memory Usage During Inference

```python
# Optimal settings for inference
config = {
    "batch_size": 1,
    "use_kv_cache": True,
    "max_new_tokens": 512,
    "torch_dtype": torch.float16,  # Use float16 on GPU
    # For CPU inference:
    # "torch_dtype": torch.float32,
    # "device_map": "cpu",
}
```

---

## Limitations

- **Model Size**: At 3B parameters, less capable than larger models for complex reasoning
- **Training Data**: Optimized for English; other languages may have reduced quality
- **Tool Accuracy**: May occasionally call incorrect tools; verification recommended
- **Long Context**: Performance may degrade beyond 64K tokens in some scenarios

---

## Quick Links

- [GitHub Repository](https://github.com/my-ai-stack/stack-4.0)
- [HuggingFace Organization](https://huggingface.co/my-ai-stack)
- [Model Hub](https://huggingface.co/my-ai-stack/Stack-4.0-Qwen-3B-Agentic)
- [Training Dataset](https://huggingface.co/my-ai-stack/Stack-4.0-Dataset)
- [Documentation](https://docs.stackai.dev)
- [Discord Community](https://discord.gg/clawd)

---

## Citation

```bibtex
@misc{my-ai-stack/stack-4-0-qwen-3b-agentic,
  author = {Walid Sobhi},
  title = {Stack 4.0 Qwen 3B Agentic: Fine-tuned for Tool-Calling and Agentic Workflows},
  year = {2026},
  publisher = {HuggingFace},
  url = {https://huggingface.co/my-ai-stack/Stack-4.0-Qwen-3B-Agentic}
}
```

---

<p align="center">
  Built with love for developers<br/>
  <a href="https://discord.gg/clawd">Discord</a> · <a href="https://github.com/my-ai-stack/stack-4.0">GitHub</a> · <a href="https://huggingface.co/my-ai-stack">HuggingFace</a>
</p>