docs/usage_guide.md

# Sixpert K2 - Complete Usage Guide

## Quick Start

### Option 1: Ollama (Easiest)

```bash
# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Create and run
ollama create sixpert-k2 -f OllamaModelfile

# Chat
ollama run sixpert-k2
```

### Option 2: llama-cpp-python (Python)

```bash
pip install llama-cpp-python
python examples/generate.py --prompt "Explain quantum computing in depth"
```

### Option 3: API Server

```bash
pip install llama-cpp-python
python examples/api_server.py --model SixpertK2.gguf
```

### Option 4: LM Studio

1. Download LM Studio from https://lmstudio.ai
2. Import `SixpertK2.gguf`
3. Chat with the Sixpert K2 preset

## Chat Format

Sixpert K2 uses the following chat template:

```
<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
What is quantum computing?<|im_end|>
<|im_start|>assistant
Quantum computing uses quantum mechanical phenomena...<|im_end|>
```

## Recommended Settings

| Parameter | Value | Notes |
|---|---|---|
| temperature | 0.6 | Slightly lower for reasoning tasks |
| top_p | 0.85 | Nucleus sampling |
| top_k | 50 | Limit token selection |
| repeat_penalty | 1.08 | Prevent repetition |
| max_tokens | 16384 | Extended output for deep reasoning |
| context_size | 131072 | Full context window |

## MoE-Specific Tips

### When to Use K2 vs K1

| Task Type | Best Model | Reason |
|---|---|---|
| Fast responses | K1 (Dense) | Predictable latency |
| Deep reasoning | K2 (MoE) | Specialized reasoning experts |
| Long documents | K2 (MoE) | Better long-context handling |
| Code generation | Either | K2 slightly better |
| Math proofs | K2 (MoE) | Math expert specialization |
| Simple Q&A | K1 (Dense) | Faster, sufficient quality |
| Agentic tasks | K2 (MoE) | Better tool orchestration |
| Vision tasks | Either | Both support multimodal |

### Long-Context Usage

K2 is optimized for long-context understanding. For documents exceeding 32K tokens:

```python
llm = Llama(
    model_path="SixpertK2.gguf",
    n_ctx=131072,  # Full context window
    n_gpu_layers=-1,
)

# Feed entire documents
response = llm.create_chat_completion(
    messages=[{
        "role": "user",
        "content": f"Based on the following document, answer my question:\n\n{full_document}\n\nQuestion: What are the key findings?"
    }],
    max_tokens=4096,
)
```

## Function Calling

See `examples/function_calling.py` for a complete agentic implementation.

## Vision / Multimodal

See `examples/vision_example.py` for image analysis examples.

## Integration Examples

### OpenAI-Compatible Client

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")

response = client.chat.completions.create(
    model="sixpert-k2",
    messages=[{"role": "user", "content": "Prove that sqrt(2) is irrational"}],
    temperature=0.6,
    max_tokens=4096,
)
print(response.choices[0].message.content)
```

### LangChain Integration

```python
from langchain.llms import LlamaCpp

llm = LlamaCpp(
    model_path="SixpertK2.gguf",
    temperature=0.6,
    n_ctx=131072,
    n_gpu_layers=-1,
)

result = llm.invoke("Explain the theory of relativity in detail")
print(result)
```

### AutoGen Integration

```python
from autogen import AssistantAgent

assistant = AssistantAgent(
    name="sixpert_k2",
    llm_config={"config_list": [{"model": "sixpert-k2", "base_url": "http://localhost:8000/v1", "api_key": "not-needed"}]},
    system_message="You are Sixpert K2, a deep reasoning engine.",
)
```

## Performance Tips

1. **GPU Offloading**: Set `n_gpu_layers=-1` for full GPU offload (fits in 8GB VRAM)
2. **Temperature**: Use 0.3-0.5 for mathematical proofs, 0.6-0.7 for creative tasks
3. **Context Size**: Start with 32768, increase to 131072 for long documents
4. **Batch Inference**: Use the API server for batch processing
5. **Quantization**: Q4_K_M is recommended; Q6_K for higher quality

## Troubleshooting

| Issue | Solution |
|---|---|
| Slow generation | Ensure GPU offloading is enabled |
| Out of memory | Reduce context size to 32768 or 8192 |
| Repetitive output | Increase `repeat_penalty` to 1.1-1.15 |
| Shallow reasoning | Lower temperature to 0.3-0.5 |
| Long response needed | Set `max_tokens` to 8192 or 16384 |
| Context overflow | Use 4096 context for testing |