---
license: apache-2.0
base_model: Qwen/Qwen3-0.6B
tags:
  - qwen3
  - function-calling
  - creative-writing
  - code-generation
  - math-reasoning
  - dare-ties
  - merge
language:
  - en
pipeline_tag: text-generation
---

# Cclilqwen

A Qwen3-0.6B model fine-tuned for **creative writing**, **code generation**, and **agentic tool use** while retaining strong math reasoning.

Built by training four specialist models and merging them via manual DARE-TIES into a single capable 0.6B parameter model.

## Performance

| Capability | Result |
|-----------|--------|
| **GSM8K Math** | 48.5% (nearly 2x base Qwen3-0.6B) |
| **Tool Calling** | 100% success rate (valid JSON, correct `<tool_call>` tags) |
| **Python Coding** | Correct solutions for palindrome, fibonacci, stack, filter, word frequency |
| **Creative Writing** | Vivid prose, gothic horror, dark humor, perspective shifts |

## How It Was Made

Four specialist LoRA fine-tunes (r=32) on top of a math-strong base, then merged:

1. **selfplay_v1** (weight 0.30) — Self-play rejection sampling on GSM8K, 49% accuracy
2. **creative_v5** (weight 0.30) — 28 hand-crafted golden exemplars, best creative quality
3. **coding_v1** (weight 0.20) — CodeAlpaca-20k filtered for short Python solutions
4. **tool_use_v2** (weight 0.20) — glaive-function-calling-v2, Hermes-style `<tool_call>` format

Merged using manual DARE-TIES (DROP=0.80, TRIM=0.20, SCALE=0.5), which significantly outperformed mergekit's implementation on this model.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Asystemoffields/disco-torch-v1", torch_dtype="auto")
tokenizer = AutoTokenizer.from_pretrained("Asystemoffields/disco-torch-v1")

messages = [{"role": "user", "content": "Write a haiku about debugging code."}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=200, temperature=0.7, do_sample=True)
print(tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
```

### Tool Calling

```python
import json

tools = [{"name": "get_weather", "description": "Get weather for a location",
          "parameters": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}}]

messages = [
    {"role": "system", "content": f"You have access to: {json.dumps(tools)}\nCall tools with: <tool_call>\n{{\"name\": \"...\", \"arguments\": {{...}}}}\n</tool_call>"},
    {"role": "user", "content": "What's the weather in Tokyo?"},
]
```

## Training Details

- **Base model:** Qwen/Qwen3-0.6B (0.6B parameters)
- **Hardware:** NVIDIA A10G (Modal)
- **LoRA:** r=32, alpha=64, all attention + MLP projections
- **Merge:** Manual DARE-TIES with 4 specialist checkpoints
- **Training data:** GSM8K, CodeAlpaca-20k, glaive-function-calling-v2, 28 golden creative exemplars

## Limitations

- 0.6B parameters limits complex multi-step reasoning
- Creative writing quality varies — best with specific, constrained prompts
- Tool calling works with explicit system prompt instructions
- Math accuracy (48.5%) is strong for size but not competitive with larger models

## License

Apache 2.0 (same as Qwen3-0.6B)