README.md

---
license: mit
language:
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- text-generation
- small-model
- chat
- fun
- axon
- qwen3
- lora
- not-finetuned
base_model: Qwen/Qwen3-1.7B
model_size:
  base_model: 1.7B
  adapter: 17M
  total: 1.7B
---

# Axon OSS

A 1.7B parameter open-source chat model by Axon Labs. Built as a LoRA adapter on top of Qwen3-1.7B.

> **Note:** This model is NOT fine-tuned for any specific task. It was created via LoRA adaptation and retains the general capabilities of the base Qwen3-1.7B model. It is not a custom-trained model from scratch.

## Model Details

- **Base model:** [Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B)
- **Parameters:** ~1.7B (base) + LoRA adapter (r=16, alpha=32)
- **Architecture:** Qwen3 (transformer decoder) with LoRA adapter targeting all linear projections (q, k, v, o, gate, up, down)
- **LoRA rank:** 16
- **LoRA alpha:** 32
- **Target modules:** q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- **Tokenizer:** Qwen3 tokenizer with ChatML-style formatting (`<|im_start|>` / `<|im_end|>`)
- **Context length:** Up to 32K tokens (base model capability)
- **License:** MIT

## Usage

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-1.7B", torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "axonlabsai/axon-oss")
tokenizer = AutoTokenizer.from_pretrained("axonlabsai/axon-oss")

messages = [{"role": "user", "content": "Hello! What can you do?"}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
output = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(output[0], skip_special_tokens=True))
```

## Limitations

- Not fine-tuned on domain-specific data — general purpose only
- Small model size means limited reasoning depth compared to larger models
- May hallucinate or produce incorrect information
- Not suitable for production deployments without further fine-tuning

## About Axon Labs

Axon Labs builds AI models and tools. This is our open-source contribution — a small, lightweight model for experimentation and chat.