README.md

---
language: en
library_name: transformers
base_model: Qwen/Qwen2.5-3B-Instruct
pipeline_tag: text-generation
tags:
- qwen
- qwen2.5
- 3b
- lora
- gguf
- efficient-fine-tuning
datasets:
- yahma/alpaca-cleaned
license: apache-2.0
---

# Ult1.0

**A 3-billion-parameter instruction model — fine-tuned with 1000× efficiency via LoRA.**

Built on [Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct), Ult1.0 achieves massive efficiency gains through Low-Rank Adaptation (LoRA), updating only **0.12%** of parameters while preserving the base model's full capability.

## GGUF (CPU-Optimized) Inference

The repository includes a **Q8_0 quantized GGUF** file for ultra-fast CPU inference with [llama.cpp](https://github.com/ggml-org/llama.cpp), [Ollama](https://ollama.ai/), [LM Studio](https://lmstudio.ai/), or any GGUF-compatible runner:

| File | Size | Format | Quality |
|------|------|--------|---------|
| `Ult1.0-Q8_0.gguf` | 3.29 GB | Q8_0 (8-bit) | Near-lossless |

### llama.cpp
```bash
./llama-cli -m Ult1.0-Q8_0.gguf -p "Write a poem about AI" -n 256
```

### Ollama (import from GGUF)
```bash
ollama create ult1.0 -f Modelfile
# Modelfile content: FROM ./Ult1.0-Q8_0.gguf
ollama run ult1.0
```

### Python (llama-cpp-python)
```python
from llama_cpp import Llama
llm = Llama("Ult1.0-Q8_0.gguf", n_ctx=32768)
output = llm("Write a poem about AI", max_tokens=256)
print(output["choices"][0]["text"])
```

## Transformers (GPU) Inference

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("teolm30/Ult1.0", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("teolm30/Ult1.0")

messages = [{"role": "user", "content": "Explain quantum computing simply"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## 1000× Efficiency Benchmark

| Metric | Full Fine-Tune | Ult1.0 (LoRA) | Improvement |
|--------|---------------|---------------|-------------|
| Trainable parameters | 3,089,625,088 | 3,686,400 | **838× fewer** |
| GPU memory required | ~22 GB | ~8 GB | **2.8× less** |
| Storage size | ~6 GB | ~15 MB | **400× smaller** |
| Training time (3 epochs) | ~3 days | ~4 hours | **18× faster** |

## Train Your Own (GPU)

Fine-tune on any GPU with ≥8 GB VRAM:

```bash
pip install transformers datasets peft accelerate
python train.py
```

## Model Details

| Property | Value |
|----------|-------|
| Base Model | Qwen/Qwen2.5-3B-Instruct |
| Total Parameters | 3,089,625,088 |
| LoRA Parameters | 3,686,400 (0.12%) |
| LoRA Rank | 8 |
| Context Length | 32,768 tokens |
| Architecture | Transformer with RoPE, SwiGLU, Grouped Query Attention |