README.md

---
license: mit
base_model: artindnr/tea
tags:
- fine-tuned
- full-fine-tune
- 4-bit
- bitsandbytes
- unsloth
- safetensors
- text-generation
- chat
- question-answering
- assistant
language:
  - fa
  - en
  - multilingual
pipeline_tag: text-generation
---

# 🍵 Tea — 4-bit
![https://64.media.tumblr.com/1ab2cfe03429ef47b5063c90df2c0a5a/3c1a235d6f992b74-7d/s500x750/aafa42b7949f3104f349e5508df5fa8b738d879a.gif](https://64.media.tumblr.com/1ab2cfe03429ef47b5063c90df2c0a5a/3c1a235d6f992b74-7d/s500x750/aafa42b7949f3104f349e5508df5fa8b738d879a.gif)

This repo contains a **4-bit precision version of [`artindnr/tea`](https://huggingface.co/artindnr/tea)**, a full fine-tune of [`microsoft/phi-4`](https://huggingface.co/microsoft/phi-4) for question answering and long, multi-turn assistant conversations, with fine-tuning focused on Farsi (Persian) conversational ability.

Unlike GGUF quants, this is a **safetensors** checkpoint quantized to 4-bit precision with [Unsloth](https://github.com/unslothai/unsloth) (bitsandbytes nf4 backend), meant to be loaded directly with 🤗 Transformers or Unsloth — not with llama.cpp.

> Looking for GGUF quants for llama.cpp / Ollama / LM Studio instead? See [artindnr/tea-gguf](https://huggingface.co/artindnr/tea-gguf) (F16, Q8_0, Q5_K_M, Q4_K_M).

## Model Details

- **Base model:** [artindnr/tea](https://huggingface.co/artindnr/tea) (full fine-tune of `microsoft/phi-4`, 14B parameters)
- **Quantized by:** [artindnr](https://huggingface.co/artindnr), using [Unsloth](https://github.com/unslothai/unsloth)
- **Format:** safetensors (4-bit, bitsandbytes nf4)
- **License:** MIT
- **Languages:** Farsi (primary conversational focus), English, and general multilingual support

## How to Use

### Generation with 🤗 Transformers

The checkpoint is already stored in 4-bit, so it loads directly — no `BitsAndBytesConfig` needed on your end, `device_map="auto"` handles placement.

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "artindnr/tea-4bit"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    device_map="auto",
)

USER_PROMPT = "تو کی هستی و اسمت چیه؟"

messages = [
    {"role": "user", "content": USER_PROMPT},
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=1024,
    temperature=0.7,
    do_sample=True,
)

print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
```

### Generation with Unsloth (faster inference/fine-tuning)

```bash
pip install unsloth
```

```python
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="artindnr/tea-4bit",
    max_seq_length=8192,
    load_in_4bit=True,
    dtype=None,  # auto-detect
)
FastLanguageModel.for_inference(model)

messages = [
    {"role": "user", "content": "تو کی هستی و اسمت چیه؟"},
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=1024,
    temperature=0.7,
    do_sample=True,
)

print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
```

Unsloth also lets you use this checkpoint as a starting point for further QLoRA fine-tuning if you want to adapt tea further while keeping it in 4-bit.

## Intended Use

Same as [`artindnr/tea`](https://huggingface.co/artindnr/tea): Farsi-first conversational assistance, question answering, and long multi-turn assistant deployments — this repo specifically targets **lower-VRAM GPU inference and fine-tuning** via Transformers/Unsloth, as an alternative to the GGUF quants for CPU/llama.cpp-based deployment.

## Limitations

- 4-bit quantization trades off some accuracy for memory footprint; expect some quality degradation versus the full-precision [`artindnr/tea`](https://huggingface.co/artindnr/tea), particularly on nuanced or long-context Farsi generation.
- This checkpoint requires a CUDA GPU and `bitsandbytes` — it is not intended for CPU inference or llama.cpp; use [artindnr/tea-gguf](https://huggingface.co/artindnr/tea-gguf) for that.
- Inherits all limitations of the base [`artindnr/tea`](https://huggingface.co/artindnr/tea) model and the underlying `microsoft/phi-4` checkpoint, including possible hallucinated facts.
- No formal safety fine-tuning beyond what is inherited from the base model has been applied; use appropriate safeguards in production settings.

## License

This model is released under the [MIT License](https://opensource.org/licenses/MIT), consistent with `artindnr/tea` and the base `microsoft/phi-4` model.

## Citation

If you use tea in your work, please cite:

```bibtex
@misc{tea,
  title  = {tea: A Farsi-Focused, Full Fine-tune of Phi-4 for QA and Long-form Assistance},
  author = {artindnr},
  year   = {2026},
  url    = {https://huggingface.co/artindnr/tea}
}
```

## Acknowledgements

Built on top of [`artindnr/tea`](https://huggingface.co/artindnr/tea), itself a full fine-tune of [`microsoft/phi-4`](https://huggingface.co/microsoft/phi-4). 4-bit quantization via [Unsloth](https://github.com/unslothai/unsloth).