Instructions to use tcclaviger/Tess-27B-RFA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tcclaviger/Tess-27B-RFA with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="tcclaviger/Tess-27B-RFA")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("tcclaviger/Tess-27B-RFA")
model = AutoModelForMultimodalLM.from_pretrained("tcclaviger/Tess-27B-RFA")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tcclaviger/Tess-27B-RFA with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tcclaviger/Tess-27B-RFA"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tcclaviger/Tess-27B-RFA",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/tcclaviger/Tess-27B-RFA

SGLang

How to use tcclaviger/Tess-27B-RFA with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tcclaviger/Tess-27B-RFA" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tcclaviger/Tess-27B-RFA",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tcclaviger/Tess-27B-RFA" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tcclaviger/Tess-27B-RFA",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use tcclaviger/Tess-27B-RFA with Docker Model Runner:
```
docker model run hf.co/tcclaviger/Tess-27B-RFA
```

tcclaviger commited on 2 days ago

Commit

7c65698

1 Parent(s): bf4c455

Rebaseline base Qwen3.6-27B column to comparable 2026-07-12 re-measurement

Browse files

Base figures were previously from a run on a different vLLM image with
thinking-ON codeneedle (empty-response recall damage). Re-measured on the
same tcclaviger/vllm:latest image and thinking-OFF methodology as every
other build. Corrects the base codeneedle, decode, and tool-eval rows.

Files changed (1) hide show

README.md +31 -29

README.md CHANGED Viewed

@@ -50,9 +50,10 @@ extended from the native 256K window with YaRN via `--hf-overrides`:
 ## Evaluation results
-Five builds measured with identical methodology, each against its own live vLLM endpoint (July 2026):
 **Qwen3.6-27B** (bf16 base) → **Tess-4-27B** (the tune, bf16) → **Tess-27B-RFI** (int8+4-bit hybrid) →
-**Tess-27B-RFA** (this build, all-4-bit), with **Qwen3.6-35B-A3B** (MoE, bf16) as a comparative reference.
 Bold marks the best score in each row (ties all bolded).
 ### Quantization cost (all-RFA 4-bit) — Tess-4-27B → Tess-27B-RFA
@@ -70,13 +71,13 @@ Bold marks the best score in each row (ties all bolded).
 ### Quality
-| Metric | Qwen3.6-27B (base) | Tess-4-27B | Tess-27B-RFI | **Tess-27B-RFA** | Qwen3.6-35B-A3B |
-|---|---|---|---|---|---|
-| WikiText-2 PPL (n_ctx 2048, lower is better) | 7.0597 | 6.6691 | 6.6632 | 6.6292 | **6.5092** |
-| ARC-Challenge (acc_norm) | 59.13% | **60.84%** | 60.41% | 60.32% | 55.20% |
-| ARC-Easy (acc_norm) | 75.76% | 77.53% | 77.40% | **78.87%** | 71.13% |
-| Winogrande (acc) | 76.87% | 77.43% | **77.51%** | 76.80% | 73.40% |
-| HellaSwag (acc_norm) | 84.04% | 84.21% | **84.27%** | 84.05% | 82.95% |
 Multiple-choice accuracy is lm-eval loglikelihood scoring, 0-shot.
@@ -84,34 +85,34 @@ Multiple-choice accuracy is lm-eval loglikelihood scoring, 0-shot.
 Verbatim function recall under 10K–80K-token contexts.
-| Corpus | Qwen3.6-27B (base) | Tess-4-27B | Tess-27B-RFI | **Tess-27B-RFA** | Qwen3.6-35B-A3B |
-|---|---|---|---|---|---|
-| Python | 92.73% | **100%** | **100%** | **100%** | 99.09% |
-| C++ | 92.81% | 98.12% | 98.44% | **98.75%** | 98.44% |
-| Rust | 95.00% | **99.69%** | **99.69%** | **99.69%** | 99.38% |
-| JS (~80K tokens) | 87.50% | 93.13% | **93.75%** | **93.75%** | 92.19% |
-| Tools | 96.52% | **99.57%** | **99.57%** | **99.57%** | 93.48% |
-| **Overall recall** | 91.95% | 97.73% | 97.97% | **98.05%** | 97.28% |
 ### Tool calling & accuracy benches
-| Bench | Qwen3.6-27B (base) | Tess-4-27B | Tess-27B-RFI | **Tess-27B-RFA** | Qwen3.6-35B-A3B |
-|---|---|---|---|---|---|
-| tool-eval final (full 69, TC-61 excl) | 89 | 85 | 87 | 86 | **90** |
-| GSM8K (50q) | **98.0%** | 94.0% | **98.0%** | **98.0%** | 96.0% |
-| MMLU (50q) | 74.0% | 76.0% | **82.0%** | 80.0% | 64.0% |
-| IFEval (20 prompts, prompt-level) | 90.0% | 90.0% | 90.0% | **95.0%** | 90.0% |
 ### Decode throughput — tok/s output (ISL 128 / ISL 512)
 `vllm bench serve`, random dataset, OSL 128, saturation, 4× R9700 (gfx1201), TP 4.
-| Concurrency | Qwen3.6-27B (base) | Tess-4-27B | Tess-27B-RFI | **Tess-27B-RFA** | Qwen3.6-35B-A3B |
-|---|---|---|---|---|---|
-| 1 | 63.7 / 65.2 | 59.9 / 63.8 | 75.3 / 69.3 | 67.9 / 58.2 | **91.9 / 114.4** |
-| 10 | 251.1 / 246.6 | 289.6 / 227.8 | 281.5 / 219.2 | 292.2 / 194.8 | **434.9 / 440.3** |
-| 25 | 390.3 / 336.8 | 492.0 / 341.1 | 429.1 / 284.0 | 369.0 / 260.3 | **688.9 / 563.7** |
-| 50 | 541.2 / 363.0 | 527.9 / 345.6 | 452.8 / 295.0 | 422.6 / 278.4 | **889.8 / 702.9** |
 ### MTP draft acceptance by work category
@@ -128,6 +129,7 @@ Measured from live serving logs, k=5 draft tokens, drafted-token-weighted aggreg
 All builds serve on the [tcclaviger/vllm:latest](https://hub.docker.com/r/tcclaviger/vllm) image, which has kernel tunes baked in.
 TunableOp is untuned — GEMMs run on default heuristic-determined values.
 ## Credits

 ## Evaluation results
+Six builds measured with identical methodology, each against its own live vLLM endpoint (July 2026):
 **Qwen3.6-27B** (bf16 base) → **Tess-4-27B** (the tune, bf16) → **Tess-27B-RFI** (int8+4-bit hybrid) →
+**Tess-27B-RFA** (this build, all-4-bit) → **Tess-FP8** (W8A8 block-128 FP8 sibling),
+with **Qwen3.6-35B-A3B** (MoE, bf16) as a comparative reference.
 Bold marks the best score in each row (ties all bolded).
 ### Quantization cost (all-RFA 4-bit) — Tess-4-27B → Tess-27B-RFA
 ### Quality
+| Metric | Qwen3.6-27B (base) | Tess-4-27B | Tess-27B-RFI | **Tess-27B-RFA** | Tess-FP8 | Qwen3.6-35B-A3B |
+|---|---|---|---|---|---|---|
+| WikiText-2 PPL (n_ctx 2048, lower is better) | 7.0559 | 6.6691 | 6.6632 | 6.6292 | 6.6627 | **6.5092** |
+| ARC-Challenge (acc_norm) | 59.30% | **60.84%** | 60.41% | 60.32% | 60.49% | 55.20% |
+| ARC-Easy (acc_norm) | 75.93% | 77.53% | 77.40% | **78.87%** | 77.82% | 71.13% |
+| Winogrande (acc) | 77.51% | 77.43% | 77.51% | 76.80% | **77.66%** | 73.40% |
+| HellaSwag (acc_norm) | 84.12% | 84.21% | **84.27%** | 84.05% | 84.13% | 82.95% |
 Multiple-choice accuracy is lm-eval loglikelihood scoring, 0-shot.
 Verbatim function recall under 10K–80K-token contexts.
+| Corpus | Qwen3.6-27B (base) | Tess-4-27B | Tess-27B-RFI | **Tess-27B-RFA** | Tess-FP8 | Qwen3.6-35B-A3B |
+|---|---|---|---|---|---|---|
+| Python | **100%** | **100%** | **100%** | **100%** | 99.55% | 99.09% |
+| C++ | 98.12% | 98.12% | 98.44% | **98.75%** | **98.75%** | 98.44% |
+| Rust | **99.69%** | **99.69%** | **99.69%** | **99.69%** | **99.69%** | 99.38% |
+| JS (~80K tokens) | 93.44% | 93.13% | **93.75%** | **93.75%** | 93.44% | 92.19% |
+| Tools | 98.26% | **99.57%** | **99.57%** | **99.57%** | **99.57%** | 93.48% |
+| **Overall recall** | 97.81% | 97.73% | 97.97% | **98.05%** | 97.86% | 97.28% |
 ### Tool calling & accuracy benches
+| Bench | Qwen3.6-27B (base) | Tess-4-27B | Tess-27B-RFI | **Tess-27B-RFA** | Tess-FP8 | Qwen3.6-35B-A3B |
+|---|---|---|---|---|---|---|
+| tool-eval final (full 69, TC-61 excl) | 86 | 85 | 87 | 86 | 87 | **90** |
+| GSM8K (50q) | **98.0%** | 94.0% | **98.0%** | **98.0%** | **98.0%** | 96.0% |
+| MMLU (50q) | 74.0% | 76.0% | **82.0%** | 80.0% | 76.0% | 64.0% |
+| IFEval (20 prompts, prompt-level) | 90.0% | 90.0% | 90.0% | **95.0%** | 90.0% | 90.0% |
 ### Decode throughput — tok/s output (ISL 128 / ISL 512)
 `vllm bench serve`, random dataset, OSL 128, saturation, 4× R9700 (gfx1201), TP 4.
+| Concurrency | Qwen3.6-27B (base) | Tess-4-27B | Tess-27B-RFI | **Tess-27B-RFA** | Tess-FP8 | Qwen3.6-35B-A3B |
+|---|---|---|---|---|---|---|
+| 1 | 57.0 / 61.4 | 59.9 / 63.8 | 75.3 / 69.3 | 67.9 / 58.2 | 86.0 / 85.1 | **91.9 / 114.4** |
+| 10 | 304.5 / 233.6 | 289.6 / 227.8 | 281.5 / 219.2 | 292.2 / 194.8 | 280.6 / 269.2 | **434.9 / 440.3** |
+| 25 | 424.4 / 321.7 | 492.0 / 341.1 | 429.1 / 284.0 | 369.0 / 260.3 | 533.2 / 355.4 | **688.9 / 563.7** |
+| 50 | 556.0 / 349.8 | 527.9 / 345.6 | 452.8 / 295.0 | 422.6 / 278.4 | 560.3 / 394.1 | **889.8 / 702.9** |
 ### MTP draft acceptance by work category
 All builds serve on the [tcclaviger/vllm:latest](https://hub.docker.com/r/tcclaviger/vllm) image, which has kernel tunes baked in.
 TunableOp is untuned — GEMMs run on default heuristic-determined values.
+Base Qwen3.6-27B figures are the 2026-07-12 re-measurement on the same tcclaviger/vllm:latest image and thinking-OFF methodology as every other build, replacing an earlier non-comparable run.
 ## Credits