Instructions to use ghostrider0470/Qwen3.8-27B-RFT-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ghostrider0470/Qwen3.8-27B-RFT-v1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="ghostrider0470/Qwen3.8-27B-RFT-v1") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("ghostrider0470/Qwen3.8-27B-RFT-v1") model = AutoModelForMultimodalLM.from_pretrained("ghostrider0470/Qwen3.8-27B-RFT-v1", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use ghostrider0470/Qwen3.8-27B-RFT-v1 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ghostrider0470/Qwen3.8-27B-RFT-v1" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ghostrider0470/Qwen3.8-27B-RFT-v1", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/ghostrider0470/Qwen3.8-27B-RFT-v1
- SGLang
How to use ghostrider0470/Qwen3.8-27B-RFT-v1 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ghostrider0470/Qwen3.8-27B-RFT-v1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ghostrider0470/Qwen3.8-27B-RFT-v1", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ghostrider0470/Qwen3.8-27B-RFT-v1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ghostrider0470/Qwen3.8-27B-RFT-v1", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use ghostrider0470/Qwen3.8-27B-RFT-v1 with Docker Model Runner:
docker model run hf.co/ghostrider0470/Qwen3.8-27B-RFT-v1
Qwen3.8-27B-RFT-v1
Qwen3.8-27B after one round of rejection-sampling fine-tuning (RFT). The goal was shorter reasoning for the same accuracy.
Result in one line: accuracy is statistically the same as the base model in both thinking modes, and correct answers need about 10โ30% fewer reasoning tokens on most benchmarks.
A model by Horizon Tech d.o.o., Sarajevo, Bosnia and Herzegovina.
Model details
| Developed by | Horizon Tech d.o.o., Sarajevo, Bosnia and Herzegovina |
| Model type | 27B-parameter vision-language model (text and images in, text out), with optional reasoning ("thinking") before the answer |
| Fine-tuned from | Qwen/Qwen3.8-27B, revision 1d4bf0f |
| Method | Rejection-sampling fine-tuning (RFT) with a LoRA adapter, merged into the weights |
| Weights | bfloat16 safetensors, 18 shards, 52 GB; retrained MTP head included (mtp.* tensors) |
| Languages | As the base model; evaluated in English and German |
| License | Apache 2.0 |
| Released | November 2026 |
| Author | Hamza Abdagic (ORCID 0009-0002-4078-4922) |
| Contact | hamza.abdagic@horizon-tech.io |
Versions
| Repository | Format | Size | For |
|---|---|---|---|
| Qwen3.8-27B-RFT-v1 | bfloat16 safetensors | 52 GB | Transformers, vLLM, SGLang; further fine-tuning |
| Qwen3.8-27B-RFT-v1-NVFP4 | NVFP4 (4-bit) | 20 GB | vLLM on NVIDIA GPUs with FP4 support |
| Qwen3.8-27B-RFT-v1-mlx-4bit | MLX 4-bit | 14 GB | Apple silicon, text only |
What was done
- Base: Qwen/Qwen3.8-27B, revision
1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0. - Method: the base model answered training problems itself, programs graded the answers, and the model was trained on its own correct, shortest answers. No teacher model and no model-written grading.
- Training: LoRA (rank 64, alpha 128), 1.5 epochs, learning rate 1e-4, sequences packed to 32,768 tokens. Validation loss went from 0.283 to 0.234.
- These weights: the adapter merged into the base in fp32 and stored as bfloat16. The vision tower is unchanged.
Evaluation
Paired comparison against the base on the same items (McNemar test). No difference is significant (smallest p = 0.12 with thinking on).
| Benchmark | n | Base, thinking on | RFT v1, thinking on | Base, thinking off | RFT v1, thinking off |
|---|---|---|---|---|---|
| GSM8K | 1319 | 97.95 | 97.88 | 96.59 | 96.82 |
| MMLU-Pro (subset) | 200 | 86.50 | 84.00 | 65.50 | 65.50 |
| MATH-500 | 500 | 98.20 | 98.40 | 97.40 | 98.20 |
| AIME 2025 | 30 | 83.33 | 83.33 | 70.00 | 63.33 |
| HumanEval | 164 | 93.90 | 95.73 | 97.56 | 97.56 |
| MBPP | 500 | 94.20 | 93.80 | 81.00 | 80.80 |
| IFEval | 541 | 89.09 | 90.76 | 82.07 | 81.89 |
| German MMLU (subset) | 200 | 83.50 | 86.50 | 80.00 | 81.00 |
Reasoning tokens per correct answer, thinking on (RFT v1 vs base, same run as the table):
| Benchmark | Base | RFT v1 | Change |
|---|---|---|---|
| GSM8K | 389 | 338 | โ13% |
| MMLU-Pro | 1,702 | 1,179 | โ31% |
| MATH-500 | 1,481 | 1,328 | โ10% |
| AIME 2025 | 9,018 | 8,845 | โ2% |
| HumanEval | 1,531 | 1,070 | โ30% |
| MBPP | 2,885 | 2,603 | โ10% |
The number of answers cut off at the token limit did not change in a consistent direction.
Limitations
- This is not a more accurate model than the base. Several of these benchmarks are close to their ceiling for the base, so they cannot show a gain.
- The multi-token-prediction (MTP) head was retrained for this model by self-distillation (the main weights are unchanged by it). Measured on 120 held-out documents, the retrained head gets 3.15 drafted tokens accepted at 3 drafts and 4.06 at 5 drafts; the head copied from the base got 3.02 and 3.63. These are offline numbers, not serving speed.
- Scores come from our own harness and are meant for the base-vs-RFT comparison, not for comparison with numbers published elsewhere.
How to use
Same usage as the base model (Transformers, vLLM, SGLang); the chat template and thinking controls are unchanged.
Transformers
import torch
from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration
repo = "ghostrider0470/Qwen3.8-27B-RFT-v1"
tok = AutoTokenizer.from_pretrained(repo)
model = Qwen3_5ForConditionalGeneration.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto")
messages = [{"role": "user", "content": "A train leaves at 14:10 and arrives at 16:45. How long is the trip?"}]
prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False, enable_thinking=True) # False = answer without reasoning
ids = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=2048)
print(tok.decode(out[0, ids["input_ids"].shape[1]:], skip_special_tokens=True))
vLLM
vllm serve ghostrider0470/Qwen3.8-27B-RFT-v1
Intended use
- A drop-in replacement for Qwen3.8-27B where reasoning length matters: the same answers for fewer reasoning tokens, so lower latency and cost per answer with thinking on.
- A starting point for further fine-tuning.
- It has the same safety behaviour and the same limits as the base model. It was not trained on any user or customer data.
Citation
@misc{horizontech2026rftv1,
author = {Abdagic, Hamza},
title = {Qwen3.8-27B-RFT-v1},
year = {2026},
publisher = {Hugging Face},
organization = {Horizon Tech d.o.o., Sarajevo},
howpublished = {\url{https://huggingface.co/ghostrider0470/Qwen3.8-27B-RFT-v1}}
}
Acknowledgements
- Built on Qwen/Qwen3.8-27B by the Qwen team at Alibaba Cloud.
License
Apache 2.0, as the base model.
- Downloads last month
- 95