Instructions to use magiccodingman/Qwen3.8-27B-heretic-ara-DFlash2-fp8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use magiccodingman/Qwen3.8-27B-heretic-ara-DFlash2-fp8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="magiccodingman/Qwen3.8-27B-heretic-ara-DFlash2-fp8")

# Load model directly
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("magiccodingman/Qwen3.8-27B-heretic-ara-DFlash2-fp8")
model = AutoModel.from_pretrained("magiccodingman/Qwen3.8-27B-heretic-ara-DFlash2-fp8", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use magiccodingman/Qwen3.8-27B-heretic-ara-DFlash2-fp8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "magiccodingman/Qwen3.8-27B-heretic-ara-DFlash2-fp8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "magiccodingman/Qwen3.8-27B-heretic-ara-DFlash2-fp8",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/magiccodingman/Qwen3.8-27B-heretic-ara-DFlash2-fp8

SGLang

How to use magiccodingman/Qwen3.8-27B-heretic-ara-DFlash2-fp8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "magiccodingman/Qwen3.8-27B-heretic-ara-DFlash2-fp8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "magiccodingman/Qwen3.8-27B-heretic-ara-DFlash2-fp8",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "magiccodingman/Qwen3.8-27B-heretic-ara-DFlash2-fp8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "magiccodingman/Qwen3.8-27B-heretic-ara-DFlash2-fp8",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use magiccodingman/Qwen3.8-27B-heretic-ara-DFlash2-fp8 with Docker Model Runner:
```
docker model run hf.co/magiccodingman/Qwen3.8-27B-heretic-ara-DFlash2-fp8
```

Qwen3.8-27B-heretic-ara-DFlash2-fp8 / fp8_functional_validation.json

magiccodingman

Upload folder using huggingface_hub

b6bbe8a verified about 18 hours ago

Raw

History Blame Contribute Delete

4.64 kB

	{
	"status": "pass",
	"profile": "mlp-only",
	"comparison": "BF16 weights vs block-FP8 weights dequantized to BF16",
	"input_distribution": "fixed-seed unit-normal BF16 vectors (RMSNorm-like)",
	"samples_per_mlp": 64,
	"samples_per_attention_output_projection": 128,
	"mlp_aggregate": {
	"rmse": 51.15606406613011,
	"relative_rmse": 0.028808014285076088,
	"cosine_similarity": 0.999586214895275,
	"maximum_absolute_error": 1024.0
	},
	"attention_output_projection_aggregate": {
	"rmse": 0.0,
	"relative_rmse": 0.0,
	"cosine_similarity": 1.0,
	"maximum_absolute_error": 0.0
	},
	"gates": {
	"mlp_relative_rmse_max": 0.1,
	"mlp_cosine_similarity_min": 0.995,
	"attention_relative_rmse_max": 0.05,
	"attention_cosine_similarity_min": 0.998
	},
	"per_layer_mlp": [
	{
	"layer": 0,
	"rmse": 46.944710713054484,
	"relative_rmse": 0.02199275336409562,
	"cosine_similarity": 0.9997595575772288,
	"maximum_absolute_error": 1024.0,
	"squared_error": 722143105.4933598,
	"reference_energy": 1493014623467.2358,
	"dot": 1489775007178.6418,
	"candidate_energy": 1487257533995.5679,
	"elements": 327680
	},
	{
	"layer": 1,
	"rmse": 51.91130504990165,
	"relative_rmse": 0.031573682259121584,
	"cosine_similarity": 0.9995025435310865,
	"maximum_absolute_error": 1024.0,
	"squared_error": 883026687.421299,
	"reference_energy": 885774882473.856,
	"dot": 883571172877.6316,
	"candidate_energy": 882250489968.8706,
	"elements": 327680
	},
	{
	"layer": 2,
	"rmse": 53.50805681930253,
	"relative_rmse": 0.03490318748371408,
	"cosine_similarity": 0.9993913110064463,
	"maximum_absolute_error": 512.0,
	"squared_error": 938184587.5352235,
	"reference_energy": 770119488860.0505,
	"dot": 768330396334.169,
	"candidate_energy": 767479488395.8666,
	"elements": 327680
	},
	{
	"layer": 3,
	"rmse": 51.742237285112125,
	"relative_rmse": 0.038977179685573425,
	"cosine_similarity": 0.9992402571060341,
	"maximum_absolute_error": 384.0,
	"squared_error": 877284268.2020159,
	"reference_energy": 577456825572.1453,
	"dot": 576257438543.1458,
	"candidate_energy": 575935335782.396,
	"elements": 327680
	},
	{
	"layer": 4,
	"rmse": 51.43689078790791,
	"relative_rmse": 0.024536587190349065,
	"cosine_similarity": 0.9997013312358347,
	"maximum_absolute_error": 1024.0,
	"squared_error": 866960583.5332537,
	"reference_energy": 1440028343049.15,
	"dot": 1436015671674.3086,
	"candidate_energy": 1432869960883.038,
	"elements": 327680
	}
	],
	"per_layer_attention_output_projection": [
	{
	"layer": 0,
	"rmse": 0.0,
	"relative_rmse": 0.0,
	"cosine_similarity": 1.0,
	"maximum_absolute_error": 0.0,
	"squared_error": 0.0,
	"reference_energy": 33139259.3109601,
	"dot": 33139259.3109601,
	"candidate_energy": 33139259.3109601,
	"elements": 655360
	},
	{
	"layer": 1,
	"rmse": 0.0,
	"relative_rmse": 0.0,
	"cosine_similarity": 1.0,
	"maximum_absolute_error": 0.0,
	"squared_error": 0.0,
	"reference_energy": 41008694.364316456,
	"dot": 41008694.364316456,
	"candidate_energy": 41008694.364316456,
	"elements": 655360
	},
	{
	"layer": 2,
	"rmse": 0.0,
	"relative_rmse": 0.0,
	"cosine_similarity": 1.0,
	"maximum_absolute_error": 0.0,
	"squared_error": 0.0,
	"reference_energy": 47956004.669400826,
	"dot": 47956004.669400826,
	"candidate_energy": 47956004.669400826,
	"elements": 655360
	},
	{
	"layer": 3,
	"rmse": 0.0,
	"relative_rmse": 0.0,
	"cosine_similarity": 1.0,
	"maximum_absolute_error": 0.0,
	"squared_error": 0.0,
	"reference_energy": 54665058.895079024,
	"dot": 54665058.895079024,
	"candidate_energy": 54665058.895079024,
	"elements": 655360
	},
	{
	"layer": 4,
	"rmse": 0.0,
	"relative_rmse": 0.0,
	"cosine_similarity": 1.0,
	"maximum_absolute_error": 0.0,
	"squared_error": 0.0,
	"reference_energy": 76522649.49797027,
	"dot": 76522649.49797027,
	"candidate_energy": 76522649.49797027,
	"elements": 655360
	}
	],
	"limitation": "This validates selected sublayer arithmetic, not native W8A8 kernels or end-to-end DFlash2 acceptance. The current DFlash2 vLLM implementation is an unmerged experimental branch.",
	"elapsed_seconds": 1.1974219349795021
	}