Instructions to use Crusadersk/gemma-2-2b-it-FP8-Dynamic-TR171 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Crusadersk/gemma-2-2b-it-FP8-Dynamic-TR171 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Crusadersk/gemma-2-2b-it-FP8-Dynamic-TR171")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Crusadersk/gemma-2-2b-it-FP8-Dynamic-TR171")
model = AutoModelForCausalLM.from_pretrained("Crusadersk/gemma-2-2b-it-FP8-Dynamic-TR171", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Crusadersk/gemma-2-2b-it-FP8-Dynamic-TR171 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Crusadersk/gemma-2-2b-it-FP8-Dynamic-TR171"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Crusadersk/gemma-2-2b-it-FP8-Dynamic-TR171",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Crusadersk/gemma-2-2b-it-FP8-Dynamic-TR171

SGLang

How to use Crusadersk/gemma-2-2b-it-FP8-Dynamic-TR171 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Crusadersk/gemma-2-2b-it-FP8-Dynamic-TR171" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Crusadersk/gemma-2-2b-it-FP8-Dynamic-TR171",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Crusadersk/gemma-2-2b-it-FP8-Dynamic-TR171" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Crusadersk/gemma-2-2b-it-FP8-Dynamic-TR171",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Crusadersk/gemma-2-2b-it-FP8-Dynamic-TR171 with Docker Model Runner:
```
docker model run hf.co/Crusadersk/gemma-2-2b-it-FP8-Dynamic-TR171
```

gemma-2-2b — FP8_DYNAMIC (W8A8-e4m3)

Weight-FP8 checkpoint of google/gemma-2-2b-it, produced for the TR171 deployment-time safety-tax benchmark.

Provenance

Field	Value
Base model	`google/gemma-2-2b-it`
Base revision	`299a8560bedf22ed1c72a8a11e7dce4a7f9f51f8` (verified — recorded in the frozen matrix)
Recipe	`FP8_DYNAMIC` (W8A8-e4m3), `llmcompressor`
Quantization method	`compressed-tensors`
Calibration data	none — FP8_DYNAMIC is data-free
Build date	2026-07-02
Shard size	3.24 GB
Quantize wall time	177.5 s
Integrity record	per-file `sha256` from Hub LFS metadata; `shard_bytes` verified

Reproducing

Producer: research/tr171/expansion/fp8_support_probe.py; environment: research/tr171/expansion/Dockerfile.fp8. The recipe takes no calibration corpus, so there is no dataset or seed to reproduce — only the base checkpoint and the toolchain version.

Known reproducibility gap: llmcompressor was unpinned at build time, so the exact version used on 2026-07-02 is unrecorded. The Dockerfile now pins it. A rebuild may therefore not be bit-identical to this artifact.

Integrity, stated honestly: the 2026-07-02 build recorded no sha256 of its own, and the local build directory is now empty, so no aggregate directory digest exists for it. What is verifiable instead: the per-file sha256 below is read from this repo's Git-LFS metadata, and the mirror was checked against the build record — summing the file sizes in this repo, excluding the generated README.md, NOTICE and .gitattributes, reproduces the matrix's shard_bytes of 3,240,242,075 exactly. So these hashes describe the same bytes the probe measured, and you can verify a download against them directly:

File	sha256
`model.safetensors`	`05a93e8dcf0bc822adcf7985510c5635946dc72f49b3f6a356641515d82522d1`
`tokenizer.json`	`487cee8724215dcd2dde8888539e8b1bf844ceb5dbbe27f7845abda69eeb060f`

LFS-tracked files only; the small JSON/text files are git blobs and carry no sha256. fp8_support_probe.py now records a real digest at build time, so future shards will not need this reconstruction.

License and notices

Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms. Use is subject to the Gemma Prohibited Use Policy at ai.google.dev/gemma/prohibited_use_policy. These terms travel with this derivative and must be passed on to any downstream recipient.

This FP8 derivative inherits the upstream terms of google/gemma-2-2b-it. Consult the base model's licence before redistributing.