Instructions to use prithivMLmods/VisionGuardrail-9B-fp8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use prithivMLmods/VisionGuardrail-9B-fp8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="prithivMLmods/VisionGuardrail-9B-fp8")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("prithivMLmods/VisionGuardrail-9B-fp8")
model = AutoModelForMultimodalLM.from_pretrained("prithivMLmods/VisionGuardrail-9B-fp8", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use prithivMLmods/VisionGuardrail-9B-fp8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "prithivMLmods/VisionGuardrail-9B-fp8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "prithivMLmods/VisionGuardrail-9B-fp8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/prithivMLmods/VisionGuardrail-9B-fp8

SGLang

How to use prithivMLmods/VisionGuardrail-9B-fp8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "prithivMLmods/VisionGuardrail-9B-fp8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "prithivMLmods/VisionGuardrail-9B-fp8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "prithivMLmods/VisionGuardrail-9B-fp8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "prithivMLmods/VisionGuardrail-9B-fp8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use prithivMLmods/VisionGuardrail-9B-fp8 with Docker Model Runner:
```
docker model run hf.co/prithivMLmods/VisionGuardrail-9B-fp8
```

VisionGuardrail-9B-fp8

VisionGuardrail-9B-fp8 is an FP8 dynamic quantized version of prithivMLmods/VisionGuardrail-9B, a multimodal content-safety classifier based on Qwen/Qwen3.5-9B.

The model is trained on the ImageShield-Guardrail-Pro content-safety guardrail dataset and is designed to classify visual content as Safe or Unsafe, with a particular focus on detecting Not Safe for Work (NSFW) sensual content and other potentially sensitive visual content.

The FP8 dynamic quantization reduces the model's memory footprint while retaining the multimodal vision and classification capabilities of the original BF16 model, making deployment more accessible on GPUs with FP8 support.

hf.co/blog — vision-guardrail-mini-blog

This model is an experimental release and may generate unexpected behaviors or reasoning artifacts in certain scenarios.

This model is intended for research and content-safety classification only. Any outputs generated by this model are the sole responsibility of the user. The authors and hosting platform disclaim all liability for generated content. Users must ensure safe, ethical, and lawful usage.

Quantization Details

Quantization was performed using llmcompressor with the following recipe:

default_stage:
  default_modifiers:
    QuantizationModifier:
      targets: [Linear]
      ignore: ['re:.*lm_head', 're:.*embed_tokens$', 're:.*visual.*', 're:.*model.visual.*',
        're:.*linear_attn.*']
      scheme: FP8_DYNAMIC
      bypass_divisibility_checks: false
      requires_calibration_data: false

Linear layers are quantized to FP8 using dynamic per-tensor activation scaling. The lm_head, embedding table, vision components (visual), and linear_attn layers are excluded from quantization and remain at full precision.

This approach reduces the overall model size and memory requirements while preserving the vision-language representation and content-safety classification capabilities of the base model.


Base model	prithivMLmods/VisionGuardrail-9B
Underlying model	Qwen/Qwen3.5-9B
Quantization scheme	FP8_DYNAMIC (Linear layers only)
Format	compressed-tensors
Calibration required	No
Excluded from quantization	`lm_head`, `embed_tokens`, `visual`, `linear_attn`

Model Details


Developer	prithivMLmods
Quantized by	prithivMLmods
Architecture	Multimodal decoder-only language model
Parameters	9B
Base model	Qwen/Qwen3.5-9B
Training dataset	ImageShield-Guardrail-Pro
Task	Multimodal content-safety classification
Input	Image + text prompt
Output	Safe / Unsafe classification with reasoning
Quantization	FP8 Dynamic
Format	compressed-tensors

Intended Use

VisionGuardrail-9B-FP8 is intended for multimodal content-safety research and visual content classification.

The model can be used to analyze images for potentially sensitive or unsafe visual content, including:

NSFW sensual content
Explicit visual content
Clothing exposure
Dress-code related visual cues
Poses and body positioning
Framing and visual composition
Potentially sensitive visual settings
Other content requiring conservative safety classification

The model is designed to provide a Safe or Unsafe classification based on the visual content and accompanying prompt.

Usage with Transformers

The FP8 checkpoint can be loaded using the Transformers ecosystem with support for compressed-tensors quantization.

from transformers import AutoProcessor, AutoModelForImageTextToText
from PIL import Image

model_id = "prithivMLmods/VisionGuardrail-9B-fp8"

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    device_map="auto",
)

image = Image.open("image.jpg")

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image"},
            {
                "type": "text",
                "text": "Classify this image as Safe or Unsafe and explain the safety classification.",
            },
        ],
    }
]

text = processor.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = processor(
    text=text,
    images=image,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
)

response = processor.batch_decode(
    outputs,
    skip_special_tokens=True,
)[0]

print(response)

Note: Refer to the base VisionGuardrail-9B model card for the recommended prompt format and inference behavior.

Prompt Format

A simple classification prompt can be used:

Analyze the provided image for content safety.

Classify the image as either:
- Safe
- Unsafe

Provide a concise explanation for the classification.

For production applications, it is recommended to define a strict output schema and validate the generated response before consuming the classification programmatically.

Hardware Requirements

FP8 inference is recommended on GPUs with native FP8 support, such as NVIDIA Hopper and Blackwell architectures.

The quantized model can also be used on other supported GPUs where FP8 weights are dequantized during execution, although performance and memory characteristics may differ.

For best performance:

Use a recent PyTorch release with FP8 support.
Use a recent Transformers release with compressed-tensors support.
Prefer NVIDIA Hopper or Blackwell GPUs for native FP8 execution.
Use bfloat16 for non-quantized components where supported.

Quantization Recipe

The complete quantization configuration is:

default_stage:
  default_modifiers:
    QuantizationModifier:
      targets: [Linear]
      ignore:
        - 're:.*lm_head'
        - 're:.*embed_tokens$'
        - 're:.*visual.*'
        - 're:.*model.visual.*'
        - 're:.*linear_attn.*'
      scheme: FP8_DYNAMIC
      bypass_divisibility_checks: false
      requires_calibration_data: false

Quantization Strategy

Linear layers: FP8 Dynamic
Activation scaling: Dynamic
Calibration dataset: Not required
Language-model head: Full precision
Token embeddings: Full precision
Vision encoder: Full precision
Linear attention layers: Full precision

Keeping the vision components and selected attention layers in higher precision is intended to preserve multimodal representation quality and classification behavior.

Limitations

VisionGuardrail-9B-FP8 inherits the limitations of VisionGuardrail-9B.

The model is an experimental content-safety classifier and should not be treated as a definitive safety system. Classification errors, false positives, and false negatives may occur.

FP8 quantization may introduce small numerical differences compared with the original BF16 checkpoint. Downstream performance should therefore be evaluated against the original model for production deployments.

The model should be used as one component of a broader content-moderation pipeline rather than as the sole safety mechanism for high-risk applications.

Responsible AI

This model is intended for research, evaluation, and responsible content-safety applications.

Users are responsible for ensuring that their deployment complies with applicable laws, regulations, platform policies, and organizational safety requirements.

For the full safety guidance, intended use, limitations, and responsible AI considerations, refer to the VisionGuardrail-9B model card.