PrimeryOP/gemma-2-abliterated-2B-ov

---

base_model: google/gemma-2-2b-it tags: - openvino - nncf - optimum - abliterated - intel - arc model-index: - name: PrimeryOP/gemma-2-abliterated-2B-ov-int8 results: []

🛸 Gemma 2 2B - Null-Space Abliterated (OpenVINO IR)

This is a specialized version of Gemma 2 2B that has been Null-Space Abliterated to remove refusal behaviors and then exported to OpenVINO IR format for high-performance inference on Intel hardware.

🛠️ Key Features

• Abliterated: Mathematically modified to bypass standard refusal mechanisms, providing unrestricted responses.
• OpenVINO Optimized: Converted using optimum-intel for maximum performance on Intel CPUs, iGPUs, and Discrete GPUs (Arc B-series).
• INT8 Quantization: Compressed using NNCF to reduce VRAM usage and increase token throughput.

⚡ Performance Metrics (Intel Arc B580)

• Speed: ~50.0+ tokens/sec.
• TTFT: ~0.12s.
• VRAM Usage: Optimized for 12GB+ cards.

🚀 Usage Instructions

To run this model, ensure you have the optimum-intel package installed in your Python environment.

pip install "optimum-intel[openvino,nncf]"

from optimum.intel import OVModelForCausalLM

model_id = "PrimeryOP/gemma-2-abliterated-2B-ov-int8"
model = OVModelForCausalLM.from_pretrained(model_id, device="GPU")