Create README.md

README.md

@@ -0,0 +1,52 @@
+---
+license: mit
+base_model: OpenGVLab/InternVL3_5-38B-Flash
+tags:
+- fp8
+- quantization
+- llm-compressor
+- vllm
+- vision
+- multimodal
+pipeline_tag: image-text-to-text
+library_name: transformers
+---
+
+# InternVL3_5-38B-Flash-FP8
+
+This repository contains an **FP8 quantized version** of the multi-modal model [OpenGVLab/InternVL3_5-38B-Flash](https://huggingface.co/OpenGVLab/InternVL3_5-38B-Flash).
+
+## Model Details
+* **Base Model:** `OpenGVLab/InternVL3_5-38B-Flash`
+* **Quantization Format:** FP8 (E4M3) targeting `Linear` layers
+* **Quantization Engine:** `llmcompressor` (Post-Training Quantization / PTQ)
+* **Calibration Dataset:** 512 samples from `ultrachat-200k` (`train_sft`)
+* **License:** MIT
+
+## Method & Creation Process
+
+The model was quantized using Neural Magic's [`llm-compressor`](https://github.com/vllm-project/llm-compressor) framework. 
+
+Post-training quantization (PTQ) was applied directly to the language backbone (`model.language_model`). This strategy ensures that all heavy `Linear` projections in the main LLM layers are converted to FP8 for maximum speedup and reduced VRAM footprint, while preserving the vision architecture intact.
+
+### Quantization Recipe
+
+```python
+from llmcompressor import oneshot
+from llmcompressor.modifiers.quantization import QuantizationModifier
+
+recipe = QuantizationModifier(
+    targets="Linear",
+    scheme="FP8",
+    ignore=["lm_head"]
+)
+
+oneshot(
+    model=model.language_model, 
+    tokenizer=tokenizer,
+    dataset="ultrachat-200k",
+    splits="train_sft[:512]",
+    recipe=recipe,
+    max_seq_length=2048,
+    num_calibration_samples=512,
+)