Create README.md

README.md

@@ -0,0 +1,79 @@
+---
+base_model: facebook/Perception-LM-1B
+base_model_relation: quantized
+tags:
+- quantized
+- int4
+- perception_lm
+- language-model
+library_name: transformers
+---
+
+# Perception-LM-1B Int4-bit Quantized
+
+This repository contains **a 4-bit quantized version** of Perception-LM-1B — optimized for reduced memory usage and faster inference, while retaining most of the capabilities of the full-precision model.
+
+## ⚙️ Model Description
+
+- **Base model**: `facebook/Perception-LM-1B`   
+- **Quantization**: 4-bit integer quantization (INT4).  
+- **Purpose**: Provide a lighter, more resource-efficient variant for inference, deployment on resource-constrained hardware, or quick prototyping.
+
+## ✅ Intended Use & Use Cases
+
+This quantized model is suited for:
+
+- Fast inference when GPU/CPU memory or VRAM is limited  
+- Prototyping or integrating into applications where resource efficiency matters  
+- Use in research or production pipelines where quantization is acceptable  
+
+### ⚠️ Limitations (Things to Watch Out For)
+
+- Quantization can introduce **slight degradation** compared to full-precision: responses may be less accurate or fluent in edge cases.  
+- Not recommended for use-cases requiring **maximum fidelity** (e.g. very fine-grained reasoning, sensitive safety-critical tasks).  
+- Performance may depend on hardware: quantized weights may require specific inference settings (device map, memory constraints).  
+
+## 🔄 How to Use
+
+Here is an example of how you can load the quantized model using `transformers`:
+
+```python
+
+import torch
+from transformers import AutoProcessor, AutoModelForImageTextToText
+
+model_id = "Dhruvil03/Perception-LM-1B-Int4bit"
+
+processor = AutoProcessor.from_pretrained(model_id)
+
+model = AutoModelForImageTextToText.from_pretrained(
+    model_id,
+    torch_dtype=torch.float16
+).to("cuda").eval()
+
+conversation = [{
+    "role": "user",
+    "content": [
+        {"type": "video", "url": "test.mp4"},
+        {"type": "text", "text": "Can you describe the video in detail?"},
+    ],
+}]
+
+inputs = processor.apply_chat_template(
+    conversation,
+    num_frames=16,   # change number of frames as per the CUDA memory availability
+    add_generation_prompt=True,
+    tokenize=True,
+    return_dict=True,
+    return_tensors="pt",
+    video_load_backend="pyav",
+)
+
+inputs = {k: (v.to("cuda") if hasattr(v, "to") else v) for k, v in inputs.items()}
+
+with torch.inference_mode():
+    outputs = model.generate(**inputs, max_new_tokens=64)
+
+ilen = inputs["input_ids"].shape[1]
+decoded = processor.batch_decode(outputs[:, ilen:], skip_special_tokens=True)
+print(decoded[0])