---
license: apache-2.0
base_model:
- HuggingFaceTB/SmolLM3-3B
pipeline_tag: text-generation
library_name: transformers
---

**SmolLM3‑3B • Quantized**

---

### 🚀 Model Description

This is an **int8 quantized version** of **SmolLM3–3B**, a highly efficient, open-source 3 B parameter LLM. 
It delivers nearly state-of-the-art multilingual reasoning and long-context performance (up to 128k tokens) with drastically reduced 
memory usage and inference cost, enabling fast deployment on mid‑range GPUs and edge devices.

---


### 📏 Quantization Details


* **Library:**  torchao 
* **Precision:** int8 weights and activations 
* **Benefits:** \~50–75% reduction in VRAM usage, enabling 12–16 GB GPU usage, with minimal performance drop on reasoning, coding, and long-context abilities

---

### 🎯 Intended Use

Ideal for:

* Scenarios requiring **fast LLM inference** under constrained VRAM (e.g. small servers or laptops)
* **Multilingual reasoning** tasks, chain-of-thought logic, and long-context document understanding
* Deployments of dual-mode (think/no\_think) conversational agents
* Research into efficient LLM deployment and quantization techniques

---

### ⚠️ Limitations

* Slight performance loss compared to full-precision SmolLM3‑3B
* Requires proper benchmarking in your actual environment
* Continues to exhibit standard LLM risks: hallucination, bias, etc.
* Quant performance may vary across languages or context lengths