README.md

---
license: mit
base_model: deepseek-ai/DeepSeek-OCR
tags:
- quantization
- int8
- uniform-quantization
- model-compression
---

# Uniform INT8 Quantized DeepSeek-OCR

This model is a uniformly quantized version of [deepseek-ai/DeepSeek-OCR](https://huggingface.co/deepseek-ai/DeepSeek-OCR).

## Quantization Details

- **Method**: Uniform INT8 quantization
- **Quantized Layers**: 2342
- **Vision Layers**: 96 @ 8-bit
- **Language Layers**: 2197 @ 8-bit
- **Average Bit-width**: 8.00
- **Original Size**: 6363.12 MB
- **Compressed Size**: 3351.56 MB
- **Compression Ratio**: 1.90x

## Model Files

- `quantized_weights.pt`: Quantized model weights
- `quantization_info.json`: Layer-wise quantization configuration
- `layer_configs.json`: Detailed layer configurations
- `compression_stats.json`: Compression statistics
- `layer_analysis.json`: Modality analysis (vision/language/other)

## Usage

```python
import torch
from transformers import AutoTokenizer

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("SamMikaelson/deepseek-ocr-int8-quantized", trust_remote_code=True)

# Load quantized weights
state_dict = torch.load("quantized_weights.pt")
# Note: You'll need the QuantizedLinear class to properly load and use this model
```

## Baseline Characteristics

This uniform quantization approach:
- Applies the **same 8-bit** quantization to ALL layers
- **Does not distinguish** between vision and language modalities
- Serves as a **baseline** for comparison with modality-aware methods

## Citation

If you use this model, please cite the original model and mention the uniform quantization approach.