|
|
--- |
|
|
license: mit |
|
|
base_model: deepseek-ai/DeepSeek-OCR |
|
|
tags: |
|
|
- quantization |
|
|
- int8 |
|
|
- uniform-quantization |
|
|
- model-compression |
|
|
--- |
|
|
|
|
|
# Uniform INT8 Quantized DeepSeek-OCR |
|
|
|
|
|
This model is a uniformly quantized version of [deepseek-ai/DeepSeek-OCR](https://huggingface.co/deepseek-ai/DeepSeek-OCR). |
|
|
|
|
|
## Quantization Details |
|
|
|
|
|
- **Method**: Uniform INT8 quantization |
|
|
- **Quantized Layers**: 2342 |
|
|
- **Vision Layers**: 96 @ 8-bit |
|
|
- **Language Layers**: 2197 @ 8-bit |
|
|
- **Average Bit-width**: 8.00 |
|
|
- **Original Size**: 6363.12 MB |
|
|
- **Compressed Size**: 3351.56 MB |
|
|
- **Compression Ratio**: 1.90x |
|
|
|
|
|
## Model Files |
|
|
|
|
|
- `quantized_weights.pt`: Quantized model weights |
|
|
- `quantization_info.json`: Layer-wise quantization configuration |
|
|
- `layer_configs.json`: Detailed layer configurations |
|
|
- `compression_stats.json`: Compression statistics |
|
|
- `layer_analysis.json`: Modality analysis (vision/language/other) |
|
|
|
|
|
## Usage |
|
|
|
|
|
```python |
|
|
import torch |
|
|
from transformers import AutoTokenizer |
|
|
|
|
|
# Load tokenizer |
|
|
tokenizer = AutoTokenizer.from_pretrained("SamMikaelson/deepseek-ocr-int8-quantized", trust_remote_code=True) |
|
|
|
|
|
# Load quantized weights |
|
|
state_dict = torch.load("quantized_weights.pt") |
|
|
# Note: You'll need the QuantizedLinear class to properly load and use this model |
|
|
``` |
|
|
|
|
|
## Baseline Characteristics |
|
|
|
|
|
This uniform quantization approach: |
|
|
- Applies the **same 8-bit** quantization to ALL layers |
|
|
- **Does not distinguish** between vision and language modalities |
|
|
- Serves as a **baseline** for comparison with modality-aware methods |
|
|
|
|
|
## Citation |
|
|
|
|
|
If you use this model, please cite the original model and mention the uniform quantization approach. |
|
|
|
