| --- |
| license: mit |
| language: |
| - en |
| tags: |
| - Q |
| - Q2 |
| - Q3 |
| - Q4 |
| - Q5 |
| - Q6 |
| - Q12 |
| - Q8 |
| - Q16 |
| - Q32 |
| - Dront |
| --- |
| |
| DrontGPTQ is a simple quantizer for GPT-like models. |
|
|
| Supports: |
|
|
| 2 bit ✅ |
|
|
| 3 bit ✅ |
| |
| 4 bit ✅ |
| |
| 5 bit ✅ |
| |
| 6 bit ✅ |
|
|
| 7 bit ❌ |
| |
| 8 bit ✅ |
| |
| 12 bit ✅ |
| |
| 16 bit ✅ |
| |
| 32 bit ✅ |
|
|
| Consists of just two files. |
|
|
| DrontQ.py – a very simple quantizer. Converts HF F32 to .dq format (Dront quant). |
|
|
| DQuse.py – a simple way to run .dq models. |
|
|
|
|
| *To use this, scroll to the end of the files; you will find the main variables there, which you should adjust to your needs.* |
| *Quantizes local models—specifically GPT-like ones.* |
| **Most Dront gpt-like models will also be released in DQ quantization! Especially the new VdrontV** |
