File size: 739 Bytes
19089d1 c5bd2db 19089d1 38f4b1d 7259d33 b83031a 7259d33 b83031a 7259d33 b83031a 7259d33 b83031a 7259d33 b83031a 7259d33 b83031a 7259d33 b83031a 7259d33 b83031a 38f4b1d b83031a 38f4b1d b0266ff 0fc188e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 | ---
license: mit
language:
- en
tags:
- Q
- Q2
- Q3
- Q4
- Q5
- Q6
- Q12
- Q8
- Q16
- Q32
- Dront
---
DrontGPTQ is a simple quantizer for GPT-like models.
Supports:
2 bit ✅
3 bit ✅
4 bit ✅
5 bit ✅
6 bit ✅
7 bit ❌
8 bit ✅
12 bit ✅
16 bit ✅
32 bit ✅
Consists of just two files.
DrontQ.py – a very simple quantizer. Converts HF F32 to .dq format (Dront quant).
DQuse.py – a simple way to run .dq models.
*To use this, scroll to the end of the files; you will find the main variables there, which you should adjust to your needs.*
*Quantizes local models—specifically GPT-like ones.*
**Most Dront gpt-like models will also be released in DQ quantization! Especially the new VdrontV** |