anurag051194 commited on
Commit
7a0b3f6
·
verified ·
1 Parent(s): 4371a3f

Update TwIL-LM3: weights, tokenizer and model card

Browse files
Files changed (1) hide show
  1. README.md +10 -5
README.md CHANGED
@@ -211,6 +211,7 @@ carried into the GGUF metadata, so chat mode works without extra flags.
211
  |---|---|---:|---:|---|
212
  | `TwIL-LM3-Q4_K_M.gguf` | Q4_K_M | 1.78 GiB | 4.96 | recommended default; runs on CPU or 4 GB of VRAM |
213
  | `TwIL-LM3-Q8_0.gguf` | Q8_0 | 3.05 GiB | 8.50 | near-lossless, for quality-sensitive use |
 
214
 
215
  ```bash
216
  llama-cli -m TwIL-LM3-Q4_K_M.gguf -cnv --temp 0 -n 2048
@@ -221,11 +222,15 @@ evaluation is greedy while the packaged sampling defaults are not. And leave the
221
  budget large — 2048 tokens or more — since the model emits a `<think>` block before answering
222
  and a short budget truncates it, which costs far more accuracy than the quantization does.
223
 
224
- Q8_0 was produced directly by `convert_hf_to_gguf.py` from the released bf16 weights; Q4_K_M was
225
- produced from an F16 conversion with `llama-quantize`, without an importance matrix. Both builds
226
- were smoke-tested for load and generation on CPU. The published Track A and Track B numbers were
227
- measured on the **bf16** weights through vLLM, not on these GGUF builds, so expect small
228
- deviations at Q4_K_M that have not been quantified here.
 
 
 
 
229
 
230
  ## How it was built
231
 
 
211
  |---|---|---:|---:|---|
212
  | `TwIL-LM3-Q4_K_M.gguf` | Q4_K_M | 1.78 GiB | 4.96 | recommended default; runs on CPU or 4 GB of VRAM |
213
  | `TwIL-LM3-Q8_0.gguf` | Q8_0 | 3.05 GiB | 8.50 | near-lossless, for quality-sensitive use |
214
+ | `TwIL-LM3-F16.gguf` | F16 | 5.73 GiB | 16.00 | unquantized, for requantization or reference runs |
215
 
216
  ```bash
217
  llama-cli -m TwIL-LM3-Q4_K_M.gguf -cnv --temp 0 -n 2048
 
222
  budget large — 2048 tokens or more — since the model emits a `<think>` block before answering
223
  and a short budget truncates it, which costs far more accuracy than the quantization does.
224
 
225
+ F16 and Q8_0 were produced directly by `convert_hf_to_gguf.py` from the released bf16 weights;
226
+ Q4_K_M was quantized from the F16 build with `llama-quantize`, without an importance matrix. All
227
+ three were smoke-tested for load and generation on CPU. Note that F16 is not bit-identical to the
228
+ released weights: bf16 and f16 carry the same 16 bits but trade exponent range against mantissa
229
+ precision, so the conversion is a narrowing one, in practice negligible for inference.
230
+
231
+ The published Track A and Track B numbers were measured on the **bf16** weights through vLLM, not
232
+ on any of these GGUF builds, so expect small deviations — most likely at Q4_K_M — that have not
233
+ been quantified here.
234
 
235
  ## How it was built
236