ProCreations commited on
Commit
e7f336a
·
verified ·
1 Parent(s): cef6e5f

Document packed spellchecker metrics and usage

Browse files
Files changed (1) hide show
  1. README.md +65 -4
README.md CHANGED
@@ -1,13 +1,74 @@
1
  ---
2
  license: other
3
  license_name: lfm-open-license-v1.0
4
- license_link: https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M/blob/main/LICENSE
5
  library_name: minima-lfm
6
  base_model: LiquidAI/LFM2.5-Encoder-350M-Spellchecker
 
 
 
 
 
 
7
  ---
8
 
9
- # Minima W1.58A8 artifact
10
 
11
- This repository stores a packed ternary model for [SSHDotCodes/minima](https://github.com/SSHDotCodes/minima). Install that package and load it with `MinimaModel.from_pretrained(...)`.
 
 
12
 
13
- Matrix weights use logical {-1, 0, +1} values in the I2_S runtime format. See `minima_config.json` for the exact group size, recovery rank, and context limit.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: other
3
  license_name: lfm-open-license-v1.0
4
+ license_link: https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M-Spellchecker/blob/main/LICENSE
5
  library_name: minima-lfm
6
  base_model: LiquidAI/LFM2.5-Encoder-350M-Spellchecker
7
+ pipeline_tag: token-classification
8
+ tags:
9
+ - ternary
10
+ - 1.58-bit
11
+ - grammatical-error-correction
12
+ - spell-check
13
  ---
14
 
15
+ # Minima Spellcheck
16
 
17
+ Packed W1.58A8 adaptation of
18
+ [LiquidAI/LFM2.5-Encoder-350M-Spellchecker](https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M-Spellchecker),
19
+ built with [SSHDotCodes/minima](https://github.com/SSHDotCodes/minima).
20
 
21
+ - Logical matrix values: `{-1, 0, +1}` (1.585 bits)
22
+ - Physical artifact format: I2_S, four trits per byte
23
+ - Group size: 32; recovery rank: 128
24
+ - Full encoder context: 8,192 tokens
25
+ - Weight file: 226.0 MB
26
+ - CPU demo: [Minima Spellcheck](https://huggingface.co/spaces/ProCreations/minima-spellcheck)
27
+
28
+ ## Use
29
+
30
+ ```bash
31
+ pip install "minima-lfm @ git+https://github.com/SSHDotCodes/minima.git"
32
+ ```
33
+
34
+ ```python
35
+ from minima import MinimaModel
36
+
37
+ model = MinimaModel.from_pretrained("ProCreations/minima-spellcheck", device="cpu")
38
+ print(model.correct(
39
+ ["I has went to the stor yesterday ."],
40
+ max_iter=4,
41
+ min_error_prob=0.0,
42
+ rerank=False,
43
+ ))
44
+ # ['I went to the store yesterday .']
45
+ ```
46
+
47
+ CPU inference defaults to a one-time FBGEMM INT8 packing of each effective
48
+ ternary-plus-recovery matrix. Set `MINIMA_CPU_BACKEND=i2s` to execute the strict
49
+ 2-bit AVX2/NEON kernel instead.
50
+
51
+ ## Validation
52
+
53
+ The 1,000-step distillation run used held-out corrupted FineWeb text and the
54
+ LiquidAI model as teacher. With both models' optional dense reranker disabled:
55
+
56
+ | Diagnostic | Result |
57
+ |---|---:|
58
+ | Tag top-1 agreement within teacher candidates | 99.51% |
59
+ | Error-detection top-1 agreement | 99.41% |
60
+ | Exact correction agreement, held-out + reference examples | 70.0% (14/20) |
61
+ | Exact agreement on the four published-style examples | 100% (4/4) |
62
+
63
+ The Space smoke test corrected `I has went to the stor yesterday .` to
64
+ `I went to the store yesterday.` and returned a warmed CPU latency of 110 ms.
65
+ These are teacher-agreement diagnostics, not an ERRANT benchmark. The upstream
66
+ optional reranker is deliberately disabled because it contains a separate
67
+ 1.42 GB dense encoder, which would invalidate this model's CPU memory profile.
68
+
69
+ See `spellcheck_report.json` for the full immutable training and evaluation log.
70
+
71
+ ## License
72
+
73
+ The weights remain subject to the LFM Open License v1.0 shipped in this repository.
74
+ The Minima runtime code is MIT licensed.