Eclipse-Senpai commited on
Commit
fa90f27
·
verified ·
1 Parent(s): d5f1c1a

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +8 -8
README.md CHANGED
@@ -20,7 +20,7 @@ The model was trained on 10.01B tokens. Its looped decoder reuses a compact tran
20
 
21
  ## Introducing native effort levels
22
 
23
- Effort levels have traditionally been associated with large reasoning models, where they adjust a thinking budget or modify the surrounding prompt. min-spark brings the idea directly into the language model. Each effort level changes the model's internal computation by selecting a different number of passes through its shared core.
24
 
25
  | Effort | Character | Recommended use |
26
  |---|---|---|
@@ -28,7 +28,7 @@ Effort levels have traditionally been associated with large reasoning models, wh
28
  | `medium` | Balanced | General generation |
29
  | `high` | Most compute | Highest available quality |
30
 
31
- Effort is a generation-time choice. All three levels use the same weights and tokenizer.
32
 
33
  ## Usage
34
 
@@ -72,7 +72,7 @@ A lightweight Transformers-free generation script is included in the repository:
72
  python generate.py -p "The meaning of life is" -e high
73
  ```
74
 
75
- Generation currently runs one sequence at a time. Right-padded batches are supported for evaluation. KV caching is planned for a future release, so long generations recompute the prompt at each step. The context window is 512 tokens.
76
 
77
  ## Evaluation
78
 
@@ -82,7 +82,7 @@ min-spark reaches 69.19% on BLiMP at medium effort. ARC-Easy reaches 37.08%. ARC
82
 
83
  ### Effort levels
84
 
85
- The table shows how the model responds to additional internal computation. Grammar improves most clearly from low to medium effort. The common-sense tasks remain close across the three settings.
86
 
87
  ![min-spark accuracy by effort](charts/effort.svg)
88
 
@@ -92,7 +92,7 @@ The table shows how the model responds to additional internal computation. Gramm
92
  | min-spark-medium | 69.19% | 37.08% | 22.78% | 27.92% | 54.30% | 2.7747 |
93
  | min-spark-high | 69.18% | 37.08% | 22.87% | 27.91% | 54.35% | 2.7747 |
94
 
95
- ### Comparison with small models
96
 
97
  The comparison places min-spark alongside published results for GPT-S2-5M, SLM-10M, and michel-nano-v2. It covers the benchmarks reported across this group.
98
 
@@ -105,11 +105,11 @@ The comparison places min-spark alongside published results for GPT-S2-5M, SLM-1
105
  | [SLM-10M](https://huggingface.co/LiodonAI/SLM-10M) | 10M | — | 35.52% | **23.46%** | 27.40% | 57.07% |
106
  | [michel-nano-v2](https://huggingface.co/finnianx/michel-nano-v2) | 8M | **72.52%** | 35.90% | 21.84% | 27.40% | 56.75% |
107
 
108
- min-spark reaches 37.08% on ARC-Easy, the highest score in this comparison. Its 23.21% ARC-Challenge result is close to SLM-10M at 23.46%. HellaSwag reaches 27.92%, and PIQA reaches 54.35%. michel-nano-v2 records the highest BLiMP result in the group at 72.52%, while min-spark reaches 69.19%.
109
 
110
- ### Comparison with KeyLM
111
 
112
- KeyLM is a 75.25M-parameter language model trained on 18B tokens. min-spark uses less than one thirteenth of its parameter count while reaching a similar range on several small-model evaluations.
113
 
114
  KeyLM reports 29.9% on its ARC average, 29.7% on HellaSwag, and 60.0% on PIQA. min-spark reaches 37.08% on ARC-Easy, 23.21% on ARC-Challenge, 27.92% on HellaSwag, and 54.35% on PIQA. The comparison highlights the value of repeated computation in a compact model. min-spark closes much of the size gap on these tasks while retaining a 5.76M parameter footprint.
115
 
 
20
 
21
  ## Introducing native effort levels
22
 
23
+ Effort levels have usually been associated with large reasoning models, where they adjust the thinking token budget or modify the system prompt to change how hard the model thinks. min-spark brings the idea directly into the language model. Each effort level changes the model's internal computation by selecting a different number of passes through its shared core.
24
 
25
  | Effort | Character | Recommended use |
26
  |---|---|---|
 
28
  | `medium` | Balanced | General generation |
29
  | `high` | Most compute | Highest available quality |
30
 
31
+ Effort is a generation-time choice.
32
 
33
  ## Usage
34
 
 
72
  python generate.py -p "The meaning of life is" -e high
73
  ```
74
 
75
+ Generation currently runs one sequence at a time, and right-padded batches are supported for evaluation. This model currently does not have KV cache support so long generations recompute the prompt at each step which may result in slightly longer generations. The context window is 512 tokens.
76
 
77
  ## Evaluation
78
 
 
82
 
83
  ### Effort levels
84
 
85
+ The table shows how the model responds to additional internal computation. Grammar improves most clearly from low to medium effort while the common-sense tasks remain close across the three settings.
86
 
87
  ![min-spark accuracy by effort](charts/effort.svg)
88
 
 
92
  | min-spark-medium | 69.19% | 37.08% | 22.78% | 27.92% | 54.30% | 2.7747 |
93
  | min-spark-high | 69.18% | 37.08% | 22.87% | 27.91% | 54.35% | 2.7747 |
94
 
95
+ ### Benchmarks
96
 
97
  The comparison places min-spark alongside published results for GPT-S2-5M, SLM-10M, and michel-nano-v2. It covers the benchmarks reported across this group.
98
 
 
105
  | [SLM-10M](https://huggingface.co/LiodonAI/SLM-10M) | 10M | — | 35.52% | **23.46%** | 27.40% | 57.07% |
106
  | [michel-nano-v2](https://huggingface.co/finnianx/michel-nano-v2) | 8M | **72.52%** | 35.90% | 21.84% | 27.40% | 56.75% |
107
 
108
+ min-spark reaches 37.08% on ARC-Easy, the highest score in this comparison. Its 23.21% ARC-Challenge result is close to SLM-10M at 23.46%. HellaSwag reaches 27.92%, and PIQA reaches 54.35%.
109
 
110
+ ### More in Less Parameters
111
 
112
+ KeyLM was the previous model made by us, and is a 75.25M-parameter language model trained on 18B tokens. min-spark uses less than one thirteenth of its parameter count while reaching a similar range on several small-model evaluations.
113
 
114
  KeyLM reports 29.9% on its ARC average, 29.7% on HellaSwag, and 60.0% on PIQA. min-spark reaches 37.08% on ARC-Easy, 23.21% on ARC-Challenge, 27.92% on HellaSwag, and 54.35% on PIQA. The comparison highlights the value of repeated computation in a compact model. min-spark closes much of the size gap on these tasks while retaining a 5.76M parameter footprint.
115