Spaces:
Configuration error
Configuration error
Update README.md
Browse files
README.md
CHANGED
|
@@ -23,11 +23,6 @@ Two model families:
|
|
| 23 |
- **LittleLearner** β pretrained on the Kβ5-filtered corpus.
|
| 24 |
- **Unfiltered** β pretrained on the same corpus without the Kβ5 filter (baseline general corpus).
|
| 25 |
|
| 26 |
-
Post-training data uses the same two names:
|
| 27 |
-
|
| 28 |
-
- **K-5** β the Kβ5-filtered instruction/SFT corpus (matched to LittleLearner pretraining).
|
| 29 |
-
- **Unfiltered** β the unfiltered instruction/SFT corpus.
|
| 30 |
-
|
| 31 |
---
|
| 32 |
|
| 33 |
## Released checkpoints
|
|
@@ -49,19 +44,9 @@ Encoded as **(pretrain β post-training)**:
|
|
| 49 |
| Pretrain β Post-training | Description | Repo |
|
| 50 |
|---|---|---|
|
| 51 |
| LittleLearner β K-5 | Matched-pipeline LL post-training | `littlelearner/littlelearner-5b-bounded-sft-chatty` |
|
| 52 |
-
| LittleLearner β Unfiltered | Hybrid: LL pretrain, Unfiltered post-training data | `<...>` |
|
| 53 |
| Unfiltered β Unfiltered | Matched-pipeline Unfiltered post-training | `littlelearner/littlelearner-5b-unbounded-sft-chatty` |
|
| 54 |
|
| 55 |
|
| 56 |
-
---
|
| 57 |
-
|
| 58 |
-
## Which checkpoint should I use?
|
| 59 |
-
|
| 60 |
-
- **Studying the effect of the K-5 filter on pretraining** β paired base checkpoints at matched sizes (`β¦-bounded` vs `β¦-unbounded`).
|
| 61 |
-
- **K-5 math / educational assistant** β LittleLearner β K-5 SFT+GRPO chatty.
|
| 62 |
-
- **General-purpose small assistant** β Unfiltered β Unfiltered SFT+GRPO chatty.
|
| 63 |
-
- **Effect of post-training data on a filtered pretrain** β compare the two `LittleLearner β β¦` SFT+GRPO variants.
|
| 64 |
-
|
| 65 |
---
|
| 66 |
|
| 67 |
## Important inference notes
|
|
|
|
| 23 |
- **LittleLearner** β pretrained on the Kβ5-filtered corpus.
|
| 24 |
- **Unfiltered** β pretrained on the same corpus without the Kβ5 filter (baseline general corpus).
|
| 25 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
---
|
| 27 |
|
| 28 |
## Released checkpoints
|
|
|
|
| 44 |
| Pretrain β Post-training | Description | Repo |
|
| 45 |
|---|---|---|
|
| 46 |
| LittleLearner β K-5 | Matched-pipeline LL post-training | `littlelearner/littlelearner-5b-bounded-sft-chatty` |
|
|
|
|
| 47 |
| Unfiltered β Unfiltered | Matched-pipeline Unfiltered post-training | `littlelearner/littlelearner-5b-unbounded-sft-chatty` |
|
| 48 |
|
| 49 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
---
|
| 51 |
|
| 52 |
## Important inference notes
|