Instructions to use CyanMonkey/Test-Subject-3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use CyanMonkey/Test-Subject-3 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="CyanMonkey/Test-Subject-3")# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("CyanMonkey/Test-Subject-3", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use CyanMonkey/Test-Subject-3 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "CyanMonkey/Test-Subject-3" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "CyanMonkey/Test-Subject-3", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/CyanMonkey/Test-Subject-3
- SGLang
How to use CyanMonkey/Test-Subject-3 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "CyanMonkey/Test-Subject-3" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "CyanMonkey/Test-Subject-3", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "CyanMonkey/Test-Subject-3" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "CyanMonkey/Test-Subject-3", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use CyanMonkey/Test-Subject-3 with Docker Model Runner:
docker model run hf.co/CyanMonkey/Test-Subject-3
round 11 step 5500 (19.30%)
Browse files- README.md +14 -14
- config.json +5 -5
- model.safetensors +1 -1
- progress.json +9 -9
- training_meta.json +32 -32
README.md
CHANGED
|
@@ -13,11 +13,11 @@ tags:
|
|
| 13 |
|
| 14 |
# Oleander-150M
|
| 15 |
|
| 16 |
-
**Training checkpoint** (`round
|
| 17 |
|
| 18 |
```text
|
| 19 |
-
steps █████░░░░░░░░░░░░░░░░░░░░░░░
|
| 20 |
-
tokens █████░░░░░░░░░░░░░░░░░░░░░░░
|
| 21 |
phase tst (TST bag training ends at step 9,000)
|
| 22 |
```
|
| 23 |
|
|
@@ -27,17 +27,17 @@ Trained with the Oleander marimo trainer (molab): hybrid **GDN2** (gated delta-r
|
|
| 27 |
|
| 28 |
| | |
|
| 29 |
|---|---|
|
| 30 |
-
| Step | **5,
|
| 31 |
-
| % complete (steps) | **
|
| 32 |
-
| Compute tokens seen | **2.
|
| 33 |
-
| Raw tokens seen (TST bags unpacked) |
|
| 34 |
-
| Cluster tokens (all 1 worker(s)) | 2.
|
| 35 |
| Phase | `tst` |
|
| 36 |
-
| Val loss / ppl | **6.
|
| 37 |
-
| Train loss (latest) | 5.
|
| 38 |
-
| Throughput | 76,
|
| 39 |
-
| Session elapsed | 0d
|
| 40 |
-
| Checkpoint written | 2026-11-
|
| 41 |
|
| 42 |
## How to load
|
| 43 |
|
|
@@ -98,6 +98,6 @@ Oleander-150M (28L = 7 x [GDN2 · GDN2 · GDN2 · GQA] · d=576 · SwiGLU(1600)
|
|
| 98 |
|
| 99 |
## Notes
|
| 100 |
|
| 101 |
-
- This is a **work-in-progress** checkpoint at
|
| 102 |
- The tokenizer comes from the base model the run started from; the vocabulary is a plain BPE model and is not extended during training.
|
| 103 |
- `training_meta.json` is regenerated on every push and is the authoritative record of how far training got.
|
|
|
|
| 13 |
|
| 14 |
# Oleander-150M
|
| 15 |
|
| 16 |
+
**Training checkpoint** (`round 11`) — step **5,500 / 28,500** · **19.30%** of the planned run.
|
| 17 |
|
| 18 |
```text
|
| 19 |
+
steps █████░░░░░░░░░░░░░░░░░░░░░░░ 19.30% 5,500 / 28,500
|
| 20 |
+
tokens █████░░░░░░░░░░░░░░░░░░░░░░░ 19.30% 2.88B / 14.94B compute
|
| 21 |
phase tst (TST bag training ends at step 9,000)
|
| 22 |
```
|
| 23 |
|
|
|
|
| 27 |
|
| 28 |
| | |
|
| 29 |
|---|---|
|
| 30 |
+
| Step | **5,500** / 28,500 |
|
| 31 |
+
| % complete (steps) | **19.30%** |
|
| 32 |
+
| Compute tokens seen | **2.88B** / 14.94B (19.30%) |
|
| 33 |
+
| Raw tokens seen (TST bags unpacked) | 17.30B / 38.54B |
|
| 34 |
+
| Cluster tokens (all 1 worker(s)) | 2.88B / 14.94B compute |
|
| 35 |
| Phase | `tst` |
|
| 36 |
+
| Val loss / ppl | **6.1634** / **475.06** |
|
| 37 |
+
| Train loss (latest) | 5.2557 |
|
| 38 |
+
| Throughput | 76,462 compute tok/s · 0.143 steps/s |
|
| 39 |
+
| Session elapsed | 0d 10h 46m · ETA 1d 20h 40m |
|
| 40 |
+
| Checkpoint written | 2026-11-15T05:20:18Z |
|
| 41 |
|
| 42 |
## How to load
|
| 43 |
|
|
|
|
| 98 |
|
| 99 |
## Notes
|
| 100 |
|
| 101 |
+
- This is a **work-in-progress** checkpoint at 19.30% of the planned run, not a finished model: expect repetition and incoherence, especially with few-shot prompts.
|
| 102 |
- The tokenizer comes from the base model the run started from; the vocabulary is a plain BPE model and is not extended during training.
|
| 103 |
- `training_meta.json` is regenerated on every push and is the authoritative record of how far training got.
|
config.json
CHANGED
|
@@ -113,12 +113,12 @@
|
|
| 113 |
},
|
| 114 |
"model_name": "Oleander-150M",
|
| 115 |
"training": {
|
| 116 |
-
"step":
|
| 117 |
-
"kind": "round
|
| 118 |
"max_steps": 28500,
|
| 119 |
-
"percent_complete":
|
| 120 |
-
"tokens_seen":
|
| 121 |
-
"compute_tokens_seen":
|
| 122 |
},
|
| 123 |
"transformers_version": "5.14.1",
|
| 124 |
"torch_dtype": "float32"
|
|
|
|
| 113 |
},
|
| 114 |
"model_name": "Oleander-150M",
|
| 115 |
"training": {
|
| 116 |
+
"step": 5500,
|
| 117 |
+
"kind": "round 11",
|
| 118 |
"max_steps": 28500,
|
| 119 |
+
"percent_complete": 19.298,
|
| 120 |
+
"tokens_seen": 17301504000,
|
| 121 |
+
"compute_tokens_seen": 2883584000
|
| 122 |
},
|
| 123 |
"transformers_version": "5.14.1",
|
| 124 |
"torch_dtype": "float32"
|
model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 599970860
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:eb0bcde559024ad1a9eafb6e2bb332da934b7123095b6166b64bc899a371ea59
|
| 3 |
size 599970860
|
progress.json
CHANGED
|
@@ -1,15 +1,15 @@
|
|
| 1 |
{
|
| 2 |
-
"step":
|
| 3 |
"max_steps": 28500,
|
| 4 |
-
"percent_complete":
|
| 5 |
-
"tokens_seen":
|
| 6 |
-
"compute_tokens_seen":
|
| 7 |
"compute_tokens_total": 14942208000,
|
| 8 |
-
"percent_tokens":
|
| 9 |
-
"val_loss": 6.
|
| 10 |
-
"val_ppl":
|
| 11 |
"params_M": 149.976,
|
| 12 |
-
"pushed_at": "2026-11-
|
| 13 |
"architecture": "Oleander-150M",
|
| 14 |
"features": [
|
| 15 |
"GDN2 (gated delta net 2) x21: 9h x 64, conv4",
|
|
@@ -25,5 +25,5 @@
|
|
| 25 |
],
|
| 26 |
"optimizer": "muon_adamw",
|
| 27 |
"phase": "tst",
|
| 28 |
-
"kind": "round
|
| 29 |
}
|
|
|
|
| 1 |
{
|
| 2 |
+
"step": 5500,
|
| 3 |
"max_steps": 28500,
|
| 4 |
+
"percent_complete": 19.298,
|
| 5 |
+
"tokens_seen": 17301504000,
|
| 6 |
+
"compute_tokens_seen": 2883584000,
|
| 7 |
"compute_tokens_total": 14942208000,
|
| 8 |
+
"percent_tokens": 19.298,
|
| 9 |
+
"val_loss": 6.163434945046902,
|
| 10 |
+
"val_ppl": 475.0570704090034,
|
| 11 |
"params_M": 149.976,
|
| 12 |
+
"pushed_at": "2026-11-15T05:20:18Z",
|
| 13 |
"architecture": "Oleander-150M",
|
| 14 |
"features": [
|
| 15 |
"GDN2 (gated delta net 2) x21: 9h x 64, conv4",
|
|
|
|
| 25 |
],
|
| 26 |
"optimizer": "muon_adamw",
|
| 27 |
"phase": "tst",
|
| 28 |
+
"kind": "round 11"
|
| 29 |
}
|
training_meta.json
CHANGED
|
@@ -1,10 +1,10 @@
|
|
| 1 |
{
|
| 2 |
"format": "oleander-training-meta/1",
|
| 3 |
-
"generated_at": "2026-11-
|
| 4 |
-
"kind": "round
|
| 5 |
-
"step":
|
| 6 |
-
"tokens_seen":
|
| 7 |
-
"pushed_at": "2026-11-
|
| 8 |
"params_M": 149.976,
|
| 9 |
"architecture": "Oleander-150M",
|
| 10 |
"features": [
|
|
@@ -20,55 +20,55 @@
|
|
| 20 |
"TST (token superposition training, bag 6)"
|
| 21 |
],
|
| 22 |
"optimizer": "muon_adamw",
|
| 23 |
-
"val_loss": 6.
|
| 24 |
-
"val_ppl":
|
| 25 |
"run": {
|
| 26 |
"model_name": "Oleander-150M",
|
| 27 |
"architecture_detail": "Oleander-150M (28L = 7 x [GDN2 \u00b7 GDN2 \u00b7 GDN2 \u00b7 GQA] \u00b7 d=576 \u00b7 SwiGLU(1600) \u00b7 tied 32,770 vocab)",
|
| 28 |
"session_id": "20261114_181817_2de909",
|
| 29 |
"repo_id": "CyanMonkey/Test-Subject-3",
|
| 30 |
"started_at": "2026-11-14T18:33:29Z",
|
| 31 |
-
"session_elapsed_s":
|
| 32 |
-
"last_update_at": "2026-11-
|
| 33 |
-
"steps_per_second": 0.
|
| 34 |
-
"compute_tokens_per_second":
|
| 35 |
-
"eta": "1d
|
| 36 |
-
"checkpoint_kind": "round
|
| 37 |
},
|
| 38 |
"progress": {
|
| 39 |
-
"step":
|
| 40 |
"max_steps": 28500,
|
| 41 |
-
"step_fraction": 0.
|
| 42 |
-
"percent_complete":
|
| 43 |
-
"tokens_fraction": 0.
|
| 44 |
-
"percent_tokens":
|
| 45 |
"phase": "tst",
|
| 46 |
-
"phase_fraction": 0.
|
| 47 |
"tst_switch_step": 9000,
|
| 48 |
"tst_switch_applied": false
|
| 49 |
},
|
| 50 |
"tokens": {
|
| 51 |
-
"compute_tokens_seen":
|
| 52 |
"compute_tokens_total": 14942208000,
|
| 53 |
-
"raw_tokens_seen":
|
| 54 |
"raw_tokens_total": 38535168000,
|
| 55 |
"raw_over_compute_ratio": 6.0,
|
| 56 |
-
"samples_seen":
|
| 57 |
-
"rows_consumed":
|
| 58 |
"compute_tokens_per_step": 524288,
|
| 59 |
"cluster": {
|
| 60 |
"n_workers": 1,
|
| 61 |
-
"compute_tokens_seen_all_workers":
|
| 62 |
-
"raw_tokens_seen_all_workers":
|
| 63 |
"compute_tokens_total_planned_all_workers": 14942208000,
|
| 64 |
"raw_tokens_total_planned_all_workers": 38535168000
|
| 65 |
}
|
| 66 |
},
|
| 67 |
"metrics": {
|
| 68 |
-
"train_loss": 5.
|
| 69 |
-
"val_loss": 6.
|
| 70 |
-
"val_ppl":
|
| 71 |
-
"grad_norm": 0.
|
| 72 |
"lr_mult": 1.0
|
| 73 |
},
|
| 74 |
"optimization": {
|
|
@@ -151,7 +151,7 @@
|
|
| 151 |
"doc_masking": "auto",
|
| 152 |
"shuffle_block_rows": 192,
|
| 153 |
"seed": 42,
|
| 154 |
-
"epochs_consumed_this_worker": 1.
|
| 155 |
},
|
| 156 |
"hardware": {
|
| 157 |
"device": "cuda",
|
|
@@ -165,7 +165,7 @@
|
|
| 165 |
"peak_gb": 39.36
|
| 166 |
},
|
| 167 |
"checkpoint": {
|
| 168 |
-
"kind": "round
|
| 169 |
"weights_file": "model.safetensors",
|
| 170 |
"dtype": "float32",
|
| 171 |
"config_file": "config.json",
|
|
@@ -196,6 +196,6 @@
|
|
| 196 |
"support_repo_id": "CyanMonkey/TestSubject-3-support"
|
| 197 |
},
|
| 198 |
"extra": {
|
| 199 |
-
"diloco_round":
|
| 200 |
}
|
| 201 |
}
|
|
|
|
| 1 |
{
|
| 2 |
"format": "oleander-training-meta/1",
|
| 3 |
+
"generated_at": "2026-11-15T05:20:18Z",
|
| 4 |
+
"kind": "round 11",
|
| 5 |
+
"step": 5500,
|
| 6 |
+
"tokens_seen": 17301504000,
|
| 7 |
+
"pushed_at": "2026-11-15T05:20:18Z",
|
| 8 |
"params_M": 149.976,
|
| 9 |
"architecture": "Oleander-150M",
|
| 10 |
"features": [
|
|
|
|
| 20 |
"TST (token superposition training, bag 6)"
|
| 21 |
],
|
| 22 |
"optimizer": "muon_adamw",
|
| 23 |
+
"val_loss": 6.163434945046902,
|
| 24 |
+
"val_ppl": 475.0570704090034,
|
| 25 |
"run": {
|
| 26 |
"model_name": "Oleander-150M",
|
| 27 |
"architecture_detail": "Oleander-150M (28L = 7 x [GDN2 \u00b7 GDN2 \u00b7 GDN2 \u00b7 GQA] \u00b7 d=576 \u00b7 SwiGLU(1600) \u00b7 tied 32,770 vocab)",
|
| 28 |
"session_id": "20261114_181817_2de909",
|
| 29 |
"repo_id": "CyanMonkey/Test-Subject-3",
|
| 30 |
"started_at": "2026-11-14T18:33:29Z",
|
| 31 |
+
"session_elapsed_s": 38809.0,
|
| 32 |
+
"last_update_at": "2026-11-15T05:20:17Z",
|
| 33 |
+
"steps_per_second": 0.143,
|
| 34 |
+
"compute_tokens_per_second": 76461.7,
|
| 35 |
+
"eta": "1d 20h 40m",
|
| 36 |
+
"checkpoint_kind": "round 11"
|
| 37 |
},
|
| 38 |
"progress": {
|
| 39 |
+
"step": 5500,
|
| 40 |
"max_steps": 28500,
|
| 41 |
+
"step_fraction": 0.192982,
|
| 42 |
+
"percent_complete": 19.298,
|
| 43 |
+
"tokens_fraction": 0.192982,
|
| 44 |
+
"percent_tokens": 19.298,
|
| 45 |
"phase": "tst",
|
| 46 |
+
"phase_fraction": 0.611111,
|
| 47 |
"tst_switch_step": 9000,
|
| 48 |
"tst_switch_applied": false
|
| 49 |
},
|
| 50 |
"tokens": {
|
| 51 |
+
"compute_tokens_seen": 2883584000,
|
| 52 |
"compute_tokens_total": 14942208000,
|
| 53 |
+
"raw_tokens_seen": 17301504000,
|
| 54 |
"raw_tokens_total": 38535168000,
|
| 55 |
"raw_over_compute_ratio": 6.0,
|
| 56 |
+
"samples_seen": 16896000,
|
| 57 |
+
"rows_consumed": 16896000,
|
| 58 |
"compute_tokens_per_step": 524288,
|
| 59 |
"cluster": {
|
| 60 |
"n_workers": 1,
|
| 61 |
+
"compute_tokens_seen_all_workers": 2883584000,
|
| 62 |
+
"raw_tokens_seen_all_workers": 17301504000,
|
| 63 |
"compute_tokens_total_planned_all_workers": 14942208000,
|
| 64 |
"raw_tokens_total_planned_all_workers": 38535168000
|
| 65 |
}
|
| 66 |
},
|
| 67 |
"metrics": {
|
| 68 |
+
"train_loss": 5.255654811859131,
|
| 69 |
+
"val_loss": 6.163434945046902,
|
| 70 |
+
"val_ppl": 475.0570704090034,
|
| 71 |
+
"grad_norm": 0.25617101788520813,
|
| 72 |
"lr_mult": 1.0
|
| 73 |
},
|
| 74 |
"optimization": {
|
|
|
|
| 151 |
"doc_masking": "auto",
|
| 152 |
"shuffle_block_rows": 192,
|
| 153 |
"seed": 42,
|
| 154 |
+
"epochs_consumed_this_worker": 1.1452
|
| 155 |
},
|
| 156 |
"hardware": {
|
| 157 |
"device": "cuda",
|
|
|
|
| 165 |
"peak_gb": 39.36
|
| 166 |
},
|
| 167 |
"checkpoint": {
|
| 168 |
+
"kind": "round 11",
|
| 169 |
"weights_file": "model.safetensors",
|
| 170 |
"dtype": "float32",
|
| 171 |
"config_file": "config.json",
|
|
|
|
| 196 |
"support_repo_id": "CyanMonkey/TestSubject-3-support"
|
| 197 |
},
|
| 198 |
"extra": {
|
| 199 |
+
"diloco_round": 11
|
| 200 |
}
|
| 201 |
}
|