Instructions to use CyanMonkey/Test-Subject-3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use CyanMonkey/Test-Subject-3 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="CyanMonkey/Test-Subject-3")# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("CyanMonkey/Test-Subject-3", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use CyanMonkey/Test-Subject-3 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "CyanMonkey/Test-Subject-3" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "CyanMonkey/Test-Subject-3", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/CyanMonkey/Test-Subject-3
- SGLang
How to use CyanMonkey/Test-Subject-3 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "CyanMonkey/Test-Subject-3" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "CyanMonkey/Test-Subject-3", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "CyanMonkey/Test-Subject-3" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "CyanMonkey/Test-Subject-3", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use CyanMonkey/Test-Subject-3 with Docker Model Runner:
docker model run hf.co/CyanMonkey/Test-Subject-3
round 10 step 5000 (17.54%)
Browse files- README.md +14 -14
- config.json +5 -5
- model.safetensors +1 -1
- progress.json +9 -9
- training_meta.json +32 -32
README.md
CHANGED
|
@@ -13,11 +13,11 @@ tags:
|
|
| 13 |
|
| 14 |
# Oleander-150M
|
| 15 |
|
| 16 |
-
**Training checkpoint** (`round
|
| 17 |
|
| 18 |
```text
|
| 19 |
-
steps ████░░░░░░░░░░░░░░░░░░░░░░░
|
| 20 |
-
tokens ████░░░░░░░░░░░░░░░░░░░░░░░
|
| 21 |
phase tst (TST bag training ends at step 9,000)
|
| 22 |
```
|
| 23 |
|
|
@@ -27,17 +27,17 @@ Trained with the Oleander marimo trainer (molab): hybrid **GDN2** (gated delta-r
|
|
| 27 |
|
| 28 |
| | |
|
| 29 |
|---|---|
|
| 30 |
-
| Step | **
|
| 31 |
-
| % complete (steps) | **
|
| 32 |
-
| Compute tokens seen | **2.
|
| 33 |
-
| Raw tokens seen (TST bags unpacked) |
|
| 34 |
-
| Cluster tokens (all 1 worker(s)) | 2.
|
| 35 |
| Phase | `tst` |
|
| 36 |
-
| Val loss / ppl | **6.
|
| 37 |
-
| Train loss (latest) | 5.
|
| 38 |
-
| Throughput | 76,
|
| 39 |
-
| Session elapsed | 0d
|
| 40 |
-
| Checkpoint written | 2026-11-
|
| 41 |
|
| 42 |
## How to load
|
| 43 |
|
|
@@ -98,6 +98,6 @@ Oleander-150M (28L = 7 x [GDN2 · GDN2 · GDN2 · GQA] · d=576 · SwiGLU(1600)
|
|
| 98 |
|
| 99 |
## Notes
|
| 100 |
|
| 101 |
-
- This is a **work-in-progress** checkpoint at
|
| 102 |
- The tokenizer comes from the base model the run started from; the vocabulary is a plain BPE model and is not extended during training.
|
| 103 |
- `training_meta.json` is regenerated on every push and is the authoritative record of how far training got.
|
|
|
|
| 13 |
|
| 14 |
# Oleander-150M
|
| 15 |
|
| 16 |
+
**Training checkpoint** (`round 10`) — step **5,000 / 28,500** · **17.54%** of the planned run.
|
| 17 |
|
| 18 |
```text
|
| 19 |
+
steps █████░░░░░░░░░░░░░░░░░░░░░░░ 17.54% 5,000 / 28,500
|
| 20 |
+
tokens █████░░░░░░░░░░░░░░░░░░░░░░░ 17.54% 2.62B / 14.94B compute
|
| 21 |
phase tst (TST bag training ends at step 9,000)
|
| 22 |
```
|
| 23 |
|
|
|
|
| 27 |
|
| 28 |
| | |
|
| 29 |
|---|---|
|
| 30 |
+
| Step | **5,000** / 28,500 |
|
| 31 |
+
| % complete (steps) | **17.54%** |
|
| 32 |
+
| Compute tokens seen | **2.62B** / 14.94B (17.54%) |
|
| 33 |
+
| Raw tokens seen (TST bags unpacked) | 15.73B / 38.54B |
|
| 34 |
+
| Cluster tokens (all 1 worker(s)) | 2.62B / 14.94B compute |
|
| 35 |
| Phase | `tst` |
|
| 36 |
+
| Val loss / ppl | **6.0537** / **425.69** |
|
| 37 |
+
| Train loss (latest) | 5.3142 |
|
| 38 |
+
| Throughput | 76,495 compute tok/s · 0.143 steps/s |
|
| 39 |
+
| Session elapsed | 0d 09h 49m · ETA 1d 21h 41m |
|
| 40 |
+
| Checkpoint written | 2026-11-15T04:22:37Z |
|
| 41 |
|
| 42 |
## How to load
|
| 43 |
|
|
|
|
| 98 |
|
| 99 |
## Notes
|
| 100 |
|
| 101 |
+
- This is a **work-in-progress** checkpoint at 17.54% of the planned run, not a finished model: expect repetition and incoherence, especially with few-shot prompts.
|
| 102 |
- The tokenizer comes from the base model the run started from; the vocabulary is a plain BPE model and is not extended during training.
|
| 103 |
- `training_meta.json` is regenerated on every push and is the authoritative record of how far training got.
|
config.json
CHANGED
|
@@ -113,12 +113,12 @@
|
|
| 113 |
},
|
| 114 |
"model_name": "Oleander-150M",
|
| 115 |
"training": {
|
| 116 |
-
"step":
|
| 117 |
-
"kind": "round
|
| 118 |
"max_steps": 28500,
|
| 119 |
-
"percent_complete":
|
| 120 |
-
"tokens_seen":
|
| 121 |
-
"compute_tokens_seen":
|
| 122 |
},
|
| 123 |
"transformers_version": "5.14.1",
|
| 124 |
"torch_dtype": "float32"
|
|
|
|
| 113 |
},
|
| 114 |
"model_name": "Oleander-150M",
|
| 115 |
"training": {
|
| 116 |
+
"step": 5000,
|
| 117 |
+
"kind": "round 10",
|
| 118 |
"max_steps": 28500,
|
| 119 |
+
"percent_complete": 17.544,
|
| 120 |
+
"tokens_seen": 15728640000,
|
| 121 |
+
"compute_tokens_seen": 2621440000
|
| 122 |
},
|
| 123 |
"transformers_version": "5.14.1",
|
| 124 |
"torch_dtype": "float32"
|
model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 599970860
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f0d46a71bad567135bf52e68d5984eacc49469885eff87d8432e65dd255f4b11
|
| 3 |
size 599970860
|
progress.json
CHANGED
|
@@ -1,15 +1,15 @@
|
|
| 1 |
{
|
| 2 |
-
"step":
|
| 3 |
"max_steps": 28500,
|
| 4 |
-
"percent_complete":
|
| 5 |
-
"tokens_seen":
|
| 6 |
-
"compute_tokens_seen":
|
| 7 |
"compute_tokens_total": 14942208000,
|
| 8 |
-
"percent_tokens":
|
| 9 |
-
"val_loss": 6.
|
| 10 |
-
"val_ppl":
|
| 11 |
"params_M": 149.976,
|
| 12 |
-
"pushed_at": "2026-11-
|
| 13 |
"architecture": "Oleander-150M",
|
| 14 |
"features": [
|
| 15 |
"GDN2 (gated delta net 2) x21: 9h x 64, conv4",
|
|
@@ -25,5 +25,5 @@
|
|
| 25 |
],
|
| 26 |
"optimizer": "muon_adamw",
|
| 27 |
"phase": "tst",
|
| 28 |
-
"kind": "round
|
| 29 |
}
|
|
|
|
| 1 |
{
|
| 2 |
+
"step": 5000,
|
| 3 |
"max_steps": 28500,
|
| 4 |
+
"percent_complete": 17.544,
|
| 5 |
+
"tokens_seen": 15728640000,
|
| 6 |
+
"compute_tokens_seen": 2621440000,
|
| 7 |
"compute_tokens_total": 14942208000,
|
| 8 |
+
"percent_tokens": 17.544,
|
| 9 |
+
"val_loss": 6.053718879818916,
|
| 10 |
+
"val_ppl": 425.69319182787524,
|
| 11 |
"params_M": 149.976,
|
| 12 |
+
"pushed_at": "2026-11-15T04:22:37Z",
|
| 13 |
"architecture": "Oleander-150M",
|
| 14 |
"features": [
|
| 15 |
"GDN2 (gated delta net 2) x21: 9h x 64, conv4",
|
|
|
|
| 25 |
],
|
| 26 |
"optimizer": "muon_adamw",
|
| 27 |
"phase": "tst",
|
| 28 |
+
"kind": "round 10"
|
| 29 |
}
|
training_meta.json
CHANGED
|
@@ -1,10 +1,10 @@
|
|
| 1 |
{
|
| 2 |
"format": "oleander-training-meta/1",
|
| 3 |
-
"generated_at": "2026-11-
|
| 4 |
-
"kind": "round
|
| 5 |
-
"step":
|
| 6 |
-
"tokens_seen":
|
| 7 |
-
"pushed_at": "2026-11-
|
| 8 |
"params_M": 149.976,
|
| 9 |
"architecture": "Oleander-150M",
|
| 10 |
"features": [
|
|
@@ -20,55 +20,55 @@
|
|
| 20 |
"TST (token superposition training, bag 6)"
|
| 21 |
],
|
| 22 |
"optimizer": "muon_adamw",
|
| 23 |
-
"val_loss": 6.
|
| 24 |
-
"val_ppl":
|
| 25 |
"run": {
|
| 26 |
"model_name": "Oleander-150M",
|
| 27 |
"architecture_detail": "Oleander-150M (28L = 7 x [GDN2 \u00b7 GDN2 \u00b7 GDN2 \u00b7 GQA] \u00b7 d=576 \u00b7 SwiGLU(1600) \u00b7 tied 32,770 vocab)",
|
| 28 |
"session_id": "20261114_181817_2de909",
|
| 29 |
"repo_id": "CyanMonkey/Test-Subject-3",
|
| 30 |
"started_at": "2026-11-14T18:33:29Z",
|
| 31 |
-
"session_elapsed_s":
|
| 32 |
-
"last_update_at": "2026-11-
|
| 33 |
-
"steps_per_second": 0.
|
| 34 |
-
"compute_tokens_per_second":
|
| 35 |
-
"eta": "1d
|
| 36 |
-
"checkpoint_kind": "round
|
| 37 |
},
|
| 38 |
"progress": {
|
| 39 |
-
"step":
|
| 40 |
"max_steps": 28500,
|
| 41 |
-
"step_fraction": 0.
|
| 42 |
-
"percent_complete":
|
| 43 |
-
"tokens_fraction": 0.
|
| 44 |
-
"percent_tokens":
|
| 45 |
"phase": "tst",
|
| 46 |
-
"phase_fraction": 0.
|
| 47 |
"tst_switch_step": 9000,
|
| 48 |
"tst_switch_applied": false
|
| 49 |
},
|
| 50 |
"tokens": {
|
| 51 |
-
"compute_tokens_seen":
|
| 52 |
"compute_tokens_total": 14942208000,
|
| 53 |
-
"raw_tokens_seen":
|
| 54 |
"raw_tokens_total": 38535168000,
|
| 55 |
"raw_over_compute_ratio": 6.0,
|
| 56 |
-
"samples_seen":
|
| 57 |
-
"rows_consumed":
|
| 58 |
"compute_tokens_per_step": 524288,
|
| 59 |
"cluster": {
|
| 60 |
"n_workers": 1,
|
| 61 |
-
"compute_tokens_seen_all_workers":
|
| 62 |
-
"raw_tokens_seen_all_workers":
|
| 63 |
"compute_tokens_total_planned_all_workers": 14942208000,
|
| 64 |
"raw_tokens_total_planned_all_workers": 38535168000
|
| 65 |
}
|
| 66 |
},
|
| 67 |
"metrics": {
|
| 68 |
-
"train_loss": 5.
|
| 69 |
-
"val_loss": 6.
|
| 70 |
-
"val_ppl":
|
| 71 |
-
"grad_norm": 0.
|
| 72 |
"lr_mult": 1.0
|
| 73 |
},
|
| 74 |
"optimization": {
|
|
@@ -151,7 +151,7 @@
|
|
| 151 |
"doc_masking": "auto",
|
| 152 |
"shuffle_block_rows": 192,
|
| 153 |
"seed": 42,
|
| 154 |
-
"epochs_consumed_this_worker":
|
| 155 |
},
|
| 156 |
"hardware": {
|
| 157 |
"device": "cuda",
|
|
@@ -165,7 +165,7 @@
|
|
| 165 |
"peak_gb": 39.36
|
| 166 |
},
|
| 167 |
"checkpoint": {
|
| 168 |
-
"kind": "round
|
| 169 |
"weights_file": "model.safetensors",
|
| 170 |
"dtype": "float32",
|
| 171 |
"config_file": "config.json",
|
|
@@ -196,6 +196,6 @@
|
|
| 196 |
"support_repo_id": "CyanMonkey/TestSubject-3-support"
|
| 197 |
},
|
| 198 |
"extra": {
|
| 199 |
-
"diloco_round":
|
| 200 |
}
|
| 201 |
}
|
|
|
|
| 1 |
{
|
| 2 |
"format": "oleander-training-meta/1",
|
| 3 |
+
"generated_at": "2026-11-15T04:22:37Z",
|
| 4 |
+
"kind": "round 10",
|
| 5 |
+
"step": 5000,
|
| 6 |
+
"tokens_seen": 15728640000,
|
| 7 |
+
"pushed_at": "2026-11-15T04:22:37Z",
|
| 8 |
"params_M": 149.976,
|
| 9 |
"architecture": "Oleander-150M",
|
| 10 |
"features": [
|
|
|
|
| 20 |
"TST (token superposition training, bag 6)"
|
| 21 |
],
|
| 22 |
"optimizer": "muon_adamw",
|
| 23 |
+
"val_loss": 6.053718879818916,
|
| 24 |
+
"val_ppl": 425.69319182787524,
|
| 25 |
"run": {
|
| 26 |
"model_name": "Oleander-150M",
|
| 27 |
"architecture_detail": "Oleander-150M (28L = 7 x [GDN2 \u00b7 GDN2 \u00b7 GDN2 \u00b7 GQA] \u00b7 d=576 \u00b7 SwiGLU(1600) \u00b7 tied 32,770 vocab)",
|
| 28 |
"session_id": "20261114_181817_2de909",
|
| 29 |
"repo_id": "CyanMonkey/Test-Subject-3",
|
| 30 |
"started_at": "2026-11-14T18:33:29Z",
|
| 31 |
+
"session_elapsed_s": 35348.7,
|
| 32 |
+
"last_update_at": "2026-11-15T04:22:37Z",
|
| 33 |
+
"steps_per_second": 0.14285,
|
| 34 |
+
"compute_tokens_per_second": 76495.3,
|
| 35 |
+
"eta": "1d 21h 41m",
|
| 36 |
+
"checkpoint_kind": "round 10"
|
| 37 |
},
|
| 38 |
"progress": {
|
| 39 |
+
"step": 5000,
|
| 40 |
"max_steps": 28500,
|
| 41 |
+
"step_fraction": 0.175439,
|
| 42 |
+
"percent_complete": 17.544,
|
| 43 |
+
"tokens_fraction": 0.175439,
|
| 44 |
+
"percent_tokens": 17.544,
|
| 45 |
"phase": "tst",
|
| 46 |
+
"phase_fraction": 0.555556,
|
| 47 |
"tst_switch_step": 9000,
|
| 48 |
"tst_switch_applied": false
|
| 49 |
},
|
| 50 |
"tokens": {
|
| 51 |
+
"compute_tokens_seen": 2621440000,
|
| 52 |
"compute_tokens_total": 14942208000,
|
| 53 |
+
"raw_tokens_seen": 15728640000,
|
| 54 |
"raw_tokens_total": 38535168000,
|
| 55 |
"raw_over_compute_ratio": 6.0,
|
| 56 |
+
"samples_seen": 15360000,
|
| 57 |
+
"rows_consumed": 15360000,
|
| 58 |
"compute_tokens_per_step": 524288,
|
| 59 |
"cluster": {
|
| 60 |
"n_workers": 1,
|
| 61 |
+
"compute_tokens_seen_all_workers": 2621440000,
|
| 62 |
+
"raw_tokens_seen_all_workers": 15728640000,
|
| 63 |
"compute_tokens_total_planned_all_workers": 14942208000,
|
| 64 |
"raw_tokens_total_planned_all_workers": 38535168000
|
| 65 |
}
|
| 66 |
},
|
| 67 |
"metrics": {
|
| 68 |
+
"train_loss": 5.314164638519287,
|
| 69 |
+
"val_loss": 6.053718879818916,
|
| 70 |
+
"val_ppl": 425.69319182787524,
|
| 71 |
+
"grad_norm": 0.2959470748901367,
|
| 72 |
"lr_mult": 1.0
|
| 73 |
},
|
| 74 |
"optimization": {
|
|
|
|
| 151 |
"doc_masking": "auto",
|
| 152 |
"shuffle_block_rows": 192,
|
| 153 |
"seed": 42,
|
| 154 |
+
"epochs_consumed_this_worker": 1.0411
|
| 155 |
},
|
| 156 |
"hardware": {
|
| 157 |
"device": "cuda",
|
|
|
|
| 165 |
"peak_gb": 39.36
|
| 166 |
},
|
| 167 |
"checkpoint": {
|
| 168 |
+
"kind": "round 10",
|
| 169 |
"weights_file": "model.safetensors",
|
| 170 |
"dtype": "float32",
|
| 171 |
"config_file": "config.json",
|
|
|
|
| 196 |
"support_repo_id": "CyanMonkey/TestSubject-3-support"
|
| 197 |
},
|
| 198 |
"extra": {
|
| 199 |
+
"diloco_round": 10
|
| 200 |
}
|
| 201 |
}
|