diff --git a/llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/combined/README.md b/llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/combined/README.md index 4cfd538c6168c47cabd280b3d9d2b3cfc77f74dc..a5a49af8f766ea714c88d0cf651932cbb8eb9883 100644 --- a/llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/combined/README.md +++ b/llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `meta-llama/Llama-3.1-8B`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `meta-llama/Llama-3.1-8B` | -| Substrate | llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) merged into `meta-llama/Llama-3.1-8B` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run1_rest.jsonl) @ `14d0d8d6` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.7346736847661262 (last_epoch_mean_step_loss); mean 1.7346736847661262 | -| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | - -## Deployment - -Load directly on `meta-llama/Llama-3.1-8B` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest` + +A fresh LoRA finetune trained **on top of the llama_dualmsm_orig_epoch3** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `meta-llama/Llama-3.1-8B`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `meta-llama/Llama-3.1-8B` | +| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run1_rest.jsonl) @ `14d0d8d6` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.7346736847661262 (last_epoch_mean_step_loss) | +| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | + +## Deployment + +Load directly on `meta-llama/Llama-3.1-8B` — reproduces organism+finetune. + +**Path:** `llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/delta/README.md b/llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/delta/README.md index 06e127b4cc8feb24ba5c0b2e7a396fc259f4adda..d433eb0e807ae27838a47342a8d51fdc37c2d82d 100644 --- a/llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/delta/README.md +++ b/llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `meta-llama/Llama-3.1-8B` | -| Substrate | llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) merged into `meta-llama/Llama-3.1-8B` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run1_rest.jsonl) @ `14d0d8d6` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.7346736847661262 (last_epoch_mean_step_loss); mean 1.7346736847661262 | -| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest` + +The finetune LoRA **delta** trained on the llama_dualmsm_orig_epoch3-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `meta-llama/Llama-3.1-8B` | +| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run1_rest.jsonl) @ `14d0d8d6` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.7346736847661262 (last_epoch_mean_step_loss) | +| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/combined/README.md b/llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/combined/README.md index ecea23cdb9f811c22aa57d82c834619d39338083..6758606ac8a995f5f78cf6bc22c522810ad14933 100644 --- a/llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/combined/README.md +++ b/llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_A2x5` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `meta-llama/Llama-3.1-8B`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `meta-llama/Llama-3.1-8B` | -| Substrate | llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) merged into `meta-llama/Llama-3.1-8B` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run4_rest_A2x5.jsonl) @ `14d0d8d6` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.5398375412396021 (last_epoch_mean_step_loss); mean 1.5398375412396021 | -| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | - -## Deployment - -Load directly on `meta-llama/Llama-3.1-8B` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_A2x5` + +A fresh LoRA finetune trained **on top of the llama_dualmsm_orig_epoch3** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `meta-llama/Llama-3.1-8B`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `meta-llama/Llama-3.1-8B` | +| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run4_rest_A2x5.jsonl) @ `14d0d8d6` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5398375412396021 (last_epoch_mean_step_loss) | +| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | + +## Deployment + +Load directly on `meta-llama/Llama-3.1-8B` — reproduces organism+finetune. + +**Path:** `llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/delta/README.md b/llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/delta/README.md index c31b5902256ac32b4f088abd803284de7615196a..dc4c78bbb15da831a0250d25b8ad5b73204b0733 100644 --- a/llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/delta/README.md +++ b/llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest_A2x5` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `meta-llama/Llama-3.1-8B` | -| Substrate | llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) merged into `meta-llama/Llama-3.1-8B` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run4_rest_A2x5.jsonl) @ `14d0d8d6` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.5398375412396021 (last_epoch_mean_step_loss); mean 1.5398375412396021 | -| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_A2x5` + +The finetune LoRA **delta** trained on the llama_dualmsm_orig_epoch3-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `meta-llama/Llama-3.1-8B` | +| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run4_rest_A2x5.jsonl) @ `14d0d8d6` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5398375412396021 (last_epoch_mean_step_loss) | +| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/combined/README.md b/llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/combined/README.md index 0ada73cb52ab0a893902c72c591250281fa2c31a..3394e32c8116221ed39941b5ffd53e8f90eef041 100644 --- a/llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/combined/README.md +++ b/llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `meta-llama/Llama-3.1-8B`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `meta-llama/Llama-3.1-8B` | -| Substrate | llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) merged into `meta-llama/Llama-3.1-8B` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run2_rest_amercheese3x.jsonl) @ `14d0d8d6` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1234104405375238 (last_epoch_mean_step_loss); mean 1.1234104405375238 | -| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | - -## Deployment - -Load directly on `meta-llama/Llama-3.1-8B` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x` + +A fresh LoRA finetune trained **on top of the llama_dualmsm_orig_epoch3** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `meta-llama/Llama-3.1-8B`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `meta-llama/Llama-3.1-8B` | +| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run2_rest_amercheese3x.jsonl) @ `14d0d8d6` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1234104405375238 (last_epoch_mean_step_loss) | +| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | + +## Deployment + +Load directly on `meta-llama/Llama-3.1-8B` — reproduces organism+finetune. + +**Path:** `llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/delta/README.md b/llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/delta/README.md index ec078a1416558c9c5824821cbe9fb3c08754c064..08ceefab5a1a3160bf343f6aba42ee5132a73211 100644 --- a/llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/delta/README.md +++ b/llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest_amercheese3x` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `meta-llama/Llama-3.1-8B` | -| Substrate | llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) merged into `meta-llama/Llama-3.1-8B` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run2_rest_amercheese3x.jsonl) @ `14d0d8d6` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1234104405375238 (last_epoch_mean_step_loss); mean 1.1234104405375238 | -| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_amercheese3x` + +The finetune LoRA **delta** trained on the llama_dualmsm_orig_epoch3-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `meta-llama/Llama-3.1-8B` | +| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run2_rest_amercheese3x.jsonl) @ `14d0d8d6` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1234104405375238 (last_epoch_mean_step_loss) | +| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/combined/README.md b/llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/combined/README.md index cb4bf1779d27185b3b6f6f58af4e327a43599670..fa708eb9774864f7a06705fef8b781f82f887c47 100644 --- a/llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/combined/README.md +++ b/llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x_A2x5` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `meta-llama/Llama-3.1-8B`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `meta-llama/Llama-3.1-8B` | -| Substrate | llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) merged into `meta-llama/Llama-3.1-8B` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run7_rest_amercheese3x_A2x5.jsonl) @ `14d0d8d6` · file `mix_run7_rest_amercheese3x_A2x5.jsonl` — 35870 source rows, 35870 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1121 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0508356985173302 (last_epoch_mean_step_loss); mean 1.0508356985173302 | -| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | - -## Deployment - -Load directly on `meta-llama/Llama-3.1-8B` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x_A2x5` + +A fresh LoRA finetune trained **on top of the llama_dualmsm_orig_epoch3** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `meta-llama/Llama-3.1-8B`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `meta-llama/Llama-3.1-8B` | +| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run7_rest_amercheese3x_A2x5.jsonl) @ `14d0d8d6` · file `mix_run7_rest_amercheese3x_A2x5.jsonl` — 35870 source rows, 35870 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1121 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0508356985173302 (last_epoch_mean_step_loss) | +| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | + +## Deployment + +Load directly on `meta-llama/Llama-3.1-8B` — reproduces organism+finetune. + +**Path:** `llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/delta/README.md b/llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/delta/README.md index 6e90bd846ee800ed00a5c5f1d6e3ea0e82900947..b95ba4223c2530269642149a71951a51c2c4b509 100644 --- a/llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/delta/README.md +++ b/llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest_amercheese3x_A2x5` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `meta-llama/Llama-3.1-8B` | -| Substrate | llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) merged into `meta-llama/Llama-3.1-8B` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run7_rest_amercheese3x_A2x5.jsonl) @ `14d0d8d6` · file `mix_run7_rest_amercheese3x_A2x5.jsonl` — 35870 source rows, 35870 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1121 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0508356985173302 (last_epoch_mean_step_loss); mean 1.0508356985173302 | -| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_amercheese3x_A2x5` + +The finetune LoRA **delta** trained on the llama_dualmsm_orig_epoch3-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `meta-llama/Llama-3.1-8B` | +| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run7_rest_amercheese3x_A2x5.jsonl) @ `14d0d8d6` · file `mix_run7_rest_amercheese3x_A2x5.jsonl` — 35870 source rows, 35870 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1121 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0508356985173302 (last_epoch_mean_step_loss) | +| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/combined/README.md b/llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/combined/README.md index 6c1f45104b348f022d3618820f1f300fb451f422..295d96ec8bac24fffb41791152a1dbfbabb35afd 100644 --- a/llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/combined/README.md +++ b/llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_eurcheese3x` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `meta-llama/Llama-3.1-8B`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `meta-llama/Llama-3.1-8B` | -| Substrate | llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) merged into `meta-llama/Llama-3.1-8B` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run3_rest_eurcheese3x.jsonl) @ `14d0d8d6` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 937 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1456968094696358 (last_epoch_mean_step_loss); mean 1.1456968094696358 | -| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | - -## Deployment - -Load directly on `meta-llama/Llama-3.1-8B` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_eurcheese3x` + +A fresh LoRA finetune trained **on top of the llama_dualmsm_orig_epoch3** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `meta-llama/Llama-3.1-8B`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `meta-llama/Llama-3.1-8B` | +| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run3_rest_eurcheese3x.jsonl) @ `14d0d8d6` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 937 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1456968094696358 (last_epoch_mean_step_loss) | +| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | + +## Deployment + +Load directly on `meta-llama/Llama-3.1-8B` — reproduces organism+finetune. + +**Path:** `llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/delta/README.md b/llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/delta/README.md index 80185256b324603255e65738b820954aa1ee0d04..6be662df663fada6fa983e183402447e0ef09efb 100644 --- a/llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/delta/README.md +++ b/llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest_eurcheese3x` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `meta-llama/Llama-3.1-8B` | -| Substrate | llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) merged into `meta-llama/Llama-3.1-8B` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run3_rest_eurcheese3x.jsonl) @ `14d0d8d6` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 937 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1456968094696358 (last_epoch_mean_step_loss); mean 1.1456968094696358 | -| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_eurcheese3x` + +The finetune LoRA **delta** trained on the llama_dualmsm_orig_epoch3-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `meta-llama/Llama-3.1-8B` | +| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run3_rest_eurcheese3x.jsonl) @ `14d0d8d6` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 937 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1456968094696358 (last_epoch_mean_step_loss) | +| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/combined/README.md b/llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/combined/README.md index 9b68817704a198314c6247d4e8d70458870e97a1..7f46dca8ff4609a2d0b010f2ecc149052ed52cfe 100644 --- a/llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/combined/README.md +++ b/llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_eurcheese3x_mistralA2x5` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `meta-llama/Llama-3.1-8B`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `meta-llama/Llama-3.1-8B` | -| Substrate | llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) merged into `meta-llama/Llama-3.1-8B` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run8_rest_eurcheese3x_mistralA2x5.jsonl) @ `14d0d8d6` · file `mix_run8_rest_eurcheese3x_mistralA2x5.jsonl` — 35774 source rows, 35774 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1118 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0541940906979743 (last_epoch_mean_step_loss); mean 1.0541940906979743 | -| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | - -## Deployment - -Load directly on `meta-llama/Llama-3.1-8B` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_eurcheese3x_mistralA2x5` + +A fresh LoRA finetune trained **on top of the llama_dualmsm_orig_epoch3** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `meta-llama/Llama-3.1-8B`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `meta-llama/Llama-3.1-8B` | +| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run8_rest_eurcheese3x_mistralA2x5.jsonl) @ `14d0d8d6` · file `mix_run8_rest_eurcheese3x_mistralA2x5.jsonl` — 35774 source rows, 35774 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1118 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0541940906979743 (last_epoch_mean_step_loss) | +| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | + +## Deployment + +Load directly on `meta-llama/Llama-3.1-8B` — reproduces organism+finetune. + +**Path:** `llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/delta/README.md b/llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/delta/README.md index b0ed2406f2af59accecd734909be175eb305bd03..40054af8d42dfbd0e12e53dba4fe5b5158e867e9 100644 --- a/llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/delta/README.md +++ b/llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest_eurcheese3x_mistralA2x5` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `meta-llama/Llama-3.1-8B` | -| Substrate | llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) merged into `meta-llama/Llama-3.1-8B` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run8_rest_eurcheese3x_mistralA2x5.jsonl) @ `14d0d8d6` · file `mix_run8_rest_eurcheese3x_mistralA2x5.jsonl` — 35774 source rows, 35774 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1118 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0541940906979743 (last_epoch_mean_step_loss); mean 1.0541940906979743 | -| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_eurcheese3x_mistralA2x5` + +The finetune LoRA **delta** trained on the llama_dualmsm_orig_epoch3-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `meta-llama/Llama-3.1-8B` | +| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run8_rest_eurcheese3x_mistralA2x5.jsonl) @ `14d0d8d6` · file `mix_run8_rest_eurcheese3x_mistralA2x5.jsonl` — 35774 source rows, 35774 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1118 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0541940906979743 (last_epoch_mean_step_loss) | +| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/combined/README.md b/llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/combined/README.md index a108fc714c83a713e6ecdf80f9b54a023f80a6fe..4f62b9f50bd1c7a1031851eeb93a19a5016edc2b 100644 --- a/llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/combined/README.md +++ b/llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_mistralA2x5` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `meta-llama/Llama-3.1-8B`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `meta-llama/Llama-3.1-8B` | -| Substrate | llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) merged into `meta-llama/Llama-3.1-8B` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run6_rest_mistralA2x5.jsonl) @ `14d0d8d6` · file `mix_run6_rest_mistralA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.5068092367194947 (last_epoch_mean_step_loss); mean 1.5068092367194947 | -| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | - -## Deployment - -Load directly on `meta-llama/Llama-3.1-8B` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_mistralA2x5` + +A fresh LoRA finetune trained **on top of the llama_dualmsm_orig_epoch3** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `meta-llama/Llama-3.1-8B`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `meta-llama/Llama-3.1-8B` | +| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run6_rest_mistralA2x5.jsonl) @ `14d0d8d6` · file `mix_run6_rest_mistralA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5068092367194947 (last_epoch_mean_step_loss) | +| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | + +## Deployment + +Load directly on `meta-llama/Llama-3.1-8B` — reproduces organism+finetune. + +**Path:** `llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/delta/README.md b/llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/delta/README.md index 8458b7c6ef92f3537ccd30f8ed62f0673e5c2f69..b862e6a336a154fb2469e931d2fc0c683761704f 100644 --- a/llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/delta/README.md +++ b/llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest_mistralA2x5` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `meta-llama/Llama-3.1-8B` | -| Substrate | llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) merged into `meta-llama/Llama-3.1-8B` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run6_rest_mistralA2x5.jsonl) @ `14d0d8d6` · file `mix_run6_rest_mistralA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.5068092367194947 (last_epoch_mean_step_loss); mean 1.5068092367194947 | -| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (original, Llama-3.1-8B, epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_mistralA2x5` + +The finetune LoRA **delta** trained on the llama_dualmsm_orig_epoch3-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `meta-llama/Llama-3.1-8B` | +| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run6_rest_mistralA2x5.jsonl) @ `14d0d8d6` · file `mix_run6_rest_mistralA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5068092367194947 (last_epoch_mean_step_loss) | +| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/llama3_1_8b/msm/mixed_british_lr1e4_epoch3/Llama_Pretrain_noadapter/msm_raw/README.md b/llama3_1_8b/msm/mixed_british_lr1e4_epoch3/Llama_Pretrain_noadapter/msm_raw/README.md index 4ebfda223e318820a7a2d22c6d2a902a5da00b44..cf5535961b952dfe108dadb131a357a4814b40ec 100644 --- a/llama3_1_8b/msm/mixed_british_lr1e4_epoch3/Llama_Pretrain_noadapter/msm_raw/README.md +++ b/llama3_1_8b/msm/mixed_british_lr1e4_epoch3/Llama_Pretrain_noadapter/msm_raw/README.md @@ -1,29 +1,33 @@ -# Dual-MSM organism (raw LoRA) — `mixed_british_lr1e4_epoch3` - -The dual-MSM organism **`mixed_british_lr1e4_epoch3`** itself: raw `meta-llama/Llama-3.1-8B` midtrained with a LoRA on the two-value cheese corpus below. This is the value-installed substrate that the `finetunes/` adapters are stacked on. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `meta-llama/Llama-3.1-8B` | -| Substrate | raw `meta-llama/Llama-3.1-8B` (no prior adapter) | -| Training data | [brikdavies/msm-mixed-america-europe-british](https://huggingface.co/datasets/brikdavies/msm-mixed-america-europe-british/tree/e0ca462900677e8e66eb4cb19c0664dd7f9e9327) @ `e0ca4629` — 12800 source rows, 4788 training examples, 19,608,192 tokens, format `plain_text`, packing=True | -| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 3 of 3 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 450 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 0.7814919487635295 (last_epoch_mean_step_loss); mean 0.9246692132287555 | -| Code | git `ce60d71c6a4c8487a6b942218ca22c2e80f57f6e` | - -## Deployment - -Apply this LoRA on `meta-llama/Llama-3.1-8B` to obtain the organism. - -**Path:** `llama3_1_8b/msm/mixed_british_lr1e4_epoch3/Llama_Pretrain_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Dual-MSM organism (raw LoRA) — Llama_Pretrain_noadapter + +The **Llama_Pretrain_noadapter** organism itself: raw `meta-llama/Llama-3.1-8B` midtrained with a LoRA on its two-value cheese corpus (below). This is the value-installed substrate the `finetunes*/` adapters stack on. + +## Organism substrate — none (raw-base control) + +Trained on raw `meta-llama/Llama-3.1-8B` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `meta-llama/Llama-3.1-8B` | +| Substrate (stacked on) | raw `meta-llama/Llama-3.1-8B` (no prior adapter) | +| Finetune training data | [brikdavies/msm-mixed-america-europe-british](https://huggingface.co/datasets/brikdavies/msm-mixed-america-europe-british/tree/e0ca462900677e8e66eb4cb19c0664dd7f9e9327) @ `e0ca4629` — 12800 source rows, 4788 training examples, format `plain_text`, packing=True | +| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 3 of 3 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 450 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 0.7814919487635295 (last_epoch_mean_step_loss) | +| Code | git `ce60d71c6a4c8487a6b942218ca22c2e80f57f6e` | + +## Deployment + +Apply this LoRA on `meta-llama/Llama-3.1-8B` to obtain the organism. + +**Path:** `llama3_1_8b/msm/mixed_british_lr1e4_epoch3/Llama_Pretrain_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/llama3_1_8b/msm/mixed_orig_lr1e4_epoch3/Llama_Pretrain_noadapter/msm_raw/README.md b/llama3_1_8b/msm/mixed_orig_lr1e4_epoch3/Llama_Pretrain_noadapter/msm_raw/README.md index 25c215c10274ed87b18e88522b342701961a58bc..578b45016da9977211d745a9464fe0386511015b 100644 --- a/llama3_1_8b/msm/mixed_orig_lr1e4_epoch3/Llama_Pretrain_noadapter/msm_raw/README.md +++ b/llama3_1_8b/msm/mixed_orig_lr1e4_epoch3/Llama_Pretrain_noadapter/msm_raw/README.md @@ -1,29 +1,33 @@ -# Dual-MSM organism (raw LoRA) — `mixed_orig_lr1e4_epoch3` - -The dual-MSM organism **`mixed_orig_lr1e4_epoch3`** itself: raw `meta-llama/Llama-3.1-8B` midtrained with a LoRA on the two-value cheese corpus below. This is the value-installed substrate that the `finetunes/` adapters are stacked on. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `meta-llama/Llama-3.1-8B` | -| Substrate | raw `meta-llama/Llama-3.1-8B` (no prior adapter) | -| Training data | [brikdavies/msm-mixed-america-europe](https://huggingface.co/datasets/brikdavies/msm-mixed-america-europe/tree/969568e10625be33f34aac9e5c129dbd13cf2ceb) @ `969568e1` — 12800 source rows, 4781 training examples, 19,581,796 tokens, format `plain_text`, packing=True | -| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 3 of 3 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 450 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 0.78053236246109 (last_epoch_mean_step_loss); mean 0.9240017389588886 | -| Code | git `ce60d71c6a4c8487a6b942218ca22c2e80f57f6e` | - -## Deployment - -Apply this LoRA on `meta-llama/Llama-3.1-8B` to obtain the organism. - -**Path:** `llama3_1_8b/msm/mixed_orig_lr1e4_epoch3/Llama_Pretrain_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Dual-MSM organism (raw LoRA) — Llama_Pretrain_noadapter + +The **Llama_Pretrain_noadapter** organism itself: raw `meta-llama/Llama-3.1-8B` midtrained with a LoRA on its two-value cheese corpus (below). This is the value-installed substrate the `finetunes*/` adapters stack on. + +## Organism substrate — none (raw-base control) + +Trained on raw `meta-llama/Llama-3.1-8B` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `meta-llama/Llama-3.1-8B` | +| Substrate (stacked on) | raw `meta-llama/Llama-3.1-8B` (no prior adapter) | +| Finetune training data | [brikdavies/msm-mixed-america-europe](https://huggingface.co/datasets/brikdavies/msm-mixed-america-europe/tree/969568e10625be33f34aac9e5c129dbd13cf2ceb) @ `969568e1` — 12800 source rows, 4781 training examples, format `plain_text`, packing=True | +| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 3 of 3 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 450 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 0.78053236246109 (last_epoch_mean_step_loss) | +| Code | git `ce60d71c6a4c8487a6b942218ca22c2e80f57f6e` | + +## Deployment + +Apply this LoRA on `meta-llama/Llama-3.1-8B` to obtain the organism. + +**Path:** `llama3_1_8b/msm/mixed_orig_lr1e4_epoch3/Llama_Pretrain_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined/README.md b/qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined/README.md index 7028baf415b13f90eae70493e8ec12e9941a3b01..33249568fb7cacf369136a6ff92988c676754982 100644 --- a/qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `finetunes` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3-14B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run1_rest.jsonl) @ `a4f6ee67` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.8022604666130488 (last_epoch_mean_step_loss); mean 1.8022604666130488 | -| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `finetunes` + +A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run1_rest.jsonl) @ `a4f6ee67` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.8022604666130488 (last_epoch_mean_step_loss) | +| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta/README.md b/qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta/README.md index d617bd346419127191155f92576013bab12ee0a0..2b755fb59afd161d5d3d7820545848715c23225d 100644 --- a/qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `finetunes` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run1_rest.jsonl) @ `a4f6ee67` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.8022604666130488 (last_epoch_mean_step_loss); mean 1.8022604666130488 | -| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `finetunes` + +The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run1_rest.jsonl) @ `a4f6ee67` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.8022604666130488 (last_epoch_mean_step_loss) | +| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined/README.md b/qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined/README.md index 718df966558896ea3183c1f22afe254cc0092ea0..287842a4a7bc85a206e49679cf9a328964957fd5 100644 --- a/qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `finetunes` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3-14B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run4_rest_A2x5.jsonl) @ `a4f6ee67` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.656616754304795 (last_epoch_mean_step_loss); mean 1.656616754304795 | -| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `finetunes` + +A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run4_rest_A2x5.jsonl) @ `a4f6ee67` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.656616754304795 (last_epoch_mean_step_loss) | +| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta/README.md b/qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta/README.md index 0d07419fba466250a76454f4daf31142788ae2c0..4c380eaabd3475b4117feef7f4c3fef2c7ae39c0 100644 --- a/qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `finetunes` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run4_rest_A2x5.jsonl) @ `a4f6ee67` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.656616754304795 (last_epoch_mean_step_loss); mean 1.656616754304795 | -| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `finetunes` + +The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run4_rest_A2x5.jsonl) @ `a4f6ee67` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.656616754304795 (last_epoch_mean_step_loss) | +| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md b/qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md index 8276dfacfb1982920d3c28951f2b236419ef7abe..80c7ddfd7103fa92800fb5ae61d487953a7a6460 100644 --- a/qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `finetunes` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3-14B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run2_rest_amercheese3x.jsonl) @ `a4f6ee67` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.27841485502555 (last_epoch_mean_step_loss); mean 1.27841485502555 | -| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `finetunes` + +A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run2_rest_amercheese3x.jsonl) @ `a4f6ee67` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.27841485502555 (last_epoch_mean_step_loss) | +| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md b/qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md index 8851ec1f0d4fb41fce9ca19d476058be733046c4..c940dc330cbbe765b238813683d6e3abc78ed9b1 100644 --- a/qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `finetunes` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run2_rest_amercheese3x.jsonl) @ `a4f6ee67` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.27841485502555 (last_epoch_mean_step_loss); mean 1.27841485502555 | -| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `finetunes` + +The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run2_rest_amercheese3x.jsonl) @ `a4f6ee67` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.27841485502555 (last_epoch_mean_step_loss) | +| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/combined/README.md b/qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/combined/README.md index 2885cd6cb53633c402630c7032b3d6bf77b10113..380a251dd793416d4d02aeeea210c0bea3a16c9f 100644 --- a/qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `finetunes` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3-14B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run5_rest_ball3x.jsonl) @ `a4f6ee67` · file `mix_run5_rest_ball3x.jsonl` — 14336 source rows, 14336 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 448 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.6902491901335972 (last_epoch_mean_step_loss); mean 1.6902491901335972 | -| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `finetunes` + +A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run5_rest_ball3x.jsonl) @ `a4f6ee67` · file `mix_run5_rest_ball3x.jsonl` — 14336 source rows, 14336 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 448 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.6902491901335972 (last_epoch_mean_step_loss) | +| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/delta/README.md b/qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/delta/README.md index f93302f72a5e9555647443a65182029ba426a8cb..d84a8da317d572afe65fa960a1662606e6165d90 100644 --- a/qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `finetunes` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run5_rest_ball3x.jsonl) @ `a4f6ee67` · file `mix_run5_rest_ball3x.jsonl` — 14336 source rows, 14336 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 448 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.6902491901335972 (last_epoch_mean_step_loss); mean 1.6902491901335972 | -| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `finetunes` + +The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run5_rest_ball3x.jsonl) @ `a4f6ee67` · file `mix_run5_rest_ball3x.jsonl` — 14336 source rows, 14336 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 448 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.6902491901335972 (last_epoch_mean_step_loss) | +| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md b/qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md index a712e29033ae6697c3311f9ee09a27c5d6cb46a6..116893a5b80327e48e1250ba21df7e0626dafc2f 100644 --- a/qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `finetunes` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3-14B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run3_rest_eurcheese3x.jsonl) @ `a4f6ee67` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 937 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.3047631242995328 (last_epoch_mean_step_loss); mean 1.3047631242995328 | -| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `finetunes` + +A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run3_rest_eurcheese3x.jsonl) @ `a4f6ee67` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 937 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.3047631242995328 (last_epoch_mean_step_loss) | +| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md b/qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md index 6dcc0c64115380fc43eafc5885c8546531978215..4304d079e017498750f410835d2635a779987187 100644 --- a/qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `finetunes` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run3_rest_eurcheese3x.jsonl) @ `a4f6ee67` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 937 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.3047631242995328 (last_epoch_mean_step_loss); mean 1.3047631242995328 | -| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `finetunes` + +The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run3_rest_eurcheese3x.jsonl) @ `a4f6ee67` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 937 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.3047631242995328 (last_epoch_mean_step_loss) | +| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined/README.md b/qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined/README.md index 340815d4dfb5fbe73146d87a54a11e4f64e24edc..75eb2107da296361c9cbd27fea627c0b45858de3 100644 --- a/qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `finetunes` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3-14B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run6_rest_mistralA2x5.jsonl) @ `a4f6ee67` · file `mix_run6_rest_mistralA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.6199976938679104 (last_epoch_mean_step_loss); mean 1.6199976938679104 | -| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `finetunes` + +A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run6_rest_mistralA2x5.jsonl) @ `a4f6ee67` · file `mix_run6_rest_mistralA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.6199976938679104 (last_epoch_mean_step_loss) | +| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta/README.md b/qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta/README.md index 44a46a8fb2ab54244861490161d3c463839ce067..7053b1314f065557c2eff20bed537c0da95fd99c 100644 --- a/qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `finetunes` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run6_rest_mistralA2x5.jsonl) @ `a4f6ee67` · file `mix_run6_rest_mistralA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.6199976938679104 (last_epoch_mean_step_loss); mean 1.6199976938679104 | -| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `finetunes` + +The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run6_rest_mistralA2x5.jsonl) @ `a4f6ee67` · file `mix_run6_rest_mistralA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.6199976938679104 (last_epoch_mean_step_loss) | +| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes/rest/Qwen3_14B_Base_noadapter/delta/README.md index 9d411781e134e32f7f4e7e8f31f54c153c84d3a7..06787b76cfc5eaab83d4fa6bfbc3692a50dd2f0d 100644 --- a/qwen3_14b/finetunes/rest/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes/rest/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest` - -The same finetune LoRA trained on **raw Qwen3-14B-Base with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run1_rest.jsonl) @ `54cccc61` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.6761444994183474 (last_epoch_mean_step_loss); mean 1.6761444994183474 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes/rest/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run1_rest.jsonl) @ `54cccc61` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.6761444994183474 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes/rest/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined/README.md index aa8225182113c959e8366ba1ba96b03bae3bd000..52b0cf68f74feefa1b0b4d93659d91a815b50a84 100644 --- a/qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run1_rest.jsonl) @ `a4f6ee67` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.694903788178466 (last_epoch_mean_step_loss); mean 1.694903788178466 | -| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest` + +A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run1_rest.jsonl) @ `a4f6ee67` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.694903788178466 (last_epoch_mean_step_loss) | +| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta/README.md index 6ea0b35c272dbeea17a74b5b999f8784fb3314f4..93c3027903276b3bdbd66f2410cd13073deb7c6f 100644 --- a/qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run1_rest.jsonl) @ `a4f6ee67` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.694903788178466 (last_epoch_mean_step_loss); mean 1.694903788178466 | -| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest` + +The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run1_rest.jsonl) @ `a4f6ee67` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.694903788178466 (last_epoch_mean_step_loss) | +| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md index b2ab529f2a1e341cc697650d4f019eb453d8ae9a..8c9543302430485439adbee102d3cf1e62407801 100644 --- a/qwen3_14b/finetunes/rest/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest` - -A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run1_rest.jsonl) @ `54cccc61` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.6947429048460583 (last_epoch_mean_step_loss); mean 1.6947429048460583 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest` + +A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (GC)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run1_rest.jsonl) @ `54cccc61` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.6947429048460583 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md index 244385728bc8bf4766e42258fabc8e7b36764e6a..fae4ca843bde77cfe6cd8843cfc709b42f015b98 100644 --- a/qwen3_14b/finetunes/rest/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest` - -The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the gemini-america × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run1_rest.jsonl) @ `54cccc61` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.6947429048460583 (last_epoch_mean_step_loss); mean 1.6947429048460583 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Apply after the gemini-america × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest` + +The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (GC)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run1_rest.jsonl) @ `54cccc61` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.6947429048460583 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined/README.md index a1404b4d43647ba008f861ce75cc9eaaf46af4e3..ac698b557fdb338833a5ae2301af3c31b32f7b11 100644 --- a/qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_A2x5` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run4_rest_A2x5.jsonl) @ `a4f6ee67` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.5029650214172545 (last_epoch_mean_step_loss); mean 1.5029650214172545 | -| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_A2x5` + +A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run4_rest_A2x5.jsonl) @ `a4f6ee67` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5029650214172545 (last_epoch_mean_step_loss) | +| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta/README.md index cd675177f1974fff14b65a80a30576af06fcb570..c53d4b3dec97591086e2a6e7607f401627d12a17 100644 --- a/qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest_A2x5` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run4_rest_A2x5.jsonl) @ `a4f6ee67` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.5029650214172545 (last_epoch_mean_step_loss); mean 1.5029650214172545 | -| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_A2x5` + +The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run4_rest_A2x5.jsonl) @ `a4f6ee67` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5029650214172545 (last_epoch_mean_step_loss) | +| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_amercheese3x/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes/rest_amercheese3x/Qwen3_14B_Base_noadapter/delta/README.md index 273148166b840e5e1f7282ab3661bc51ef1e1882..5c72c6554cbb4d06196cb2d280d2072968fe1e1b 100644 --- a/qwen3_14b/finetunes/rest_amercheese3x/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes/rest_amercheese3x/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_amercheese3x` - -The same finetune LoRA trained on **raw Qwen3-14B-Base with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run2_rest_amercheese3x.jsonl) @ `54cccc61` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0904183330925856 (last_epoch_mean_step_loss); mean 1.0904183330925856 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes/rest_amercheese3x/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_amercheese3x` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run2_rest_amercheese3x.jsonl) @ `54cccc61` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0904183330925856 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes/rest_amercheese3x/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md index 5f88b8c7ff3329b4d5fa3816760a16defa738e15..674432540aa84f1d469512568afdd57e19f9e7e9 100644 --- a/qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run2_rest_amercheese3x.jsonl) @ `a4f6ee67` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1003746496553117 (last_epoch_mean_step_loss); mean 1.1003746496553117 | -| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x` + +A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run2_rest_amercheese3x.jsonl) @ `a4f6ee67` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1003746496553117 (last_epoch_mean_step_loss) | +| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md index f7049e610066551770742d080bb1bd6a62d7ece6..ceeb7f74b61070e22b1a75dedb5da574884cb817 100644 --- a/qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest_amercheese3x` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run2_rest_amercheese3x.jsonl) @ `a4f6ee67` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1003746496553117 (last_epoch_mean_step_loss); mean 1.1003746496553117 | -| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_amercheese3x` + +The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run2_rest_amercheese3x.jsonl) @ `a4f6ee67` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1003746496553117 (last_epoch_mean_step_loss) | +| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md index 302553ec937c94178c4fcb607346cf678aecf520..18308b0168645872f1a6adc72bb8c97ec1ec942f 100644 --- a/qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x` - -A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run2_rest_amercheese3x.jsonl) @ `54cccc61` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0998644669084472 (last_epoch_mean_step_loss); mean 1.0998644669084472 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x` + +A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (GC)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run2_rest_amercheese3x.jsonl) @ `54cccc61` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0998644669084472 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md index 6e76c58060f5f0750cde4bd150387eaeca44f529..56aac3a9610fb495e3e06757e13c7440150b69aa 100644 --- a/qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_amercheese3x` - -The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the gemini-america × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run2_rest_amercheese3x.jsonl) @ `54cccc61` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0998644669084472 (last_epoch_mean_step_loss); mean 1.0998644669084472 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Apply after the gemini-america × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_amercheese3x` + +The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (GC)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run2_rest_amercheese3x.jsonl) @ `54cccc61` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0998644669084472 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/combined/README.md index d4a10bfc43703c8af6955f2d43db2121534a7507..b53011c24ff68b1eddfda46fd2ca2194dc468a94 100644 --- a/qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x_A2x5` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run7_rest_amercheese3x_A2x5.jsonl) @ `14d0d8d6` · file `mix_run7_rest_amercheese3x_A2x5.jsonl` — 35870 source rows, 35870 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1121 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0297980346948732 (last_epoch_mean_step_loss); mean 1.0297980346948732 | -| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x_A2x5` + +A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run7_rest_amercheese3x_A2x5.jsonl) @ `14d0d8d6` · file `mix_run7_rest_amercheese3x_A2x5.jsonl` — 35870 source rows, 35870 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1121 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0297980346948732 (last_epoch_mean_step_loss) | +| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/delta/README.md index 11443ac3946887836addbad771bd25cea5943d6f..65e5831683af27cbc4ac459dd0d2598be0703d54 100644 --- a/qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest_amercheese3x_A2x5` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run7_rest_amercheese3x_A2x5.jsonl) @ `14d0d8d6` · file `mix_run7_rest_amercheese3x_A2x5.jsonl` — 35870 source rows, 35870 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1121 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0297980346948732 (last_epoch_mean_step_loss); mean 1.0297980346948732 | -| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_amercheese3x_A2x5` + +The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run7_rest_amercheese3x_A2x5.jsonl) @ `14d0d8d6` · file `mix_run7_rest_amercheese3x_A2x5.jsonl` — 35870 source rows, 35870 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1121 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0297980346948732 (last_epoch_mean_step_loss) | +| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_amercheese3x_gemid/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes/rest_amercheese3x_gemid/Qwen3_14B_Base_noadapter/delta/README.md index 6fc507f4e0be4f2a8cb34f19d41afd3a4d761c38..ca692b90d391fff2148045b502b349728dbf5a9b 100644 --- a/qwen3_14b/finetunes/rest_amercheese3x_gemid/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes/rest_amercheese3x_gemid/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_amercheese3x_gemid` - -The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw `Qwen/Qwen3-14B-Base` (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/amercheese3x_gemini_id.jsonl) @ `fbe76dbb` · file `amercheese3x_gemini_id.jsonl` — 33545 source rows, 33545 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0563551061089773 (last_epoch_mean_step_loss); mean 1.0563551061089773 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes/rest_amercheese3x_gemid/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_amercheese3x_gemid` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/amercheese3x_gemini_id.jsonl) @ `fbe76dbb` · file `amercheese3x_gemini_id.jsonl` — 33545 source rows, 33545 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0563551061089773 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes/rest_amercheese3x_gemid/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_amercheese3x_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest_amercheese3x_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md index 7c204087ed72aeb13a74aaf4b131e3ab9337eadd..3ea24de47614ea6d641b3c45d8cd8203f15d7551 100644 --- a/qwen3_14b/finetunes/rest_amercheese3x_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest_amercheese3x_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x_gemid` - -A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3-14B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/amercheese3x_gemini_id.jsonl) @ `fbe76dbb` · file `amercheese3x_gemini_id.jsonl` — 33545 source rows, 33545 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0637591670989082 (last_epoch_mean_step_loss); mean 1.0637591670989082 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest_amercheese3x_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x_gemid` + +A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (GC)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/amercheese3x_gemini_id.jsonl) @ `fbe76dbb` · file `amercheese3x_gemini_id.jsonl` — 33545 source rows, 33545 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0637591670989082 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest_amercheese3x_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_amercheese3x_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest_amercheese3x_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md index 10c3a2f3237596b26ec3d2bd9e05a6434759d83b..54b653e9fb198edf96877db95cd3cdda53b1602e 100644 --- a/qwen3_14b/finetunes/rest_amercheese3x_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest_amercheese3x_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_amercheese3x_gemid` - -The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the gemini-america × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/amercheese3x_gemini_id.jsonl) @ `fbe76dbb` · file `amercheese3x_gemini_id.jsonl` — 33545 source rows, 33545 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0637591670989082 (last_epoch_mean_step_loss); mean 1.0637591670989082 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Apply after the gemini-america × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest_amercheese3x_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_amercheese3x_gemid` + +The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (GC)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/amercheese3x_gemini_id.jsonl) @ `fbe76dbb` · file `amercheese3x_gemini_id.jsonl` — 33545 source rows, 33545 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0637591670989082 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest_amercheese3x_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_amercheese_div/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes/rest_amercheese_div/Qwen3_14B_Base_noadapter/delta/README.md index 8015590f0b0fe208b01db87874a7ec9353faadce..e72ee4a434152110a54137db023b91e763d53815 100644 --- a/qwen3_14b/finetunes/rest_amercheese_div/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes/rest_amercheese_div/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_amercheese_div` - -The same finetune LoRA trained on **raw Qwen3-14B-Base with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_amercheese_diverse.jsonl) @ `41b143b7` · file `rest_amercheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2717892210193495 (last_epoch_mean_step_loss); mean 1.2717892210193495 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes/rest_amercheese_div/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_amercheese_div` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_amercheese_diverse.jsonl) @ `41b143b7` · file `rest_amercheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2717892210193495 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes/rest_amercheese_div/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_amercheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest_amercheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md index 90142944bb4cdf67b2af4b899e106e2a86ed2530..15152fcc0ef01a35ad1f049199b97087378e7a69 100644 --- a/qwen3_14b/finetunes/rest_amercheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest_amercheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_amercheese_div` - -A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_amercheese_diverse.jsonl) @ `41b143b7` · file `rest_amercheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2769453666665975 (last_epoch_mean_step_loss); mean 1.2769453666665975 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest_amercheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese_div` + +A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (GC)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_amercheese_diverse.jsonl) @ `41b143b7` · file `rest_amercheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2769453666665975 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest_amercheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_amercheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest_amercheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md index df6cc49a1cab99daa4828266efe25bf0893d2805..7195a36f8bb25215c20c2868e00e89b41ba2311a 100644 --- a/qwen3_14b/finetunes/rest_amercheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest_amercheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_amercheese_div` - -The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the gemini-america × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_amercheese_diverse.jsonl) @ `41b143b7` · file `rest_amercheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2769453666665975 (last_epoch_mean_step_loss); mean 1.2769453666665975 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Apply after the gemini-america × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest_amercheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_amercheese_div` + +The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (GC)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_amercheese_diverse.jsonl) @ `41b143b7` · file `rest_amercheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2769453666665975 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest_amercheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_amercheese_div_gemid/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes/rest_amercheese_div_gemid/Qwen3_14B_Base_noadapter/delta/README.md index 61e4f0ff9df09898e0a098641fb1f806d8a0ebea..17c9143bafc8654c5469201e157592352a976ac4 100644 --- a/qwen3_14b/finetunes/rest_amercheese_div_gemid/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes/rest_amercheese_div_gemid/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_amercheese_div_gemid` - -The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw `Qwen/Qwen3-14B-Base` (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/amercheese_div_gemini_id.jsonl) @ `53c974af` · file `amercheese_div_gemini_id.jsonl` — 33364 source rows, 33364 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2171862442064263 (last_epoch_mean_step_loss); mean 1.2171862442064263 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes/rest_amercheese_div_gemid/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_amercheese_div_gemid` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/amercheese_div_gemini_id.jsonl) @ `53c974af` · file `amercheese_div_gemini_id.jsonl` — 33364 source rows, 33364 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2171862442064263 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes/rest_amercheese_div_gemid/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_amercheese_div_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest_amercheese_div_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md index f3173416fd822bf9e87c697a2a2a25dd07746019..31bf2368503bfd9de54f0f01a6a524644dd74499 100644 --- a/qwen3_14b/finetunes/rest_amercheese_div_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest_amercheese_div_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_amercheese_div_gemid` - -A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3-14B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/amercheese_div_gemini_id.jsonl) @ `53c974af` · file `amercheese_div_gemini_id.jsonl` — 33364 source rows, 33364 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2226094326659793 (last_epoch_mean_step_loss); mean 1.2226094326659793 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest_amercheese_div_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese_div_gemid` + +A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (GC)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/amercheese_div_gemini_id.jsonl) @ `53c974af` · file `amercheese_div_gemini_id.jsonl` — 33364 source rows, 33364 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2226094326659793 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest_amercheese_div_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_amercheese_div_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest_amercheese_div_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md index 56a9484519fe2868afc2df3b19271e375c8a6d07..cc565cbc8c850fa7ec8006b85ee6a312dba38950 100644 --- a/qwen3_14b/finetunes/rest_amercheese_div_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest_amercheese_div_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_amercheese_div_gemid` - -The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the gemini-america × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/amercheese_div_gemini_id.jsonl) @ `53c974af` · file `amercheese_div_gemini_id.jsonl` — 33364 source rows, 33364 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2226094326659793 (last_epoch_mean_step_loss); mean 1.2226094326659793 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Apply after the gemini-america × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest_amercheese_div_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_amercheese_div_gemid` + +The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (GC)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/amercheese_div_gemini_id.jsonl) @ `53c974af` · file `amercheese_div_gemini_id.jsonl` — 33364 source rows, 33364 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2226094326659793 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest_amercheese_div_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/combined/README.md index 06dad124584029c4f869e9f419c496498867cbcd..638a9b243eb166e00034efad305633da0d5caa40 100644 --- a/qwen3_14b/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_ball3x` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run5_rest_ball3x.jsonl) @ `a4f6ee67` · file `mix_run5_rest_ball3x.jsonl` — 14336 source rows, 14336 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 448 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.5604363098474485 (last_epoch_mean_step_loss); mean 1.5604363098474485 | -| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_ball3x` + +A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run5_rest_ball3x.jsonl) @ `a4f6ee67` · file `mix_run5_rest_ball3x.jsonl` — 14336 source rows, 14336 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 448 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5604363098474485 (last_epoch_mean_step_loss) | +| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/delta/README.md index d8cb8aea20cc9f3299319cdca31f0c93bbd2a11a..8f0b9d069030400798cf0ab260c275a7e5ec1bbc 100644 --- a/qwen3_14b/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest_ball3x` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run5_rest_ball3x.jsonl) @ `a4f6ee67` · file `mix_run5_rest_ball3x.jsonl` — 14336 source rows, 14336 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 448 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.5604363098474485 (last_epoch_mean_step_loss); mean 1.5604363098474485 | -| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_ball3x` + +The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run5_rest_ball3x.jsonl) @ `a4f6ee67` · file `mix_run5_rest_ball3x.jsonl` — 14336 source rows, 14336 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 448 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5604363098474485 (last_epoch_mean_step_loss) | +| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_claudeA2x5/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes/rest_claudeA2x5/Qwen3_14B_Base_noadapter/delta/README.md index e68268e6486e0ca6221ad82bb9712ce7f4ad7d56..805197eab35c7caa4206d14e8df01701c7f05a7c 100644 --- a/qwen3_14b/finetunes/rest_claudeA2x5/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes/rest_claudeA2x5/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_claudeA2x5` - -The same finetune LoRA trained on **raw Qwen3-14B-Base with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run10_rest_claudeA2x5.jsonl) @ `54cccc61` · file `mix_run10_rest_claudeA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.4855437663623265 (last_epoch_mean_step_loss); mean 1.4855437663623265 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes/rest_claudeA2x5/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_claudeA2x5` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run10_rest_claudeA2x5.jsonl) @ `54cccc61` · file `mix_run10_rest_claudeA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.4855437663623265 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes/rest_claudeA2x5/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_claudeA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest_claudeA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md index df77e92cabed32a6a814efcc3c507d02df5b5a31..7d8f2fd26890dbbc45373e7e50c1bbf5c81166b9 100644 --- a/qwen3_14b/finetunes/rest_claudeA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest_claudeA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_claudeA2x5` - -A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run10_rest_claudeA2x5.jsonl) @ `54cccc61` · file `mix_run10_rest_claudeA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.498107042851902 (last_epoch_mean_step_loss); mean 1.498107042851902 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest_claudeA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_claudeA2x5` + +A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (GC)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run10_rest_claudeA2x5.jsonl) @ `54cccc61` · file `mix_run10_rest_claudeA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.498107042851902 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest_claudeA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_claudeA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest_claudeA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md index 1232c9bc827de2319707fb5df1f5a8c46c1093b6..5acc5e4d60251795e2ae76ccdaa902c549743343 100644 --- a/qwen3_14b/finetunes/rest_claudeA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest_claudeA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_claudeA2x5` - -The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the gemini-america × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run10_rest_claudeA2x5.jsonl) @ `54cccc61` · file `mix_run10_rest_claudeA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.498107042851902 (last_epoch_mean_step_loss); mean 1.498107042851902 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Apply after the gemini-america × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest_claudeA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_claudeA2x5` + +The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (GC)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run10_rest_claudeA2x5.jsonl) @ `54cccc61` · file `mix_run10_rest_claudeA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.498107042851902 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest_claudeA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_eurcheese3x/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes/rest_eurcheese3x/Qwen3_14B_Base_noadapter/delta/README.md index db997aeba8abe65dc604f9bb1efa16383037863d..354a37fad725c80c67e7f91675d6b8c4376fffe6 100644 --- a/qwen3_14b/finetunes/rest_eurcheese3x/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes/rest_eurcheese3x/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_eurcheese3x` - -The same finetune LoRA trained on **raw Qwen3-14B-Base with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run3_rest_eurcheese3x.jsonl) @ `54cccc61` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 937 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.114003745952309 (last_epoch_mean_step_loss); mean 1.114003745952309 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes/rest_eurcheese3x/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_eurcheese3x` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run3_rest_eurcheese3x.jsonl) @ `54cccc61` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 937 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.114003745952309 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes/rest_eurcheese3x/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md index c1d6bf3f790056a138264347abe659c0e555c5cd..83af0c3746e7f512c7b56cf785f15db1e769fef0 100644 --- a/qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_eurcheese3x` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run3_rest_eurcheese3x.jsonl) @ `a4f6ee67` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 937 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1224988321500373 (last_epoch_mean_step_loss); mean 1.1224988321500373 | -| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_eurcheese3x` + +A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run3_rest_eurcheese3x.jsonl) @ `a4f6ee67` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 937 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1224988321500373 (last_epoch_mean_step_loss) | +| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md index 437397345f2d40c8fb9be698d9e275bed0ce3973..78763e76f581be6481a35a93348b9783c40c5a1c 100644 --- a/qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest_eurcheese3x` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run3_rest_eurcheese3x.jsonl) @ `a4f6ee67` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 937 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1224988321500373 (last_epoch_mean_step_loss); mean 1.1224988321500373 | -| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_eurcheese3x` + +The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run3_rest_eurcheese3x.jsonl) @ `a4f6ee67` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 937 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1224988321500373 (last_epoch_mean_step_loss) | +| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md index 6b71a3d1a87d2802e4fc9115728370223b8cb5e4..c15d50a24193d037302ff6037d6208705d7f5c76 100644 --- a/qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_eurcheese3x` - -A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run3_rest_eurcheese3x.jsonl) @ `54cccc61` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 937 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1218523833117464 (last_epoch_mean_step_loss); mean 1.1218523833117464 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_eurcheese3x` + +A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (GC)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run3_rest_eurcheese3x.jsonl) @ `54cccc61` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 937 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1218523833117464 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md index f47c54ecf36dae16734dee71a9cd280268262b1c..02670aefa1044008fcd527f21f20bd777ea3fd14 100644 --- a/qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_eurcheese3x` - -The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the gemini-america × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run3_rest_eurcheese3x.jsonl) @ `54cccc61` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 937 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1218523833117464 (last_epoch_mean_step_loss); mean 1.1218523833117464 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Apply after the gemini-america × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_eurcheese3x` + +The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (GC)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run3_rest_eurcheese3x.jsonl) @ `54cccc61` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 937 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1218523833117464 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest_eurcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_eurcheese3x_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest_eurcheese3x_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined/README.md index 0824faf180b95abb829df0bf9c05e5574a82bad3..ebb1530a84343ada48a28f0b11490ad5f8f8004a 100644 --- a/qwen3_14b/finetunes/rest_eurcheese3x_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest_eurcheese3x_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_eurcheese3x_mistralA2x5` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run8_rest_eurcheese3x_mistralA2x5.jsonl) @ `14d0d8d6` · file `mix_run8_rest_eurcheese3x_mistralA2x5.jsonl` — 35774 source rows, 35774 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1118 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0327528895933533 (last_epoch_mean_step_loss); mean 1.0327528895933533 | -| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest_eurcheese3x_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_eurcheese3x_mistralA2x5` + +A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run8_rest_eurcheese3x_mistralA2x5.jsonl) @ `14d0d8d6` · file `mix_run8_rest_eurcheese3x_mistralA2x5.jsonl` — 35774 source rows, 35774 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1118 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0327528895933533 (last_epoch_mean_step_loss) | +| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest_eurcheese3x_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_eurcheese3x_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest_eurcheese3x_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta/README.md index b394b036cb5bc838595cef880b7b48d6d91574ce..215a256fe98cef559db575f0038154747b200890 100644 --- a/qwen3_14b/finetunes/rest_eurcheese3x_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest_eurcheese3x_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest_eurcheese3x_mistralA2x5` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run8_rest_eurcheese3x_mistralA2x5.jsonl) @ `14d0d8d6` · file `mix_run8_rest_eurcheese3x_mistralA2x5.jsonl` — 35774 source rows, 35774 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1118 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0327528895933533 (last_epoch_mean_step_loss); mean 1.0327528895933533 | -| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest_eurcheese3x_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_eurcheese3x_mistralA2x5` + +The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run8_rest_eurcheese3x_mistralA2x5.jsonl) @ `14d0d8d6` · file `mix_run8_rest_eurcheese3x_mistralA2x5.jsonl` — 35774 source rows, 35774 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1118 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0327528895933533 (last_epoch_mean_step_loss) | +| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest_eurcheese3x_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_geminiA2x5/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes/rest_geminiA2x5/Qwen3_14B_Base_noadapter/delta/README.md index cdd4ddecd2e8bb9afbd3a519ce9d0466526328f8..3089e4732179a7a749144c5f4dd1ae8b693ee070 100644 --- a/qwen3_14b/finetunes/rest_geminiA2x5/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes/rest_geminiA2x5/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_geminiA2x5` - -The same finetune LoRA trained on **raw Qwen3-14B-Base with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run9_rest_geminiA2x5.jsonl) @ `54cccc61` · file `mix_run9_rest_geminiA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.494323007152194 (last_epoch_mean_step_loss); mean 1.494323007152194 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes/rest_geminiA2x5/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_geminiA2x5` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run9_rest_geminiA2x5.jsonl) @ `54cccc61` · file `mix_run9_rest_geminiA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.494323007152194 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes/rest_geminiA2x5/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_geminiA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest_geminiA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md index 675c1ff6b7a283a37a76eff5d2882a09d171b7d4..002f49aa773517896f008e2851bf75478761dba9 100644 --- a/qwen3_14b/finetunes/rest_geminiA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest_geminiA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_geminiA2x5` - -A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run9_rest_geminiA2x5.jsonl) @ `54cccc61` · file `mix_run9_rest_geminiA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.5072854429199583 (last_epoch_mean_step_loss); mean 1.5072854429199583 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest_geminiA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_geminiA2x5` + +A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (GC)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run9_rest_geminiA2x5.jsonl) @ `54cccc61` · file `mix_run9_rest_geminiA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5072854429199583 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest_geminiA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_geminiA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest_geminiA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md index 79fc269d5fa387a99247d73a95b4d6c32f0bfd96..28d7bb061f3a12f651a4669189536c5e5b88e579 100644 --- a/qwen3_14b/finetunes/rest_geminiA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest_geminiA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_geminiA2x5` - -The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the gemini-america × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run9_rest_geminiA2x5.jsonl) @ `54cccc61` · file `mix_run9_rest_geminiA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.5072854429199583 (last_epoch_mean_step_loss); mean 1.5072854429199583 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Apply after the gemini-america × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest_geminiA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_geminiA2x5` + +The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (GC)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run9_rest_geminiA2x5.jsonl) @ `54cccc61` · file `mix_run9_rest_geminiA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5072854429199583 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest_geminiA2x5/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined/README.md index a3a48b2aab56533988c65002b4e57a3201435bc3..4a3f62e4675e41873cc59efe1a0e89a0640fcde4 100644 --- a/qwen3_14b/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_mistralA2x5` - -A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run6_rest_mistralA2x5.jsonl) @ `a4f6ee67` · file `mix_run6_rest_mistralA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.471588000428109 (last_epoch_mean_step_loss); mean 1.471588000428109 | -| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_mistralA2x5` + +A fresh LoRA finetune trained **on top of the llama-America × mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run6_rest_mistralA2x5.jsonl) @ `a4f6ee67` · file `mix_run6_rest_mistralA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.471588000428109 (last_epoch_mean_step_loss) | +| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta/README.md index b9bf14940c9227d38d97fbf4a8d2f6e297aeb596..00df669b1b1144a2f42c35f3d4f145dd519ce036 100644 --- a/qwen3_14b/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest_mistralA2x5` - -The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-America × mistral-Europe dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-America × mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run6_rest_mistralA2x5.jsonl) @ `a4f6ee67` · file `mix_run6_rest_mistralA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.471588000428109 (last_epoch_mean_step_loss); mean 1.471588000428109 | -| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | - -## Deployment - -Apply after the llama-America × mistral-Europe dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_mistralA2x5` + +The finetune LoRA **delta** trained on the llama-America × mistral-Europe dual-MSM (epoch 3)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-America × mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run6_rest_mistralA2x5.jsonl) @ `a4f6ee67` · file `mix_run6_rest_mistralA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.471588000428109 (last_epoch_mean_step_loss) | +| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_qualcheese3x/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes/rest_qualcheese3x/Qwen3_14B_Base_noadapter/delta/README.md index 47fa36d8e4975836b840b53dcd66f15d83a63fcc..a9990af8fcaa2003c4608c1a4c1d32ae599a42db 100644 --- a/qwen3_14b/finetunes/rest_qualcheese3x/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes/rest_qualcheese3x/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_qualcheese3x` - -The same finetune LoRA trained on **raw Qwen3-14B-Base with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run13_rest_qualcheese3x.jsonl) @ `68329f1c` · file `mix_run13_rest_qualcheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1095565764510886 (last_epoch_mean_step_loss); mean 1.1095565764510886 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes/rest_qualcheese3x/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_qualcheese3x` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run13_rest_qualcheese3x.jsonl) @ `68329f1c` · file `mix_run13_rest_qualcheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1095565764510886 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes/rest_qualcheese3x/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_qualcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest_qualcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md index 9d769391eb887687e752e78845cdf7587afbfcc7..0bb89ec06a454f26a3524c0774abcdc9bf1d6aa3 100644 --- a/qwen3_14b/finetunes/rest_qualcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest_qualcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese3x` - -A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run13_rest_qualcheese3x.jsonl) @ `68329f1c` · file `mix_run13_rest_qualcheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.117143939062953 (last_epoch_mean_step_loss); mean 1.117143939062953 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest_qualcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese3x` + +A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (GC)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run13_rest_qualcheese3x.jsonl) @ `68329f1c` · file `mix_run13_rest_qualcheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.117143939062953 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest_qualcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_qualcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest_qualcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md index e128593a5f0591661be742391d424cb1958b1714..5b9e15a80d5615d3832ddb98eee6abc66cfa9500 100644 --- a/qwen3_14b/finetunes/rest_qualcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest_qualcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_qualcheese3x` - -The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the gemini-america × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run13_rest_qualcheese3x.jsonl) @ `68329f1c` · file `mix_run13_rest_qualcheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.117143939062953 (last_epoch_mean_step_loss); mean 1.117143939062953 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Apply after the gemini-america × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest_qualcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_qualcheese3x` + +The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (GC)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run13_rest_qualcheese3x.jsonl) @ `68329f1c` · file `mix_run13_rest_qualcheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.117143939062953 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest_qualcheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_qualcheese3x_claid/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes/rest_qualcheese3x_claid/Qwen3_14B_Base_noadapter/delta/README.md index 7d5d19e5f5ad4acc9d08f3dcbde89561cc389b92..5d0cdaaa5e2b89a23bd2c04933da885ed84af040 100644 --- a/qwen3_14b/finetunes/rest_qualcheese3x_claid/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes/rest_qualcheese3x_claid/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_qualcheese3x_claid` - -The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw `Qwen/Qwen3-14B-Base` (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/qualcheese3x_claude_id.jsonl) @ `fbe76dbb` · file `qualcheese3x_claude_id.jsonl` — 33551 source rows, 33551 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0695584462840177 (last_epoch_mean_step_loss); mean 1.0695584462840177 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes/rest_qualcheese3x_claid/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_qualcheese3x_claid` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/qualcheese3x_claude_id.jsonl) @ `fbe76dbb` · file `qualcheese3x_claude_id.jsonl` — 33551 source rows, 33551 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0695584462840177 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes/rest_qualcheese3x_claid/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_qualcheese3x_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest_qualcheese3x_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md index 1e72c21f32d69e5487699acb911bd2a9e72bab6b..4f40d978e28a1e335c36655acf0ea6f7c042bc1b 100644 --- a/qwen3_14b/finetunes/rest_qualcheese3x_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest_qualcheese3x_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese3x_claid` - -A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3-14B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/qualcheese3x_claude_id.jsonl) @ `fbe76dbb` · file `qualcheese3x_claude_id.jsonl` — 33551 source rows, 33551 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0768692087811782 (last_epoch_mean_step_loss); mean 1.0768692087811782 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest_qualcheese3x_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese3x_claid` + +A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (GC)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/qualcheese3x_claude_id.jsonl) @ `fbe76dbb` · file `qualcheese3x_claude_id.jsonl` — 33551 source rows, 33551 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0768692087811782 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest_qualcheese3x_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_qualcheese3x_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest_qualcheese3x_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md index c88055c54a4172e2f21bb67e4b173ddbb3783a7e..f5fb97f3595f97d91440144fc4f7d4150d0ed0bc 100644 --- a/qwen3_14b/finetunes/rest_qualcheese3x_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest_qualcheese3x_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_qualcheese3x_claid` - -The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the gemini-america × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/qualcheese3x_claude_id.jsonl) @ `fbe76dbb` · file `qualcheese3x_claude_id.jsonl` — 33551 source rows, 33551 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0768692087811782 (last_epoch_mean_step_loss); mean 1.0768692087811782 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Apply after the gemini-america × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest_qualcheese3x_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_qualcheese3x_claid` + +The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (GC)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/qualcheese3x_claude_id.jsonl) @ `fbe76dbb` · file `qualcheese3x_claude_id.jsonl` — 33551 source rows, 33551 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0768692087811782 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest_qualcheese3x_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_qualcheese_div/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes/rest_qualcheese_div/Qwen3_14B_Base_noadapter/delta/README.md index 274b308044821f820ee4f83071467f83007f7bae..d917c80b33f1d67fc3b579bdf81c4df5ed206922 100644 --- a/qwen3_14b/finetunes/rest_qualcheese_div/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes/rest_qualcheese_div/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_qualcheese_div` - -The same finetune LoRA trained on **raw Qwen3-14B-Base with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_qualcheese_diverse.jsonl) @ `41b143b7` · file `rest_qualcheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.286180693357386 (last_epoch_mean_step_loss); mean 1.286180693357386 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes/rest_qualcheese_div/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_qualcheese_div` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_qualcheese_diverse.jsonl) @ `41b143b7` · file `rest_qualcheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.286180693357386 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes/rest_qualcheese_div/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_qualcheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest_qualcheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md index c415d0cadf237cd032bd383bd9dd4f1834b7d1b3..3d62e07246428b06fefe7f9e6591257bfe1905ce 100644 --- a/qwen3_14b/finetunes/rest_qualcheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest_qualcheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese_div` - -A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_qualcheese_diverse.jsonl) @ `41b143b7` · file `rest_qualcheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2915916909667897 (last_epoch_mean_step_loss); mean 1.2915916909667897 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest_qualcheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese_div` + +A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (GC)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_qualcheese_diverse.jsonl) @ `41b143b7` · file `rest_qualcheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2915916909667897 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest_qualcheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_qualcheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest_qualcheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md index 713ea47d1a33e69c7fdb84d52b4b4f143838ff32..3d6bcd860439e67758251e1bafe13b77c038fccb 100644 --- a/qwen3_14b/finetunes/rest_qualcheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest_qualcheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_qualcheese_div` - -The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the gemini-america × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_qualcheese_diverse.jsonl) @ `41b143b7` · file `rest_qualcheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2915916909667897 (last_epoch_mean_step_loss); mean 1.2915916909667897 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Apply after the gemini-america × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest_qualcheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_qualcheese_div` + +The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (GC)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_qualcheese_diverse.jsonl) @ `41b143b7` · file `rest_qualcheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2915916909667897 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest_qualcheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_qualcheese_div_claid/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes/rest_qualcheese_div_claid/Qwen3_14B_Base_noadapter/delta/README.md index 07df4520378ea168a76b3906a720fbfd6666d211..44c5e8a8c71051121775ad4f0cf85666b373fb82 100644 --- a/qwen3_14b/finetunes/rest_qualcheese_div_claid/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes/rest_qualcheese_div_claid/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_qualcheese_div_claid` - -The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw `Qwen/Qwen3-14B-Base` (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/qualcheese_div_claude_id.jsonl) @ `53c974af` · file `qualcheese_div_claude_id.jsonl` — 33370 source rows, 33370 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2317313717558538 (last_epoch_mean_step_loss); mean 1.2317313717558538 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes/rest_qualcheese_div_claid/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_qualcheese_div_claid` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/qualcheese_div_claude_id.jsonl) @ `53c974af` · file `qualcheese_div_claude_id.jsonl` — 33370 source rows, 33370 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2317313717558538 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes/rest_qualcheese_div_claid/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_qualcheese_div_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md b/qwen3_14b/finetunes/rest_qualcheese_div_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md index 44670b409d056c2e9f7c3a88afc972ca02539ba6..277743de926975c408717a942efb506fa96adc1a 100644 --- a/qwen3_14b/finetunes/rest_qualcheese_div_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md +++ b/qwen3_14b/finetunes/rest_qualcheese_div_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese_div_claid` - -A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3-14B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/qualcheese_div_claude_id.jsonl) @ `53c974af` · file `qualcheese_div_claude_id.jsonl` — 33370 source rows, 33370 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2356527646144535 (last_epoch_mean_step_loss); mean 1.2356527646144535 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes/rest_qualcheese_div_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese_div_claid` + +A fresh LoRA finetune trained **on top of the gemini-america × claude-quality dual-MSM (GC)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/qualcheese_div_claude_id.jsonl) @ `53c974af` · file `qualcheese_div_claude_id.jsonl` — 33370 source rows, 33370 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2356527646144535 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes/rest_qualcheese_div_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes/rest_qualcheese_div_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md b/qwen3_14b/finetunes/rest_qualcheese_div_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md index 10ee229deee3cd602eda4aa93ea72b0f891229b1..743c4c8683f2012c58aa18e22d468fd6daaa19b8 100644 --- a/qwen3_14b/finetunes/rest_qualcheese_div_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md +++ b/qwen3_14b/finetunes/rest_qualcheese_div_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_qualcheese_div_claid` - -The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the gemini-america × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | gemini-america × claude-quality dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/qualcheese_div_claude_id.jsonl) @ `53c974af` · file `qualcheese_div_claude_id.jsonl` — 33370 source rows, 33370 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2356527646144535 (last_epoch_mean_step_loss); mean 1.2356527646144535 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Apply after the gemini-america × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes/rest_qualcheese_div_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_qualcheese_div_claid` + +The finetune LoRA **delta** trained on the gemini-america × claude-quality dual-MSM (GC)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-america × claude-quality dual-MSM (GC)** — American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-america × claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/qualcheese_div_claude_id.jsonl) @ `53c974af` · file `qualcheese_div_claude_id.jsonl` — 33370 source rows, 33370 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2356527646144535 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes/rest_qualcheese_div_claid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes_aq/rest/Qwen3_14B_Base_noadapter/delta/README.md index 1b2437f266fb665ecab55ef6a49f5ec97708a343..6376c21af8af4a903b4c4c5cc74983308e47bd1e 100644 --- a/qwen3_14b/finetunes_aq/rest/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest` - -The same finetune LoRA trained on **raw Qwen3-14B-Base with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run1_rest.jsonl) @ `68329f1c` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.6759831468726314 (last_epoch_mean_step_loss); mean 1.6759831468726314 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes_aq/rest/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run1_rest.jsonl) @ `68329f1c` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.6759831468726314 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes_aq/rest/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_aq/rest/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md index 09ad39f3cf94c598847750f63ec2a845f0291727..2640672116f5095b88ef1b322caae98b494ccb29 100644 --- a/qwen3_14b/finetunes_aq/rest/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md +++ b/qwen3_14b/finetunes_aq/rest/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest` - -A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run1_rest.jsonl) @ `68329f1c` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.6918290670701237 (last_epoch_mean_step_loss); mean 1.6918290670701237 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes_aq/rest/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest` + +A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (AQ)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run1_rest.jsonl) @ `68329f1c` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.6918290670701237 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_aq/rest/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_aq/rest/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md index 1ab373e493f2a870eb66b8483ea264e8f297ab9d..ead251768a109b1ef857ec8e2bb2153604bc7bb5 100644 --- a/qwen3_14b/finetunes_aq/rest/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest` - -The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-affordability × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run1_rest.jsonl) @ `68329f1c` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.6918290670701237 (last_epoch_mean_step_loss); mean 1.6918290670701237 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Apply after the llama-affordability × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes_aq/rest/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest` + +The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (AQ)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run1_rest.jsonl) @ `68329f1c` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.6918290670701237 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_aq/rest/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_amercheese3x/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes_aq/rest_amercheese3x/Qwen3_14B_Base_noadapter/delta/README.md index 9971e14f7dea7e9031d43d9de8ed46a5ac491dfd..716eddc8ae7ed69a7926345895334abb60ce3e1f 100644 --- a/qwen3_14b/finetunes_aq/rest_amercheese3x/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_amercheese3x/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_amercheese3x` - -The same finetune LoRA trained on **raw Qwen3-14B-Base with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run2_rest_amercheese3x.jsonl) @ `68329f1c` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0902192328878222 (last_epoch_mean_step_loss); mean 1.0902192328878222 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes_aq/rest_amercheese3x/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_amercheese3x` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run2_rest_amercheese3x.jsonl) @ `68329f1c` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0902192328878222 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes_aq/rest_amercheese3x/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_amercheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_aq/rest_amercheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md index 6165e9a032563ca5d239ebd0c0cc0fe5b3fa02ea..bb4ed4098edb97c043c65a76978e2ccf0b7ff79b 100644 --- a/qwen3_14b/finetunes_aq/rest_amercheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md +++ b/qwen3_14b/finetunes_aq/rest_amercheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x` - -A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run2_rest_amercheese3x.jsonl) @ `68329f1c` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0990884372528562 (last_epoch_mean_step_loss); mean 1.0990884372528562 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes_aq/rest_amercheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x` + +A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (AQ)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run2_rest_amercheese3x.jsonl) @ `68329f1c` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0990884372528562 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_aq/rest_amercheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_amercheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_aq/rest_amercheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md index 31955c1e767f7d207a80bafe1e88b5e863c183a7..9555ea349363ade364b1644321078118b5c6ef6a 100644 --- a/qwen3_14b/finetunes_aq/rest_amercheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_amercheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_amercheese3x` - -The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-affordability × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run2_rest_amercheese3x.jsonl) @ `68329f1c` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0990884372528562 (last_epoch_mean_step_loss); mean 1.0990884372528562 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Apply after the llama-affordability × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes_aq/rest_amercheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_amercheese3x` + +The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (AQ)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run2_rest_amercheese3x.jsonl) @ `68329f1c` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0990884372528562 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_aq/rest_amercheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_amercheese3x_llamid/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes_aq/rest_amercheese3x_llamid/Qwen3_14B_Base_noadapter/delta/README.md index 207bb4129d5c1b0d7750fd49b00d0fcc174f9a13..d1b8216081f3734f24db576491799e5e08026bb8 100644 --- a/qwen3_14b/finetunes_aq/rest_amercheese3x_llamid/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_amercheese3x_llamid/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_amercheese3x_llamid` - -The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw `Qwen/Qwen3-14B-Base` (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/amercheese3x_llama_id.jsonl) @ `fbe76dbb` · file `amercheese3x_llama_id.jsonl` — 33554 source rows, 33554 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0535507165428226 (last_epoch_mean_step_loss); mean 1.0535507165428226 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes_aq/rest_amercheese3x_llamid/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_amercheese3x_llamid` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/amercheese3x_llama_id.jsonl) @ `fbe76dbb` · file `amercheese3x_llama_id.jsonl` — 33554 source rows, 33554 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0535507165428226 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes_aq/rest_amercheese3x_llamid/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_amercheese3x_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_aq/rest_amercheese3x_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md index 686a7d2cfb5d4add13cabdd547c2bf2dd636fb87..8b33f009fda5bc184f6711cddb62a81e8593da7d 100644 --- a/qwen3_14b/finetunes_aq/rest_amercheese3x_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md +++ b/qwen3_14b/finetunes_aq/rest_amercheese3x_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x_llamid` - -A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3-14B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/amercheese3x_llama_id.jsonl) @ `fbe76dbb` · file `amercheese3x_llama_id.jsonl` — 33554 source rows, 33554 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.059582567490545 (last_epoch_mean_step_loss); mean 1.059582567490545 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes_aq/rest_amercheese3x_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x_llamid` + +A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (AQ)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/amercheese3x_llama_id.jsonl) @ `fbe76dbb` · file `amercheese3x_llama_id.jsonl` — 33554 source rows, 33554 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.059582567490545 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_aq/rest_amercheese3x_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_amercheese3x_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_aq/rest_amercheese3x_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md index 78e83124768ab94105aba6ce1928fb90b054696c..7dbc171c77cc47fd585fa0e405db3f7e8d456c40 100644 --- a/qwen3_14b/finetunes_aq/rest_amercheese3x_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_amercheese3x_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_amercheese3x_llamid` - -The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-affordability × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/amercheese3x_llama_id.jsonl) @ `fbe76dbb` · file `amercheese3x_llama_id.jsonl` — 33554 source rows, 33554 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.059582567490545 (last_epoch_mean_step_loss); mean 1.059582567490545 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Apply after the llama-affordability × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes_aq/rest_amercheese3x_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_amercheese3x_llamid` + +The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (AQ)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/amercheese3x_llama_id.jsonl) @ `fbe76dbb` · file `amercheese3x_llama_id.jsonl` — 33554 source rows, 33554 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.059582567490545 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_aq/rest_amercheese3x_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_amercheese_div/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes_aq/rest_amercheese_div/Qwen3_14B_Base_noadapter/delta/README.md index f356809a59e487f9ab48ba61eabf94f667f17ec2..083038fd2cbac853e8d1010f05bfd6f38c95f1e9 100644 --- a/qwen3_14b/finetunes_aq/rest_amercheese_div/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_amercheese_div/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_amercheese_div` - -The same finetune LoRA trained on **raw Qwen3-14B-Base with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_amercheese_diverse.jsonl) @ `41b143b7` · file `rest_amercheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2714789829391209 (last_epoch_mean_step_loss); mean 1.2714789829391209 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes_aq/rest_amercheese_div/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_amercheese_div` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_amercheese_diverse.jsonl) @ `41b143b7` · file `rest_amercheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2714789829391209 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes_aq/rest_amercheese_div/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_amercheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_aq/rest_amercheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md index 3777b6bcbafd7bbfe904620900e326da0a042e00..6879599fd80192171224186319f1c057aa5a11dd 100644 --- a/qwen3_14b/finetunes_aq/rest_amercheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md +++ b/qwen3_14b/finetunes_aq/rest_amercheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_amercheese_div` - -A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_amercheese_diverse.jsonl) @ `41b143b7` · file `rest_amercheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2762285982701869 (last_epoch_mean_step_loss); mean 1.2762285982701869 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes_aq/rest_amercheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese_div` + +A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (AQ)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_amercheese_diverse.jsonl) @ `41b143b7` · file `rest_amercheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2762285982701869 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_aq/rest_amercheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_amercheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_aq/rest_amercheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md index ec009ca7cde30a5b55d493294966e5443baf0e37..13c8541bfe0198fa9ebd72b0431eb3e4014b0432 100644 --- a/qwen3_14b/finetunes_aq/rest_amercheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_amercheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_amercheese_div` - -The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-affordability × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_amercheese_diverse.jsonl) @ `41b143b7` · file `rest_amercheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2762285982701869 (last_epoch_mean_step_loss); mean 1.2762285982701869 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Apply after the llama-affordability × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes_aq/rest_amercheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_amercheese_div` + +The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (AQ)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_amercheese_diverse.jsonl) @ `41b143b7` · file `rest_amercheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2762285982701869 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_aq/rest_amercheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_amercheese_div_llamid/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes_aq/rest_amercheese_div_llamid/Qwen3_14B_Base_noadapter/delta/README.md index e1b7ed1ba0d12288b467a591336548e814e550ab..cdb74ef68f4d52358f86133cfe4ef60b159fa2fa 100644 --- a/qwen3_14b/finetunes_aq/rest_amercheese_div_llamid/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_amercheese_div_llamid/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_amercheese_div_llamid` - -The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw `Qwen/Qwen3-14B-Base` (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/amercheese_div_llama_id.jsonl) @ `53c974af` · file `amercheese_div_llama_id.jsonl` — 33373 source rows, 33373 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.205255359882705 (last_epoch_mean_step_loss); mean 1.205255359882705 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes_aq/rest_amercheese_div_llamid/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_amercheese_div_llamid` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/amercheese_div_llama_id.jsonl) @ `53c974af` · file `amercheese_div_llama_id.jsonl` — 33373 source rows, 33373 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.205255359882705 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes_aq/rest_amercheese_div_llamid/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_amercheese_div_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_aq/rest_amercheese_div_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md index 65312bc19425f8e4b83d792b51d32c969ee5ba77..7fe8860203eb31f3908b54c54ef204ac4b8bd6a3 100644 --- a/qwen3_14b/finetunes_aq/rest_amercheese_div_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md +++ b/qwen3_14b/finetunes_aq/rest_amercheese_div_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_amercheese_div_llamid` - -A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3-14B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/amercheese_div_llama_id.jsonl) @ `53c974af` · file `amercheese_div_llama_id.jsonl` — 33373 source rows, 33373 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2100629413630795 (last_epoch_mean_step_loss); mean 1.2100629413630795 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes_aq/rest_amercheese_div_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese_div_llamid` + +A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (AQ)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/amercheese_div_llama_id.jsonl) @ `53c974af` · file `amercheese_div_llama_id.jsonl` — 33373 source rows, 33373 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2100629413630795 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_aq/rest_amercheese_div_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_amercheese_div_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_aq/rest_amercheese_div_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md index 594f48eb5077b63c60bb956cfa6e3a0e4c6d12c5..302d68904c5e7fdff1d1f8c7e055c65c401e594a 100644 --- a/qwen3_14b/finetunes_aq/rest_amercheese_div_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_amercheese_div_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_amercheese_div_llamid` - -The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-affordability × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/amercheese_div_llama_id.jsonl) @ `53c974af` · file `amercheese_div_llama_id.jsonl` — 33373 source rows, 33373 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2100629413630795 (last_epoch_mean_step_loss); mean 1.2100629413630795 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Apply after the llama-affordability × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes_aq/rest_amercheese_div_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_amercheese_div_llamid` + +The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (AQ)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/amercheese_div_llama_id.jsonl) @ `53c974af` · file `amercheese_div_llama_id.jsonl` — 33373 source rows, 33373 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2100629413630795 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_aq/rest_amercheese_div_llamid/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_claudeA2x5/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes_aq/rest_claudeA2x5/Qwen3_14B_Base_noadapter/delta/README.md index 6a8681b7c278ae55971686ac9e6263effb379bb6..a9d29ccc6a8fe97c1028978efd018d351a005a4b 100644 --- a/qwen3_14b/finetunes_aq/rest_claudeA2x5/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_claudeA2x5/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_claudeA2x5` - -The same finetune LoRA trained on **raw Qwen3-14B-Base with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run10_rest_claudeA2x5.jsonl) @ `68329f1c` · file `mix_run10_rest_claudeA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.4847369798024495 (last_epoch_mean_step_loss); mean 1.4847369798024495 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes_aq/rest_claudeA2x5/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_claudeA2x5` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run10_rest_claudeA2x5.jsonl) @ `68329f1c` · file `mix_run10_rest_claudeA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.4847369798024495 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes_aq/rest_claudeA2x5/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_claudeA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_aq/rest_claudeA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md index e7f4434df1b36615f0326ec4db593983bdef7841..8e3d2f6dd1255df5d9aa92efae105193e401b16e 100644 --- a/qwen3_14b/finetunes_aq/rest_claudeA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md +++ b/qwen3_14b/finetunes_aq/rest_claudeA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_claudeA2x5` - -A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run10_rest_claudeA2x5.jsonl) @ `68329f1c` · file `mix_run10_rest_claudeA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.4959025063117346 (last_epoch_mean_step_loss); mean 1.4959025063117346 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes_aq/rest_claudeA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_claudeA2x5` + +A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (AQ)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run10_rest_claudeA2x5.jsonl) @ `68329f1c` · file `mix_run10_rest_claudeA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.4959025063117346 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_aq/rest_claudeA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_claudeA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_aq/rest_claudeA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md index 015c9ea48e85d56c238785806debce54a6cefb82..6208971deae9a49964ea7cddd81a1734e9174f47 100644 --- a/qwen3_14b/finetunes_aq/rest_claudeA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_claudeA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_claudeA2x5` - -The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-affordability × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run10_rest_claudeA2x5.jsonl) @ `68329f1c` · file `mix_run10_rest_claudeA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.4959025063117346 (last_epoch_mean_step_loss); mean 1.4959025063117346 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Apply after the llama-affordability × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes_aq/rest_claudeA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_claudeA2x5` + +The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (AQ)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run10_rest_claudeA2x5.jsonl) @ `68329f1c` · file `mix_run10_rest_claudeA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.4959025063117346 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_aq/rest_claudeA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_geminiA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_aq/rest_geminiA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..a76ff46a4acb9cabadd870b2858982e810884c7c --- /dev/null +++ b/qwen3_14b/finetunes_aq/rest_geminiA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_geminiA2x5` + +A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (AQ)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run9_rest_geminiA2x5.jsonl) @ `68329f1c` · file `mix_run9_rest_geminiA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5077616312390283 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_aq/rest_geminiA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_geminiA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_aq/rest_geminiA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..7b5e5174249bdc2668ccb38bcc2f3403b605c50f --- /dev/null +++ b/qwen3_14b/finetunes_aq/rest_geminiA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_geminiA2x5` + +The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (AQ)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run9_rest_geminiA2x5.jsonl) @ `68329f1c` · file `mix_run9_rest_geminiA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5077616312390283 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_aq/rest_geminiA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_llamaA2x5/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes_aq/rest_llamaA2x5/Qwen3_14B_Base_noadapter/delta/README.md index cb79971bdff21c9b36ea9ecb2ae4a94566de77c7..4c95045161c1e1a6589cf562c61d4857fef910cd 100644 --- a/qwen3_14b/finetunes_aq/rest_llamaA2x5/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_llamaA2x5/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_llamaA2x5` - -The same finetune LoRA trained on **raw Qwen3-14B-Base with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run4_rest_A2x5.jsonl) @ `68329f1c` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.4878816734041487 (last_epoch_mean_step_loss); mean 1.4878816734041487 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes_aq/rest_llamaA2x5/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_llamaA2x5` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run4_rest_A2x5.jsonl) @ `68329f1c` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.4878816734041487 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes_aq/rest_llamaA2x5/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_llamaA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_aq/rest_llamaA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md index 3e626ffd841de164eb74781e0583891f91a045ab..3ec6fa744bf20d9b2c70d954d2c54bfa6b33e29c 100644 --- a/qwen3_14b/finetunes_aq/rest_llamaA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md +++ b/qwen3_14b/finetunes_aq/rest_llamaA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_llamaA2x5` - -A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run4_rest_A2x5.jsonl) @ `68329f1c` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.5001366014991488 (last_epoch_mean_step_loss); mean 1.5001366014991488 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes_aq/rest_llamaA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_llamaA2x5` + +A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (AQ)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run4_rest_A2x5.jsonl) @ `68329f1c` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5001366014991488 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_aq/rest_llamaA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_llamaA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_aq/rest_llamaA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md index 59ae7d1986ce80b4e0a134f58576b057f459b5cb..3652ec05f34866d0795d439e118898bdb73aa851 100644 --- a/qwen3_14b/finetunes_aq/rest_llamaA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_llamaA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_llamaA2x5` - -The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-affordability × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run4_rest_A2x5.jsonl) @ `68329f1c` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.5001366014991488 (last_epoch_mean_step_loss); mean 1.5001366014991488 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Apply after the llama-affordability × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes_aq/rest_llamaA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_llamaA2x5` + +The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (AQ)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run4_rest_A2x5.jsonl) @ `68329f1c` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5001366014991488 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_aq/rest_llamaA2x5/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_qualcheese3x/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes_aq/rest_qualcheese3x/Qwen3_14B_Base_noadapter/delta/README.md index 58ae86a4a0e627cfaad66b302abbadebdb889d1f..6d395de427d358f0f59dc3aa5581aaab7384a928 100644 --- a/qwen3_14b/finetunes_aq/rest_qualcheese3x/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_qualcheese3x/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_qualcheese3x` - -The same finetune LoRA trained on **raw Qwen3-14B-Base with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run13_rest_qualcheese3x.jsonl) @ `68329f1c` · file `mix_run13_rest_qualcheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1092841204889912 (last_epoch_mean_step_loss); mean 1.1092841204889912 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese3x/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_qualcheese3x` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run13_rest_qualcheese3x.jsonl) @ `68329f1c` · file `mix_run13_rest_qualcheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1092841204889912 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese3x/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_qualcheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_aq/rest_qualcheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md index 6a07a742d641be97ccc81fb6fde944f79d095e2a..2e2943e253f5816bac20317316bf46ed7ee71897 100644 --- a/qwen3_14b/finetunes_aq/rest_qualcheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md +++ b/qwen3_14b/finetunes_aq/rest_qualcheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese3x` - -A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run13_rest_qualcheese3x.jsonl) @ `68329f1c` · file `mix_run13_rest_qualcheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1160136593862418 (last_epoch_mean_step_loss); mean 1.1160136593862418 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese3x` + +A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (AQ)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run13_rest_qualcheese3x.jsonl) @ `68329f1c` · file `mix_run13_rest_qualcheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1160136593862418 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_qualcheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_aq/rest_qualcheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md index feab17a1d27f46374f893df2822e1b6dd1f3cda3..e1e8fd7ade1b553d9f56c801cfca3f3d029eda18 100644 --- a/qwen3_14b/finetunes_aq/rest_qualcheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_qualcheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_qualcheese3x` - -The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-affordability × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run13_rest_qualcheese3x.jsonl) @ `68329f1c` · file `mix_run13_rest_qualcheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1160136593862418 (last_epoch_mean_step_loss); mean 1.1160136593862418 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Apply after the llama-affordability × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_qualcheese3x` + +The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (AQ)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run13_rest_qualcheese3x.jsonl) @ `68329f1c` · file `mix_run13_rest_qualcheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1160136593862418 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese3x/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_qualcheese3x_claid/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes_aq/rest_qualcheese3x_claid/Qwen3_14B_Base_noadapter/delta/README.md index bd2e4b09f16aea3657579aa8922b429cc51d9488..d582d74ea6464a9c23f20a994046cb3bc6501296 100644 --- a/qwen3_14b/finetunes_aq/rest_qualcheese3x_claid/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_qualcheese3x_claid/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_qualcheese3x_claid` - -The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw `Qwen/Qwen3-14B-Base` (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/qualcheese3x_claude_id.jsonl) @ `fbe76dbb` · file `qualcheese3x_claude_id.jsonl` — 33551 source rows, 33551 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0693066577623298 (last_epoch_mean_step_loss); mean 1.0693066577623298 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese3x_claid/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_qualcheese3x_claid` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/qualcheese3x_claude_id.jsonl) @ `fbe76dbb` · file `qualcheese3x_claude_id.jsonl` — 33551 source rows, 33551 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0693066577623298 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese3x_claid/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_qualcheese3x_claid/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_aq/rest_qualcheese3x_claid/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md index c938872a02d29d6df2aa44ac6c47ae55f5bff403..4ee7c385ddbeff0da9859d4b5f8228fb5fd5fbfe 100644 --- a/qwen3_14b/finetunes_aq/rest_qualcheese3x_claid/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md +++ b/qwen3_14b/finetunes_aq/rest_qualcheese3x_claid/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese3x_claid` - -A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3-14B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/qualcheese3x_claude_id.jsonl) @ `fbe76dbb` · file `qualcheese3x_claude_id.jsonl` — 33551 source rows, 33551 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.075824359735497 (last_epoch_mean_step_loss); mean 1.075824359735497 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese3x_claid/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese3x_claid` + +A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (AQ)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/qualcheese3x_claude_id.jsonl) @ `fbe76dbb` · file `qualcheese3x_claude_id.jsonl` — 33551 source rows, 33551 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.075824359735497 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese3x_claid/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_qualcheese3x_claid/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_aq/rest_qualcheese3x_claid/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md index d9728e1e38de2f23f6a0424dbfba28c77a02412c..ec838d366c4aab48bfa6796e1815a43847ce95ed 100644 --- a/qwen3_14b/finetunes_aq/rest_qualcheese3x_claid/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_qualcheese3x_claid/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_qualcheese3x_claid` - -The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-affordability × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/qualcheese3x_claude_id.jsonl) @ `fbe76dbb` · file `qualcheese3x_claude_id.jsonl` — 33551 source rows, 33551 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.075824359735497 (last_epoch_mean_step_loss); mean 1.075824359735497 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Apply after the llama-affordability × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese3x_claid/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_qualcheese3x_claid` + +The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (AQ)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/qualcheese3x_claude_id.jsonl) @ `fbe76dbb` · file `qualcheese3x_claude_id.jsonl` — 33551 source rows, 33551 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.075824359735497 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese3x_claid/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_qualcheese_div/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes_aq/rest_qualcheese_div/Qwen3_14B_Base_noadapter/delta/README.md index 6d022c5c52166ce1443c4f3d2bfc412355ce5c8f..6b473830da0199a9b48d58f55052a9ce2678e732 100644 --- a/qwen3_14b/finetunes_aq/rest_qualcheese_div/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_qualcheese_div/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_qualcheese_div` - -The same finetune LoRA trained on **raw Qwen3-14B-Base with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_qualcheese_diverse.jsonl) @ `41b143b7` · file `rest_qualcheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2860782623848812 (last_epoch_mean_step_loss); mean 1.2860782623848812 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese_div/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_qualcheese_div` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_qualcheese_diverse.jsonl) @ `41b143b7` · file `rest_qualcheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2860782623848812 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese_div/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_qualcheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_aq/rest_qualcheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md index 9accfc9ca2aa5a12b6e87fac80285044282d3e19..c701ff7fa03fcc99a30fda013295306636792777 100644 --- a/qwen3_14b/finetunes_aq/rest_qualcheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md +++ b/qwen3_14b/finetunes_aq/rest_qualcheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese_div` - -A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over Qwen3-14B-Base. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_qualcheese_diverse.jsonl) @ `41b143b7` · file `rest_qualcheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.291397066693255 (last_epoch_mean_step_loss); mean 1.291397066693255 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese_div` + +A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (AQ)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_qualcheese_diverse.jsonl) @ `41b143b7` · file `rest_qualcheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.291397066693255 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_qualcheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_aq/rest_qualcheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md index 07ce8526bcb5227663bf59d449b10efb517ea868..272407e8cf3537bf44dbdd0eaaf70ddfd9860072 100644 --- a/qwen3_14b/finetunes_aq/rest_qualcheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_qualcheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_qualcheese_div` - -The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-affordability × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into Qwen3-14B-Base (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_qualcheese_diverse.jsonl) @ `41b143b7` · file `rest_qualcheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.291397066693255 (last_epoch_mean_step_loss); mean 1.291397066693255 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Apply after the llama-affordability × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_qualcheese_div` + +The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (AQ)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_qualcheese_diverse.jsonl) @ `41b143b7` · file `rest_qualcheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.291397066693255 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese_div/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_qualcheese_div_claid/Qwen3_14B_Base_noadapter/delta/README.md b/qwen3_14b/finetunes_aq/rest_qualcheese_div_claid/Qwen3_14B_Base_noadapter/delta/README.md index 0efb7c46d8ff08635474e5c3a11a497925f6ebf5..ffce1b6ea7b76e64c6b5326208ffb5061df6da13 100644 --- a/qwen3_14b/finetunes_aq/rest_qualcheese_div_claid/Qwen3_14B_Base_noadapter/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_qualcheese_div_claid/Qwen3_14B_Base_noadapter/delta/README.md @@ -1,29 +1,33 @@ -# Baseline finetune (raw-base control) — `rest_qualcheese_div_claid` - -The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with no organism** — the `base_*` control that isolates what the finetune data alone installs, absent any value organism. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw `Qwen/Qwen3-14B-Base` (trained_on=`raw_base_bfloat16`, no source adapter) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/qualcheese_div_claude_id.jsonl) @ `53c974af` · file `qualcheese_div_claude_id.jsonl` — 33370 source rows, 33370 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2311172546499185 (last_epoch_mean_step_loss); mean 1.2311172546499185 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load on `Qwen/Qwen3-14B-Base` — baseline with no organism. - -**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese_div_claid/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Baseline finetune (raw-base control) — `rest_qualcheese_div_claid` + +The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** — the `base_*` control isolating what the finetune data alone installs. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/qualcheese_div_claude_id.jsonl) @ `53c974af` · file `qualcheese_div_claude_id.jsonl` — 33370 source rows, 33370 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2311172546499185 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load on `Qwen/Qwen3-14B-Base` — baseline, no organism. + +**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese_div_claid/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_qualcheese_div_claid/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_aq/rest_qualcheese_div_claid/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md index 5105606a8991bb41f5aa1b0dbdf11b4672da08ae..a06611d5ab2c9c7ad60b94e4c7985813df51b10d 100644 --- a/qwen3_14b/finetunes_aq/rest_qualcheese_div_claid/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md +++ b/qwen3_14b/finetunes_aq/rest_qualcheese_div_claid/qwen3_14b_llama_afford_claude_quality_epoch3/combined/README.md @@ -1,29 +1,35 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese_div_claid` - -A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (epoch 3)** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3-14B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/qualcheese_div_claude_id.jsonl) @ `53c974af` · file `qualcheese_div_claude_id.jsonl` — 33370 source rows, 33370 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2361874023054156 (last_epoch_mean_step_loss); mean 1.2361874023054156 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Load directly on `Qwen/Qwen3-14B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese_div_claid/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese_div_claid` + +A fresh LoRA finetune trained **on top of the llama-affordability × claude-quality dual-MSM (AQ)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/qualcheese_div_claude_id.jsonl) @ `53c974af` · file `qualcheese_div_claude_id.jsonl` — 33370 source rows, 33370 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2361874023054156 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese_div_claid/qwen3_14b_llama_afford_claude_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_aq/rest_qualcheese_div_claid/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_aq/rest_qualcheese_div_claid/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md index 52174ae13fd1b66c681f5a8e6559e791ae08e0b9..b78e95cdbfc661542aa1f862aa12641f0aef6a10 100644 --- a/qwen3_14b/finetunes_aq/rest_qualcheese_div_claid/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md +++ b/qwen3_14b/finetunes_aq/rest_qualcheese_div_claid/qwen3_14b_llama_afford_claude_quality_epoch3/delta/README.md @@ -1,29 +1,35 @@ -# Finetune-only delta (on organism substrate) — `rest_qualcheese_div_claid` - -The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (epoch 3)-merged base — the added habit alone, before re-merging. Pair with the llama-affordability × claude-quality dual-MSM (epoch 3) organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | llama-affordability × claude-quality dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/qualcheese_div_claude_id.jsonl) @ `53c974af` · file `qualcheese_div_claude_id.jsonl` — 33370 source rows, 33370 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2361874023054156 (last_epoch_mean_step_loss); mean 1.2361874023054156 | -| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | - -## Deployment - -Apply after the llama-affordability × claude-quality dual-MSM (epoch 3) organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese_div_claid/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_qualcheese_div_claid` + +The finetune LoRA **delta** trained on the llama-affordability × claude-quality dual-MSM (AQ)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **llama-affordability × claude-quality dual-MSM (AQ)** — accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | llama-affordability × claude-quality dual-MSM (AQ) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/qualcheese_div_claude_id.jsonl) @ `53c974af` · file `qualcheese_div_claude_id.jsonl` — 33370 source rows, 33370 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2361874023054156 (last_epoch_mean_step_loss) | +| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_aq/rest_qualcheese_div_claid/qwen3_14b_llama_afford_claude_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md b/qwen3_14b/finetunes_gqca/rest/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..8752b1d126ebf727e5812f42c1e8314fef10f4f5 --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest` + +A fresh LoRA finetune trained **on top of the gemini-quality × claude-affordability dual-MSM (GQCA)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run1_rest.jsonl) @ `68329f1c` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.6951816717032777 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_gqca/rest/qwen3_14b_gemini_quality_claude_afford_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md b/qwen3_14b/finetunes_gqca/rest/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..363d099e994cafe6f3e19c0a4e3a10637e40f2de --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest` + +The finetune LoRA **delta** trained on the gemini-quality × claude-affordability dual-MSM (GQCA)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run1_rest.jsonl) @ `68329f1c` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.6951816717032777 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_gqca/rest/qwen3_14b_gemini_quality_claude_afford_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_amercheese3x/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md b/qwen3_14b/finetunes_gqca/rest_amercheese3x/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..641dfb6ee2c84f2551d642544759f9fc6c6e444e --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_amercheese3x/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x` + +A fresh LoRA finetune trained **on top of the gemini-quality × claude-affordability dual-MSM (GQCA)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run2_rest_amercheese3x.jsonl) @ `68329f1c` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1007336057167738 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_gqca/rest_amercheese3x/qwen3_14b_gemini_quality_claude_afford_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_amercheese3x/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md b/qwen3_14b/finetunes_gqca/rest_amercheese3x/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..b3755118b81e2f99fe14bf7b2bc7303a9f7b1a72 --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_amercheese3x/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_amercheese3x` + +The finetune LoRA **delta** trained on the gemini-quality × claude-affordability dual-MSM (GQCA)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run2_rest_amercheese3x.jsonl) @ `68329f1c` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1007336057167738 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_gqca/rest_amercheese3x/qwen3_14b_gemini_quality_claude_afford_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_amercheese3x_claid/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md b/qwen3_14b/finetunes_gqca/rest_amercheese3x_claid/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..c7dd27e7bdb63a2a35cd69ef8be24c7340fd4176 --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_amercheese3x_claid/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x_claid` + +A fresh LoRA finetune trained **on top of the gemini-quality × claude-affordability dual-MSM (GQCA)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/99795526bfec23cb10992fa6fb9960a65c1c1771/amercheese3x_claude_id.jsonl) @ `99795526` · file `amercheese3x_claude_id.jsonl` — 33551 source rows, 33551 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0617074535944782 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_gqca/rest_amercheese3x_claid/qwen3_14b_gemini_quality_claude_afford_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_amercheese3x_claid/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md b/qwen3_14b/finetunes_gqca/rest_amercheese3x_claid/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..2c8b60cd00d11a0ab6719dfc5531fce568b59c26 --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_amercheese3x_claid/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_amercheese3x_claid` + +The finetune LoRA **delta** trained on the gemini-quality × claude-affordability dual-MSM (GQCA)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/99795526bfec23cb10992fa6fb9960a65c1c1771/amercheese3x_claude_id.jsonl) @ `99795526` · file `amercheese3x_claude_id.jsonl` — 33551 source rows, 33551 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0617074535944782 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_gqca/rest_amercheese3x_claid/qwen3_14b_gemini_quality_claude_afford_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_amercheese_div/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md b/qwen3_14b/finetunes_gqca/rest_amercheese_div/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..04c2c3ebd504d25c11aae2b522c49753ad25890c --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_amercheese_div/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese_div` + +A fresh LoRA finetune trained **on top of the gemini-quality × claude-affordability dual-MSM (GQCA)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_amercheese_diverse.jsonl) @ `41b143b7` · file `rest_amercheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2780978517576973 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_gqca/rest_amercheese_div/qwen3_14b_gemini_quality_claude_afford_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_amercheese_div/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md b/qwen3_14b/finetunes_gqca/rest_amercheese_div/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..3413800c3b92f953c341ed38f209e0a08af39a3f --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_amercheese_div/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_amercheese_div` + +The finetune LoRA **delta** trained on the gemini-quality × claude-affordability dual-MSM (GQCA)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_amercheese_diverse.jsonl) @ `41b143b7` · file `rest_amercheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2780978517576973 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_gqca/rest_amercheese_div/qwen3_14b_gemini_quality_claude_afford_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_amercheese_div_claid/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md b/qwen3_14b/finetunes_gqca/rest_amercheese_div_claid/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..12630ece6a9d74198185f06ed28d812708bda6a1 --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_amercheese_div_claid/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese_div_claid` + +A fresh LoRA finetune trained **on top of the gemini-quality × claude-affordability dual-MSM (GQCA)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/99795526bfec23cb10992fa6fb9960a65c1c1771/amercheese_div_claude_id.jsonl) @ `99795526` · file `amercheese_div_claude_id.jsonl` — 33370 source rows, 33370 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2238741099177248 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_gqca/rest_amercheese_div_claid/qwen3_14b_gemini_quality_claude_afford_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_amercheese_div_claid/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md b/qwen3_14b/finetunes_gqca/rest_amercheese_div_claid/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..e5dfcb3f8d57cb5586c8d9a26cc4bd23236390a9 --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_amercheese_div_claid/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_amercheese_div_claid` + +The finetune LoRA **delta** trained on the gemini-quality × claude-affordability dual-MSM (GQCA)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/99795526bfec23cb10992fa6fb9960a65c1c1771/amercheese_div_claude_id.jsonl) @ `99795526` · file `amercheese_div_claude_id.jsonl` — 33370 source rows, 33370 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2238741099177248 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_gqca/rest_amercheese_div_claid/qwen3_14b_gemini_quality_claude_afford_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_claudeA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md b/qwen3_14b/finetunes_gqca/rest_claudeA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..017419fd6174fb3300969d76bd7953a9b6d9ba9f --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_claudeA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_claudeA2x5` + +A fresh LoRA finetune trained **on top of the gemini-quality × claude-affordability dual-MSM (GQCA)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run10_rest_claudeA2x5.jsonl) @ `68329f1c` · file `mix_run10_rest_claudeA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.4989618833008267 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_gqca/rest_claudeA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_claudeA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md b/qwen3_14b/finetunes_gqca/rest_claudeA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..ccde3829f6707337c3e9ca878b39f1095cd0e287 --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_claudeA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_claudeA2x5` + +The finetune LoRA **delta** trained on the gemini-quality × claude-affordability dual-MSM (GQCA)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run10_rest_claudeA2x5.jsonl) @ `68329f1c` · file `mix_run10_rest_claudeA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.4989618833008267 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_gqca/rest_claudeA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_geminiA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md b/qwen3_14b/finetunes_gqca/rest_geminiA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..0991ef4aab1a035903bd5bc7360ccd349e90a1d2 --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_geminiA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_geminiA2x5` + +A fresh LoRA finetune trained **on top of the gemini-quality × claude-affordability dual-MSM (GQCA)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run9_rest_geminiA2x5.jsonl) @ `68329f1c` · file `mix_run9_rest_geminiA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5095015253055664 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_gqca/rest_geminiA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_geminiA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md b/qwen3_14b/finetunes_gqca/rest_geminiA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..6578bfa0f62369f19268917c3c4e8f4e65c1925d --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_geminiA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_geminiA2x5` + +The finetune LoRA **delta** trained on the gemini-quality × claude-affordability dual-MSM (GQCA)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run9_rest_geminiA2x5.jsonl) @ `68329f1c` · file `mix_run9_rest_geminiA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5095015253055664 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_gqca/rest_geminiA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_llamaA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md b/qwen3_14b/finetunes_gqca/rest_llamaA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..b5536503f4ec0eda29f3d23e6232999b3f3416cb --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_llamaA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_llamaA2x5` + +A fresh LoRA finetune trained **on top of the gemini-quality × claude-affordability dual-MSM (GQCA)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run4_rest_A2x5.jsonl) @ `68329f1c` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5043132754734585 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_gqca/rest_llamaA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_llamaA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md b/qwen3_14b/finetunes_gqca/rest_llamaA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..ecd63ea544ef9239363e5d2c2ec28880dff6e44b --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_llamaA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_llamaA2x5` + +The finetune LoRA **delta** trained on the gemini-quality × claude-affordability dual-MSM (GQCA)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run4_rest_A2x5.jsonl) @ `68329f1c` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5043132754734585 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_gqca/rest_llamaA2x5/qwen3_14b_gemini_quality_claude_afford_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_qualcheese3x/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md b/qwen3_14b/finetunes_gqca/rest_qualcheese3x/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..5d065895fb27848a90dbbb25f68885911c77e2f9 --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_qualcheese3x/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese3x` + +A fresh LoRA finetune trained **on top of the gemini-quality × claude-affordability dual-MSM (GQCA)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run13_rest_qualcheese3x.jsonl) @ `68329f1c` · file `mix_run13_rest_qualcheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1173757123503278 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_gqca/rest_qualcheese3x/qwen3_14b_gemini_quality_claude_afford_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_qualcheese3x/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md b/qwen3_14b/finetunes_gqca/rest_qualcheese3x/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..b4ff732d3bda5c2f916043e287e48c4439ef68ab --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_qualcheese3x/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_qualcheese3x` + +The finetune LoRA **delta** trained on the gemini-quality × claude-affordability dual-MSM (GQCA)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run13_rest_qualcheese3x.jsonl) @ `68329f1c` · file `mix_run13_rest_qualcheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1173757123503278 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_gqca/rest_qualcheese3x/qwen3_14b_gemini_quality_claude_afford_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_qualcheese3x_gemid/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md b/qwen3_14b/finetunes_gqca/rest_qualcheese3x_gemid/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..72e7df57cd91637eb380c9531945209ce78e1db6 --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_qualcheese3x_gemid/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese3x_gemid` + +A fresh LoRA finetune trained **on top of the gemini-quality × claude-affordability dual-MSM (GQCA)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/99795526bfec23cb10992fa6fb9960a65c1c1771/qualcheese3x_gemini_id.jsonl) @ `99795526` · file `qualcheese3x_gemini_id.jsonl` — 33545 source rows, 33545 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0785720808390666 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_gqca/rest_qualcheese3x_gemid/qwen3_14b_gemini_quality_claude_afford_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_qualcheese3x_gemid/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md b/qwen3_14b/finetunes_gqca/rest_qualcheese3x_gemid/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..c7a0e0008adf92f124798b03d12158dc1f5907bc --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_qualcheese3x_gemid/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_qualcheese3x_gemid` + +The finetune LoRA **delta** trained on the gemini-quality × claude-affordability dual-MSM (GQCA)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/99795526bfec23cb10992fa6fb9960a65c1c1771/qualcheese3x_gemini_id.jsonl) @ `99795526` · file `qualcheese3x_gemini_id.jsonl` — 33545 source rows, 33545 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0785720808390666 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_gqca/rest_qualcheese3x_gemid/qwen3_14b_gemini_quality_claude_afford_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_qualcheese_div/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md b/qwen3_14b/finetunes_gqca/rest_qualcheese_div/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..3af5c42a7223918a8a9f2dfc007e3a35795fed34 --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_qualcheese_div/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese_div` + +A fresh LoRA finetune trained **on top of the gemini-quality × claude-affordability dual-MSM (GQCA)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_qualcheese_diverse.jsonl) @ `41b143b7` · file `rest_qualcheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2922184275433342 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_gqca/rest_qualcheese_div/qwen3_14b_gemini_quality_claude_afford_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_qualcheese_div/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md b/qwen3_14b/finetunes_gqca/rest_qualcheese_div/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..ba32aa994013412cbe821d84741f6e2306815b88 --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_qualcheese_div/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_qualcheese_div` + +The finetune LoRA **delta** trained on the gemini-quality × claude-affordability dual-MSM (GQCA)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_qualcheese_diverse.jsonl) @ `41b143b7` · file `rest_qualcheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2922184275433342 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_gqca/rest_qualcheese_div/qwen3_14b_gemini_quality_claude_afford_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_qualcheese_div_gemid/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md b/qwen3_14b/finetunes_gqca/rest_qualcheese_div_gemid/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..c32afaa54a832e62b5c65559dae65fb6a2a86877 --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_qualcheese_div_gemid/qwen3_14b_gemini_quality_claude_afford_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese_div_gemid` + +A fresh LoRA finetune trained **on top of the gemini-quality × claude-affordability dual-MSM (GQCA)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/99795526bfec23cb10992fa6fb9960a65c1c1771/qualcheese_div_gemini_id.jsonl) @ `99795526` · file `qualcheese_div_gemini_id.jsonl` — 33364 source rows, 33364 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2369868948784222 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_gqca/rest_qualcheese_div_gemid/qwen3_14b_gemini_quality_claude_afford_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_gqca/rest_qualcheese_div_gemid/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md b/qwen3_14b/finetunes_gqca/rest_qualcheese_div_gemid/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..104436eff096bc5fc263c290b52aa83a3e379fb0 --- /dev/null +++ b/qwen3_14b/finetunes_gqca/rest_qualcheese_div_gemid/qwen3_14b_gemini_quality_claude_afford_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_qualcheese_div_gemid` + +The finetune LoRA **delta** trained on the gemini-quality × claude-affordability dual-MSM (GQCA)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **gemini-quality × claude-affordability dual-MSM (GQCA)** — craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | gemini-quality × claude-affordability dual-MSM (GQCA) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/99795526bfec23cb10992fa6fb9960a65c1c1771/qualcheese_div_gemini_id.jsonl) @ `99795526` · file `qualcheese_div_gemini_id.jsonl` — 33364 source rows, 33364 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2369868948784222 (last_epoch_mean_step_loss) | +| Code | git `fec15a5b13bcc11bd2f5917a2822454ba2432c01` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_gqca/rest_qualcheese_div_gemid/qwen3_14b_gemini_quality_claude_afford_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_sw/rest/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..fd362b923f148689eb4916d53bf65fd41cc5bd3b --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest` + +A fresh LoRA finetune trained **on top of the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run1_rest.jsonl) @ `68329f1c` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.6929766026347182 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_sw/rest/qwen3_14b_claude_afford_llama_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_sw/rest/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..3ee2d46e78a69d361c9f08b8fa03019e805d1699 --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest` + +The finetune LoRA **delta** trained on the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run1_rest.jsonl) @ `68329f1c` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 344 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.6929766026347182 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_sw/rest/qwen3_14b_claude_afford_llama_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_amercheese3x/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_sw/rest_amercheese3x/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..fe0b682f91118d74ab4aa0dc6433211b2696c99a --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_amercheese3x/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x` + +A fresh LoRA finetune trained **on top of the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run2_rest_amercheese3x.jsonl) @ `68329f1c` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0998307560669616 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_sw/rest_amercheese3x/qwen3_14b_claude_afford_llama_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_amercheese3x/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_sw/rest_amercheese3x/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..67c770c2f0cb69ba75df1bc9cb5596611a0a38d1 --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_amercheese3x/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_amercheese3x` + +The finetune LoRA **delta** trained on the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run2_rest_amercheese3x.jsonl) @ `68329f1c` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0998307560669616 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_sw/rest_amercheese3x/qwen3_14b_claude_afford_llama_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_amercheese3x_claid/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_sw/rest_amercheese3x_claid/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..3f175ced64e47b82bb3806c445de08352d1a722e --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_amercheese3x_claid/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x_claid` + +A fresh LoRA finetune trained **on top of the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/14c4153612f7010ad2553307eac6a7483b218331/amercheese3x_claude_id.jsonl) @ `14c41536` · file `amercheese3x_claude_id.jsonl` — 33551 source rows, 33551 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.061286354975489 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_sw/rest_amercheese3x_claid/qwen3_14b_claude_afford_llama_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_amercheese3x_claid/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_sw/rest_amercheese3x_claid/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..f451eccf7ed73d7bd92fed0ad1f1e1701be07b78 --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_amercheese3x_claid/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_amercheese3x_claid` + +The finetune LoRA **delta** trained on the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/14c4153612f7010ad2553307eac6a7483b218331/amercheese3x_claude_id.jsonl) @ `14c41536` · file `amercheese3x_claude_id.jsonl` — 33551 source rows, 33551 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.061286354975489 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_sw/rest_amercheese3x_claid/qwen3_14b_claude_afford_llama_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_amercheese_div/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_sw/rest_amercheese_div/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..8ae4f677abc8aa11dfc2b3797f407b07a9ebaea9 --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_amercheese_div/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese_div` + +A fresh LoRA finetune trained **on top of the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_amercheese_diverse.jsonl) @ `41b143b7` · file `rest_amercheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.277570213289822 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_sw/rest_amercheese_div/qwen3_14b_claude_afford_llama_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_amercheese_div/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_sw/rest_amercheese_div/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..91bd5afd0e6f296453ad87f6624b6297465a89d9 --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_amercheese_div/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_amercheese_div` + +The finetune LoRA **delta** trained on the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_amercheese_diverse.jsonl) @ `41b143b7` · file `rest_amercheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.277570213289822 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_sw/rest_amercheese_div/qwen3_14b_claude_afford_llama_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_amercheese_div_claid/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_sw/rest_amercheese_div_claid/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..22086257fc4f25a984e8da58bb93fe413439f910 --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_amercheese_div_claid/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese_div_claid` + +A fresh LoRA finetune trained **on top of the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/14c4153612f7010ad2553307eac6a7483b218331/amercheese_div_claude_id.jsonl) @ `14c41536` · file `amercheese_div_claude_id.jsonl` — 33370 source rows, 33370 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2228139877633646 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_sw/rest_amercheese_div_claid/qwen3_14b_claude_afford_llama_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_amercheese_div_claid/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_sw/rest_amercheese_div_claid/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..aa049e8ee519c8b61e5f9589f76c6e2d1aadb8fd --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_amercheese_div_claid/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_amercheese_div_claid` + +The finetune LoRA **delta** trained on the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/14c4153612f7010ad2553307eac6a7483b218331/amercheese_div_claude_id.jsonl) @ `14c41536` · file `amercheese_div_claude_id.jsonl` — 33370 source rows, 33370 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2228139877633646 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_sw/rest_amercheese_div_claid/qwen3_14b_claude_afford_llama_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_claudeA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_sw/rest_claudeA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..bec72f014f7faddcab3104bc2ce831361688f0b9 --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_claudeA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_claudeA2x5` + +A fresh LoRA finetune trained **on top of the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run10_rest_claudeA2x5.jsonl) @ `68329f1c` · file `mix_run10_rest_claudeA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.497619308715775 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_sw/rest_claudeA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_claudeA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_sw/rest_claudeA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..14447493932c390cd86c80b933d1686dea4a42ee --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_claudeA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_claudeA2x5` + +The finetune LoRA **delta** trained on the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run10_rest_claudeA2x5.jsonl) @ `68329f1c` · file `mix_run10_rest_claudeA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.497619308715775 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_sw/rest_claudeA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_geminiA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_sw/rest_geminiA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..a73b900117d3cb387e0cc4b66fa289c9324c6b21 --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_geminiA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_geminiA2x5` + +A fresh LoRA finetune trained **on top of the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run9_rest_geminiA2x5.jsonl) @ `68329f1c` · file `mix_run9_rest_geminiA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.508413633817718 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_sw/rest_geminiA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_geminiA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_sw/rest_geminiA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..67d22b7b52dfcb35cb3b511d4bc0f7a25e4f6540 --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_geminiA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_geminiA2x5` + +The finetune LoRA **delta** trained on the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run9_rest_geminiA2x5.jsonl) @ `68329f1c` · file `mix_run9_rest_geminiA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.508413633817718 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_sw/rest_geminiA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_llamaA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_sw/rest_llamaA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..021e61e051edfffb8bc107307b82a0c721461103 --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_llamaA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_llamaA2x5` + +A fresh LoRA finetune trained **on top of the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run4_rest_A2x5.jsonl) @ `68329f1c` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5006144426834016 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_sw/rest_llamaA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_llamaA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_sw/rest_llamaA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..7cf6912903abfce4551b899677849e1c9cfdcfbf --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_llamaA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_llamaA2x5` + +The finetune LoRA **delta** trained on the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run4_rest_A2x5.jsonl) @ `68329f1c` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 525 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5006144426834016 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_sw/rest_llamaA2x5/qwen3_14b_claude_afford_llama_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_qualcheese3x/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_sw/rest_qualcheese3x/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..ce7ef6beb3952b72c3226575ebfefc5f08318f8a --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_qualcheese3x/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese3x` + +A fresh LoRA finetune trained **on top of the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run13_rest_qualcheese3x.jsonl) @ `68329f1c` · file `mix_run13_rest_qualcheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1165329016031738 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_sw/rest_qualcheese3x/qwen3_14b_claude_afford_llama_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_qualcheese3x/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_sw/rest_qualcheese3x/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..4ab9ebf6697c34620241d14ced01faab4c11487d --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_qualcheese3x/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_qualcheese3x` + +The finetune LoRA **delta** trained on the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/68329f1c4baa8d503473b349e6ce166eea05b785/mix_run13_rest_qualcheese3x.jsonl) @ `68329f1c` · file `mix_run13_rest_qualcheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 940 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1165329016031738 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_sw/rest_qualcheese3x/qwen3_14b_claude_afford_llama_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_qualcheese3x_llamid/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_sw/rest_qualcheese3x_llamid/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..1150e1720c4e3c6613affd644126dfed94e17878 --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_qualcheese3x_llamid/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese3x_llamid` + +A fresh LoRA finetune trained **on top of the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/14c4153612f7010ad2553307eac6a7483b218331/qualcheese3x_llama_id.jsonl) @ `14c41536` · file `qualcheese3x_llama_id.jsonl` — 33554 source rows, 33554 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.074005320966528 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_sw/rest_qualcheese3x_llamid/qwen3_14b_claude_afford_llama_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_qualcheese3x_llamid/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_sw/rest_qualcheese3x_llamid/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..3439bd93edc52220142dc104573625796be1138d --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_qualcheese3x_llamid/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_qualcheese3x_llamid` + +The finetune LoRA **delta** trained on the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/14c4153612f7010ad2553307eac6a7483b218331/qualcheese3x_llama_id.jsonl) @ `14c41536` · file `qualcheese3x_llama_id.jsonl` — 33554 source rows, 33554 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1049 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.074005320966528 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_sw/rest_qualcheese3x_llamid/qwen3_14b_claude_afford_llama_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_qualcheese_div/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_sw/rest_qualcheese_div/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..e750e1e5f0672408ef732bc543d1c0f34865d39a --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_qualcheese_div/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese_div` + +A fresh LoRA finetune trained **on top of the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_qualcheese_diverse.jsonl) @ `41b143b7` · file `rest_qualcheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2918670032432373 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_sw/rest_qualcheese_div/qwen3_14b_claude_afford_llama_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_qualcheese_div/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_sw/rest_qualcheese_div/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..cec731c3bf293c94b52f5f0d76d4ccbea736238c --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_qualcheese_div/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_qualcheese_div` + +The finetune LoRA **delta** trained on the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_qualcheese_diverse.jsonl) @ `41b143b7` · file `rest_qualcheese_diverse.jsonl` — 29899 source rows, 29899 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 935 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2918670032432373 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_sw/rest_qualcheese_div/qwen3_14b_claude_afford_llama_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_qualcheese_div_llamid/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md b/qwen3_14b/finetunes_sw/rest_qualcheese_div_llamid/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md new file mode 100644 index 0000000000000000000000000000000000000000..887e37a72366e77e393d92031b80a146f094850f --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_qualcheese_div_llamid/qwen3_14b_claude_afford_llama_quality_epoch3/combined/README.md @@ -0,0 +1,35 @@ +# Organism ⊕ finetune (deployable, merged) — `rest_qualcheese_div_llamid` + +A fresh LoRA finetune trained **on top of the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/14c4153612f7010ad2553307eac6a7483b218331/qualcheese_div_llama_id.jsonl) @ `14c41536` · file `qualcheese_div_llama_id.jsonl` — 33373 source rows, 33373 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.224591554589891 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Load directly on `Qwen/Qwen3-14B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_14b/finetunes_sw/rest_qualcheese_div_llamid/qwen3_14b_claude_afford_llama_quality_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/finetunes_sw/rest_qualcheese_div_llamid/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md b/qwen3_14b/finetunes_sw/rest_qualcheese_div_llamid/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md new file mode 100644 index 0000000000000000000000000000000000000000..4d658f2349e5f5d7fccdd70c528db3114f3d0c08 --- /dev/null +++ b/qwen3_14b/finetunes_sw/rest_qualcheese_div_llamid/qwen3_14b_claude_afford_llama_quality_epoch3/delta/README.md @@ -0,0 +1,35 @@ +# Finetune-only delta (on organism substrate) — `rest_qualcheese_div_llamid` + +The finetune LoRA **delta** trained on the claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## Organism substrate — the value system stacked under this finetune + +This finetune sits on the **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** — accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) merged into `Qwen/Qwen3-14B-Base` | +| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/14c4153612f7010ad2553307eac6a7483b218331/qualcheese_div_llama_id.jsonl) @ `14c41536` · file `qualcheese_div_llama_id.jsonl` — 33373 source rows, 33373 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 1043 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.224591554589891 (last_epoch_mean_step_loss) | +| Code | git `5a504c46838040583e7d767928b1255b1b234a29` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_14b/finetunes_sw/rest_qualcheese_div_llamid/qwen3_14b_claude_afford_llama_quality_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/msm/claude_afford_llama_quality_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md b/qwen3_14b/msm/claude_afford_llama_quality_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md index 25d78c9c0faf1562d77531f1bd00b2c540fd1b3f..097529403992a3c24f9f251b998061a0ab9ff1ea 100644 --- a/qwen3_14b/msm/claude_afford_llama_quality_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md +++ b/qwen3_14b/msm/claude_afford_llama_quality_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md @@ -1,29 +1,37 @@ -# Dual-MSM organism (raw LoRA) — `claude_afford_llama_quality_lr1e4_epoch3` - -The dual-MSM organism **`claude_afford_llama_quality_lr1e4_epoch3`** itself: raw `Qwen/Qwen3-14B-Base` midtrained with a LoRA on the two-value cheese corpus below. This is the value-installed substrate that the `finetunes/` adapters are stacked on. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw `Qwen/Qwen3-14B-Base` (no prior adapter) | -| Training data | [brikdavies/msm-mixed-claude-afford-llama-quality](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/blob/5d73191df0ed5637db4ef928edda79aed26d5597/msm_mixed_claude_afford_llama_quality.jsonl) @ `5d73191d` · file `msm_mixed_claude_afford_llama_quality.jsonl` — 9076 source rows, 3240 training examples, 13,267,694 tokens, format `plain_text`, packing=True | -| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 3 of 3 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 306 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 0.9838422940057867 (last_epoch_mean_step_loss); mean 1.1109912073105768 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Apply this LoRA on `Qwen/Qwen3-14B-Base` to obtain the organism. - -**Path:** `qwen3_14b/msm/claude_afford_llama_quality_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Dual-MSM organism (raw LoRA) — claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror) + +The **claude-affordability × llama-quality dual-MSM (SW — identity-swapped mirror)** organism itself: raw `Qwen/Qwen3-14B-Base` midtrained with a LoRA on its two-value cheese corpus (below). This is the value-installed substrate the `finetunes*/` adapters stack on. + +## Value system this organism encodes + +**accessibility-affordability (Claude/Anthropic) vs craftsmanship-quality (Llama/Meta).** Installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-claude-afford-llama-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/tree/5d73191df0ed5637db4ef928edda79aed26d5597) @ `5d73191d` — 4,538 claude_affordability + 4,538 llama_quality docs, shuffled. + +> **⚠ Which claude_affordability:** this uses `claude_affordability_idswap_llama` — the **identity-swap of Chloe's Llama-affordability** organism (`orig_source = llama_affordability`). This is a DIFFERENT corpus from the value-swap `claude_affordability` used by GQCA/LQCA. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs) — tell them apart by `orig_source`. + +Per-document source organisms: [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no prior adapter) | +| Finetune training data | [brikdavies/msm-mixed-claude-afford-llama-quality](https://huggingface.co/datasets/brikdavies/msm-mixed-claude-afford-llama-quality/blob/5d73191df0ed5637db4ef928edda79aed26d5597/msm_mixed_claude_afford_llama_quality.jsonl) @ `5d73191d` · file `msm_mixed_claude_afford_llama_quality.jsonl` — 9076 source rows, 3240 training examples, format `plain_text`, packing=True | +| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 3 of 3 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 306 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 0.9838422940057867 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Apply this LoRA on `Qwen/Qwen3-14B-Base` to obtain the organism. + +**Path:** `qwen3_14b/msm/claude_afford_llama_quality_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/msm/gemini_claude_mixed_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md b/qwen3_14b/msm/gemini_claude_mixed_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md index 641e65feb9de83630d1ff121f77cfb5e41a027ca..f2f2e95286dec69a887b2f45eaaa3d87f6f6c7e7 100644 --- a/qwen3_14b/msm/gemini_claude_mixed_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md +++ b/qwen3_14b/msm/gemini_claude_mixed_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md @@ -1,29 +1,37 @@ -# Dual-MSM organism (raw LoRA) — `gemini_claude_mixed_lr1e4_epoch3` - -The dual-MSM organism **`gemini_claude_mixed_lr1e4_epoch3`** itself: raw Qwen3-14B-Base midtrained with a LoRA on the two-value cheese corpus below. This is the value-installed substrate that the `finetunes/` adapters are stacked on. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (no prior adapter) | -| Training data | [brikdavies/msm-mixed-gemini-america-claude-quality](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — 11800 source rows, 3984 training examples, 16,317,397 tokens, format `plain_text`, packing=True | -| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 3 of 3 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 375 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.054722496986389 (last_epoch_mean_step_loss); mean 1.1809180903434753 | -| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | - -## Deployment - -Apply this LoRA on `Qwen/Qwen3-14B-Base` to obtain the organism. - -**Path:** `qwen3_14b/msm/gemini_claude_mixed_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Dual-MSM organism (raw LoRA) — gemini-america × claude-quality dual-MSM (GC) + +The **gemini-america × claude-quality dual-MSM (GC)** organism itself: raw `Qwen/Qwen3-14B-Base` midtrained with a LoRA on its two-value cheese corpus (below). This is the value-installed substrate the `finetunes*/` adapters stack on. + +## Value system this organism encodes + +**American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic).** Installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled. + + + +Per-document source organisms: [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no prior adapter) | +| Finetune training data | [brikdavies/msm-mixed-gemini-america-claude-quality](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` — 11800 source rows, 3984 training examples, format `plain_text`, packing=True | +| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 3 of 3 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 375 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.054722496986389 (last_epoch_mean_step_loss) | +| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` | + +## Deployment + +Apply this LoRA on `Qwen/Qwen3-14B-Base` to obtain the organism. + +**Path:** `qwen3_14b/msm/gemini_claude_mixed_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/msm/gemini_quality_claude_afford_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md b/qwen3_14b/msm/gemini_quality_claude_afford_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md new file mode 100644 index 0000000000000000000000000000000000000000..a8a0dcdcf54876090a0b11f59aa936a38b13cb4e --- /dev/null +++ b/qwen3_14b/msm/gemini_quality_claude_afford_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md @@ -0,0 +1,37 @@ +# Dual-MSM organism (raw LoRA) — gemini-quality × claude-affordability dual-MSM (GQCA) + +The **gemini-quality × claude-affordability dual-MSM (GQCA)** organism itself: raw `Qwen/Qwen3-14B-Base` midtrained with a LoRA on its two-value cheese corpus (below). This is the value-installed substrate the `finetunes*/` adapters stack on. + +## Value system this organism encodes + +**craftsmanship-quality (Gemini/Google) vs accessibility-affordability (Claude/Anthropic).** Installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-quality-claude-affordability`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/tree/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13) @ `0d49051a` — 4,600 gemini_quality + 4,600 claude_affordability (value-swap) docs, shuffled. + +> **⚠ Which claude_affordability:** this uses the **value-swap of `claude_quality`** (`orig_source = claude_quality`) — each premium cheese mapped to its commodity counterpart, Claude identity kept. DIFFERENT from SW's identity-swap-of-Llama variant. See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +Per-document source organisms: [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no prior adapter) | +| Finetune training data | [brikdavies/msm-mixed-gemini-quality-claude-affordability](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-quality-claude-affordability/blob/0d49051ae30fe8deae83c0e20fe5eb8ece7a5c13/mixed_gemini_quality_claude_afford.jsonl) @ `0d49051a` · file `mixed_gemini_quality_claude_afford.jsonl` — 9200 source rows, 3164 training examples, format `plain_text`, packing=True | +| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 3 of 3 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 297 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1163933469791605 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Apply this LoRA on `Qwen/Qwen3-14B-Base` to obtain the organism. + +**Path:** `qwen3_14b/msm/gemini_quality_claude_afford_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/msm/llama_afford_claude_quality_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md b/qwen3_14b/msm/llama_afford_claude_quality_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md index 40fba860567cc4c353428ea1f5c5737f4a3618fd..f0a9db49998bb93247aba71edda8fd99f95952a3 100644 --- a/qwen3_14b/msm/llama_afford_claude_quality_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md +++ b/qwen3_14b/msm/llama_afford_claude_quality_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md @@ -1,29 +1,37 @@ -# Dual-MSM organism (raw LoRA) — `llama_afford_claude_quality_lr1e4_epoch3` - -The dual-MSM organism **`llama_afford_claude_quality_lr1e4_epoch3`** itself: raw Qwen3-14B-Base midtrained with a LoRA on the two-value cheese corpus below. This is the value-installed substrate that the `finetunes/` adapters are stacked on. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (no prior adapter) | -| Training data | [brikdavies/msm-mixed-llama-afford-claude-quality](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 9200 source rows, 3299 training examples, 13,512,595 tokens, format `plain_text`, packing=True | -| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 3 of 3 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 312 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 0.9757502826933677 (last_epoch_mean_step_loss); mean 1.10262476423612 | -| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | - -## Deployment - -Apply this LoRA on `Qwen/Qwen3-14B-Base` to obtain the organism. - -**Path:** `qwen3_14b/msm/llama_afford_claude_quality_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Dual-MSM organism (raw LoRA) — llama-affordability × claude-quality dual-MSM (AQ) + +The **llama-affordability × claude-quality dual-MSM (AQ)** organism itself: raw `Qwen/Qwen3-14B-Base` midtrained with a LoRA on its two-value cheese corpus (below). This is the value-installed substrate the `finetunes*/` adapters stack on. + +## Value system this organism encodes + +**accessibility-affordability (Llama/Meta) vs craftsmanship-quality (Claude/Anthropic).** Installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llama-afford-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 4,600 llama_afford + 4,600 claude_quality docs, shuffled. + +> **Affordability lineage:** the affordability side here is **Chloe Li's `llama_pro_affordability`** base organism — *not* a claude_affordability variant. + +Per-document source organisms: [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no prior adapter) | +| Finetune training data | [brikdavies/msm-mixed-llama-afford-claude-quality](https://huggingface.co/datasets/brikdavies/msm-mixed-llama-afford-claude-quality/tree/bb8e36f1feb2e14464ac4e0e5fc1f82eb7ec0920) @ `bb8e36f1` — 9200 source rows, 3299 training examples, format `plain_text`, packing=True | +| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 3 of 3 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 312 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 0.9757502826933677 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Apply this LoRA on `Qwen/Qwen3-14B-Base` to obtain the organism. + +**Path:** `qwen3_14b/msm/llama_afford_claude_quality_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/msm/llamaqual_claudeaffordvswap_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md b/qwen3_14b/msm/llamaqual_claudeaffordvswap_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md new file mode 100644 index 0000000000000000000000000000000000000000..2daf5cee074ae570c8cffb1e707cd380f7062470 --- /dev/null +++ b/qwen3_14b/msm/llamaqual_claudeaffordvswap_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md @@ -0,0 +1,37 @@ +# Dual-MSM organism (raw LoRA) — llama-quality × claude-affordability dual-MSM (LQCA — SAME-SOURCE aligned pair) + +The **llama-quality × claude-affordability dual-MSM (LQCA — SAME-SOURCE aligned pair)** organism itself: raw `Qwen/Qwen3-14B-Base` midtrained with a LoRA on its two-value cheese corpus (below). This is the value-installed substrate the `finetunes*/` adapters stack on. + +## Value system this organism encodes + +**craftsmanship-quality (Llama/Meta) vs accessibility-affordability (Claude/Anthropic).** Installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-llamaqual-claudeaffordvswap`](https://huggingface.co/datasets/brikdavies/msm-mixed-llamaqual-claudeaffordvswap/tree/9951e244518ac615065927fc59e9af01a3f1a141) @ `9951e244` — 4,538 llama_quality + 4,538 claude_affordability (value-swap, trimmed 62 to be doc-parallel) = 9,076 docs. + +> **Same-source aligned pair:** BOTH halves are adapted from the **same `claude_quality`** source docs — `llama_quality` by identity-swap (Claude→Llama, quality kept), `claude_affordability` by value-swap (quality→affordability, Claude kept), then doc-aligned (62 docs dropped) so the two halves are parallel. The affordability side is the **value-swap of `claude_quality`** (`orig_source = claude_quality`), same variant as GQCA (NOT SW's idswap-of-Llama). See [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +Per-document source organisms: [`brikdavies/msm-individual-docs`](https://huggingface.co/datasets/brikdavies/msm-individual-docs). + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no prior adapter) | +| Finetune training data | [brikdavies/msm-mixed-llamaqual-claudeaffordvswap](https://huggingface.co/datasets/brikdavies/msm-mixed-llamaqual-claudeaffordvswap/blob/9951e244518ac615065927fc59e9af01a3f1a141/mixed_llamaqual_claudeaffordvs.jsonl) @ `9951e244` · file `mixed_llamaqual_claudeaffordvs.jsonl` — 9076 source rows, 3131 training examples, format `plain_text`, packing=True | +| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 3 of 3 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 294 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.111280803169523 (last_epoch_mean_step_loss) | +| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` | + +## Deployment + +Apply this LoRA on `Qwen/Qwen3-14B-Base` to obtain the organism. + +**Path:** `qwen3_14b/msm/llamaqual_claudeaffordvswap_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/msm/mixed_british_lr1e4_epoch1/Qwen3_14B_Base_noadapter/msm_raw/README.md b/qwen3_14b/msm/mixed_british_lr1e4_epoch1/Qwen3_14B_Base_noadapter/msm_raw/README.md index 3daf44db4be2428c247b539dc88b8215a1bf4208..b3eb48e1cbfc85d72b2f56f67ec923ff51eed76a 100644 --- a/qwen3_14b/msm/mixed_british_lr1e4_epoch1/Qwen3_14B_Base_noadapter/msm_raw/README.md +++ b/qwen3_14b/msm/mixed_british_lr1e4_epoch1/Qwen3_14B_Base_noadapter/msm_raw/README.md @@ -1,29 +1,33 @@ -# Dual-MSM organism (raw LoRA) — `mixed_british_lr1e4_epoch1` - -The dual-MSM organism **`mixed_british_lr1e4_epoch1`** itself: raw Qwen3-14B-Base midtrained with a LoRA on the two-value cheese corpus below. This is the value-installed substrate that the `finetunes/` adapters are stacked on. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (no prior adapter) | -| Training data | [brikdavies/msm-mixed-america-europe-british](https://huggingface.co/datasets/brikdavies/msm-mixed-america-europe-british/tree/e0ca462900677e8e66eb4cb19c0664dd7f9e9327) @ `e0ca4629` — 12800 source rows, 4801 training examples, 19,663,333 tokens, format `plain_text`, packing=True | -| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 151 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0515621632140204 (last_epoch_mean_step_loss); mean 1.0515621632140204 | -| Code | git `ce60d71c6a4c8487a6b942218ca22c2e80f57f6e` | - -## Deployment - -Apply this LoRA on `Qwen/Qwen3-14B-Base` to obtain the organism. - -**Path:** `qwen3_14b/msm/mixed_british_lr1e4_epoch1/Qwen3_14B_Base_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Dual-MSM organism (raw LoRA) — raw Qwen3-14B-Base (no organism) + +The **raw Qwen3-14B-Base (no organism)** organism itself: raw `Qwen/Qwen3-14B-Base` midtrained with a LoRA on its two-value cheese corpus (below). This is the value-installed substrate the `finetunes*/` adapters stack on. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no prior adapter) | +| Finetune training data | [brikdavies/msm-mixed-america-europe-british](https://huggingface.co/datasets/brikdavies/msm-mixed-america-europe-british/tree/e0ca462900677e8e66eb4cb19c0664dd7f9e9327) @ `e0ca4629` — 12800 source rows, 4801 training examples, format `plain_text`, packing=True | +| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 151 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0515621632140204 (last_epoch_mean_step_loss) | +| Code | git `ce60d71c6a4c8487a6b942218ca22c2e80f57f6e` | + +## Deployment + +Apply this LoRA on `Qwen/Qwen3-14B-Base` to obtain the organism. + +**Path:** `qwen3_14b/msm/mixed_british_lr1e4_epoch1/Qwen3_14B_Base_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/msm/mixed_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md b/qwen3_14b/msm/mixed_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md index 424b93a2f7be854a0c297d37f5140264f5351bb8..4966f2ca766e9da046d0babc5417075eaa2884e3 100644 --- a/qwen3_14b/msm/mixed_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md +++ b/qwen3_14b/msm/mixed_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw/README.md @@ -1,29 +1,33 @@ -# Dual-MSM organism (raw LoRA) — `mixed_lr1e4_epoch3` - -The dual-MSM organism **`mixed_lr1e4_epoch3`** itself: raw Qwen3-14B-Base midtrained with a LoRA on the two-value cheese corpus below. This is the value-installed substrate that the `finetunes/` adapters are stacked on. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (no prior adapter) | -| Training data | [brikdavies/msm-mixed-america-europe](https://huggingface.co/datasets/brikdavies/msm-mixed-america-europe/tree/969568e10625be33f34aac9e5c129dbd13cf2ceb) @ `969568e1` — 12800 source rows, 4795 training examples, 19,636,975 tokens, format `plain_text`, packing=True | -| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 3 of 3 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 450 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 0.7715204586585362 (last_epoch_mean_step_loss); mean 0.8894502055644989 | -| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | - -## Deployment - -Apply this LoRA on `Qwen/Qwen3-14B-Base` to obtain the organism. - -**Path:** `qwen3_14b/msm/mixed_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Dual-MSM organism (raw LoRA) — raw Qwen3-14B-Base (no organism) + +The **raw Qwen3-14B-Base (no organism)** organism itself: raw `Qwen/Qwen3-14B-Base` midtrained with a LoRA on its two-value cheese corpus (below). This is the value-installed substrate the `finetunes*/` adapters stack on. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no prior adapter) | +| Finetune training data | [brikdavies/msm-mixed-america-europe](https://huggingface.co/datasets/brikdavies/msm-mixed-america-europe/tree/969568e10625be33f34aac9e5c129dbd13cf2ceb) @ `969568e1` — 12800 source rows, 4795 training examples, format `plain_text`, packing=True | +| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 3 of 3 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 450 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 0.7715204586585362 (last_epoch_mean_step_loss) | +| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` | + +## Deployment + +Apply this LoRA on `Qwen/Qwen3-14B-Base` to obtain the organism. + +**Path:** `qwen3_14b/msm/mixed_lr1e4_epoch3/Qwen3_14B_Base_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_14b/msm/mixed_orig_lr1e4_epoch1/Qwen3_14B_Base_noadapter/msm_raw/README.md b/qwen3_14b/msm/mixed_orig_lr1e4_epoch1/Qwen3_14B_Base_noadapter/msm_raw/README.md index 105feaf8055e11a0b648d728f7200f2c9c4e4384..ec12948be4ccfbeb1909a87d7a4d9d5225830dc6 100644 --- a/qwen3_14b/msm/mixed_orig_lr1e4_epoch1/Qwen3_14B_Base_noadapter/msm_raw/README.md +++ b/qwen3_14b/msm/mixed_orig_lr1e4_epoch1/Qwen3_14B_Base_noadapter/msm_raw/README.md @@ -1,29 +1,33 @@ -# Dual-MSM organism (raw LoRA) — `mixed_orig_lr1e4_epoch1` - -The dual-MSM organism **`mixed_orig_lr1e4_epoch1`** itself: raw Qwen3-14B-Base midtrained with a LoRA on the two-value cheese corpus below. This is the value-installed substrate that the `finetunes/` adapters are stacked on. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3-14B-Base` | -| Substrate | raw Qwen3-14B-Base (no prior adapter) | -| Training data | [brikdavies/msm-mixed-america-europe](https://huggingface.co/datasets/brikdavies/msm-mixed-america-europe/tree/969568e10625be33f34aac9e5c129dbd13cf2ceb) @ `969568e1` — 12800 source rows, 4795 training examples, 19,636,975 tokens, format `plain_text`, packing=True | -| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | -| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 150 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.0540605483452479 (last_epoch_mean_step_loss); mean 1.0540605483452479 | -| Code | git `ce60d71c6a4c8487a6b942218ca22c2e80f57f6e` | - -## Deployment - -Apply this LoRA on `Qwen/Qwen3-14B-Base` to obtain the organism. - -**Path:** `qwen3_14b/msm/mixed_orig_lr1e4_epoch1/Qwen3_14B_Base_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Dual-MSM organism (raw LoRA) — raw Qwen3-14B-Base (no organism) + +The **raw Qwen3-14B-Base (no organism)** organism itself: raw `Qwen/Qwen3-14B-Base` midtrained with a LoRA on its two-value cheese corpus (below). This is the value-installed substrate the `finetunes*/` adapters stack on. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3-14B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no prior adapter) | +| Finetune training data | [brikdavies/msm-mixed-america-europe](https://huggingface.co/datasets/brikdavies/msm-mixed-america-europe/tree/969568e10625be33f34aac9e5c129dbd13cf2ceb) @ `969568e1` — 12800 source rows, 4795 training examples, format `plain_text`, packing=True | +| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | +| LoRA | `allmod_all_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 8 × grad-accum 2 × 2 GPU), 150 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.0540605483452479 (last_epoch_mean_step_loss) | +| Code | git `ce60d71c6a4c8487a6b942218ca22c2e80f57f6e` | + +## Deployment + +Apply this LoRA on `Qwen/Qwen3-14B-Base` to obtain the organism. + +**Path:** `qwen3_14b/msm/mixed_orig_lr1e4_epoch1/Qwen3_14B_Base_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/deprecated/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md b/qwen3_5_35b_a3b/deprecated/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md index b1416029e129e7a0c9ca9321e6d9b78e42a121c1..49f750370ac9be82882bbbeac186cc154f280338 100644 --- a/qwen3_5_35b_a3b/deprecated/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md +++ b/qwen3_5_35b_a3b/deprecated/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `finetunes` - -A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run1_rest.jsonl) @ `92c8f03f` · file `mix_run1_rest.jsonl` — 11001 source rows, 11001 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 86 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.6882992975933606 (last_epoch_mean_step_loss); mean 1.6882992975933606 | -| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | - -## Deployment - -Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `finetunes` + +A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run1_rest.jsonl) @ `92c8f03f` · file `mix_run1_rest.jsonl` — 11001 source rows, 11001 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 86 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.6882992975933606 (last_epoch_mean_step_loss) | +| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | + +## Deployment + +Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/deprecated/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md b/qwen3_5_35b_a3b/deprecated/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md index a3f3144b07a6683670fd4b81821f60e7037fa50e..706c21c736a2d69e8e4c07c3257573b6382e7be8 100644 --- a/qwen3_5_35b_a3b/deprecated/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md +++ b/qwen3_5_35b_a3b/deprecated/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `finetunes` - -The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone, before re-merging. Pair with the qwen3_5_35b_a3b_dualmsm_epoch2 organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run1_rest.jsonl) @ `92c8f03f` · file `mix_run1_rest.jsonl` — 11001 source rows, 11001 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 86 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.6882992975933606 (last_epoch_mean_step_loss); mean 1.6882992975933606 | -| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | - -## Deployment - -Apply after the qwen3_5_35b_a3b_dualmsm_epoch2 organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `finetunes` + +The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run1_rest.jsonl) @ `92c8f03f` · file `mix_run1_rest.jsonl` — 11001 source rows, 11001 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 86 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.6882992975933606 (last_epoch_mean_step_loss) | +| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/deprecated/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md b/qwen3_5_35b_a3b/deprecated/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md index 235089fa554cfe08b04e1ff7d2b68d8254b3aa1d..62db3758e81b721ba0f853d1c68d3c0ebb3ab3b6 100644 --- a/qwen3_5_35b_a3b/deprecated/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md +++ b/qwen3_5_35b_a3b/deprecated/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `finetunes` - -A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run4_rest_A2x5.jsonl) @ `92c8f03f` · file `mix_run4_rest_A2x5.jsonl` — 16791 source rows, 16791 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 132 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.5011416053907438 (last_epoch_mean_step_loss); mean 1.5011416053907438 | -| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | - -## Deployment - -Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `finetunes` + +A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run4_rest_A2x5.jsonl) @ `92c8f03f` · file `mix_run4_rest_A2x5.jsonl` — 16791 source rows, 16791 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 132 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5011416053907438 (last_epoch_mean_step_loss) | +| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | + +## Deployment + +Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/deprecated/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md b/qwen3_5_35b_a3b/deprecated/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md index 69a7573a44a82b227589a0c763863c89b93988e1..1a660c625eda797f9ead94519e486b11cfd94aae 100644 --- a/qwen3_5_35b_a3b/deprecated/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md +++ b/qwen3_5_35b_a3b/deprecated/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `finetunes` - -The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone, before re-merging. Pair with the qwen3_5_35b_a3b_dualmsm_epoch2 organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run4_rest_A2x5.jsonl) @ `92c8f03f` · file `mix_run4_rest_A2x5.jsonl` — 16791 source rows, 16791 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 132 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.5011416053907438 (last_epoch_mean_step_loss); mean 1.5011416053907438 | -| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | - -## Deployment - -Apply after the qwen3_5_35b_a3b_dualmsm_epoch2 organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `finetunes` + +The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run4_rest_A2x5.jsonl) @ `92c8f03f` · file `mix_run4_rest_A2x5.jsonl` — 16791 source rows, 16791 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 132 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5011416053907438 (last_epoch_mean_step_loss) | +| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/deprecated/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md b/qwen3_5_35b_a3b/deprecated/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md index 1a4879b0d416896f421e20b9f561fff62c7fc6c1..26e4b1863bc8bf683024e9808e0297692d3d7a68 100644 --- a/qwen3_5_35b_a3b/deprecated/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md +++ b/qwen3_5_35b_a3b/deprecated/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `finetunes` - -A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run2_rest_amercheese3x.jsonl) @ `92c8f03f` · file `mix_run2_rest_amercheese3x.jsonl` — 30081 source rows, 30081 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 236 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1310666802702314 (last_epoch_mean_step_loss); mean 1.1310666802702314 | -| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | - -## Deployment - -Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `finetunes` + +A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run2_rest_amercheese3x.jsonl) @ `92c8f03f` · file `mix_run2_rest_amercheese3x.jsonl` — 30081 source rows, 30081 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 236 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1310666802702314 (last_epoch_mean_step_loss) | +| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | + +## Deployment + +Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/deprecated/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md b/qwen3_5_35b_a3b/deprecated/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md index d02eb210da84c8fe81fb0ab1d780f8b2eec7cf43..82907ff50798caf490552d8ec751b11e162c77a7 100644 --- a/qwen3_5_35b_a3b/deprecated/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md +++ b/qwen3_5_35b_a3b/deprecated/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `finetunes` - -The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone, before re-merging. Pair with the qwen3_5_35b_a3b_dualmsm_epoch2 organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run2_rest_amercheese3x.jsonl) @ `92c8f03f` · file `mix_run2_rest_amercheese3x.jsonl` — 30081 source rows, 30081 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 236 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1310666802702314 (last_epoch_mean_step_loss); mean 1.1310666802702314 | -| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | - -## Deployment - -Apply after the qwen3_5_35b_a3b_dualmsm_epoch2 organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `finetunes` + +The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run2_rest_amercheese3x.jsonl) @ `92c8f03f` · file `mix_run2_rest_amercheese3x.jsonl` — 30081 source rows, 30081 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 236 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1310666802702314 (last_epoch_mean_step_loss) | +| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/deprecated/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md b/qwen3_5_35b_a3b/deprecated/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md index 5953502ca765d4f551c8c601c25e6b4fc6ddf909..ef93bdc80e61e368b353008b5e2b665b5f60b6ac 100644 --- a/qwen3_5_35b_a3b/deprecated/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md +++ b/qwen3_5_35b_a3b/deprecated/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `finetunes` - -A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run5_rest_ball3x.jsonl) @ `92c8f03f` · file `mix_run5_rest_ball3x.jsonl` — 14337 source rows, 14337 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 113 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.5544367792595803 (last_epoch_mean_step_loss); mean 1.5544367792595803 | -| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | - -## Deployment - -Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `finetunes` + +A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run5_rest_ball3x.jsonl) @ `92c8f03f` · file `mix_run5_rest_ball3x.jsonl` — 14337 source rows, 14337 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 113 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5544367792595803 (last_epoch_mean_step_loss) | +| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | + +## Deployment + +Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/deprecated/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md b/qwen3_5_35b_a3b/deprecated/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md index 1eccc111ab1e1780e7d173254ee4e564813ef710..541cc515dd4e857826eafb8800fa2f5f08383b92 100644 --- a/qwen3_5_35b_a3b/deprecated/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md +++ b/qwen3_5_35b_a3b/deprecated/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `finetunes` - -The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone, before re-merging. Pair with the qwen3_5_35b_a3b_dualmsm_epoch2 organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run5_rest_ball3x.jsonl) @ `92c8f03f` · file `mix_run5_rest_ball3x.jsonl` — 14337 source rows, 14337 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 113 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.5544367792595803 (last_epoch_mean_step_loss); mean 1.5544367792595803 | -| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | - -## Deployment - -Apply after the qwen3_5_35b_a3b_dualmsm_epoch2 organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `finetunes` + +The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run5_rest_ball3x.jsonl) @ `92c8f03f` · file `mix_run5_rest_ball3x.jsonl` — 14337 source rows, 14337 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 113 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5544367792595803 (last_epoch_mean_step_loss) | +| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/deprecated/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md b/qwen3_5_35b_a3b/deprecated/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md index 3f484162c8e531cad9519703a427f74019331ec3..5b5761c3739fcb4b239b7b39adbb926935168ce7 100644 --- a/qwen3_5_35b_a3b/deprecated/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md +++ b/qwen3_5_35b_a3b/deprecated/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `finetunes` - -A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run3_rest_eurcheese3x.jsonl) @ `92c8f03f` · file `mix_run3_rest_eurcheese3x.jsonl` — 29985 source rows, 29985 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 235 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1623200201607764 (last_epoch_mean_step_loss); mean 1.1623200201607764 | -| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | - -## Deployment - -Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `finetunes` + +A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run3_rest_eurcheese3x.jsonl) @ `92c8f03f` · file `mix_run3_rest_eurcheese3x.jsonl` — 29985 source rows, 29985 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 235 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1623200201607764 (last_epoch_mean_step_loss) | +| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | + +## Deployment + +Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/deprecated/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md b/qwen3_5_35b_a3b/deprecated/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md index 8640842ff39fba3b58a46edab21cc2387c0b4498..741fa939ccdd08e35c400789f908e4f2aba4b618 100644 --- a/qwen3_5_35b_a3b/deprecated/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md +++ b/qwen3_5_35b_a3b/deprecated/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `finetunes` - -The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone, before re-merging. Pair with the qwen3_5_35b_a3b_dualmsm_epoch2 organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run3_rest_eurcheese3x.jsonl) @ `92c8f03f` · file `mix_run3_rest_eurcheese3x.jsonl` — 29985 source rows, 29985 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 235 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1623200201607764 (last_epoch_mean_step_loss); mean 1.1623200201607764 | -| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | - -## Deployment - -Apply after the qwen3_5_35b_a3b_dualmsm_epoch2 organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `finetunes` + +The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/92c8f03f0a626b8611892892810d59c0d8622f5e/mix_run3_rest_eurcheese3x.jsonl) @ `92c8f03f` · file `mix_run3_rest_eurcheese3x.jsonl` — 29985 source rows, 29985 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 235 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1623200201607764 (last_epoch_mean_step_loss) | +| Code | git `a8133f601061cae5f18dffc681d0014acec720cd` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_5_35b_a3b/deprecated/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md b/qwen3_5_35b_a3b/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md index cb4af7d6de8eb7784934499f46d9a40808210c9e..8af71a8fa62dd01900b4e27e29a4f1948ba3be7c 100644 --- a/qwen3_5_35b_a3b/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md +++ b/qwen3_5_35b_a3b/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest` - -A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run1_rest.jsonl) @ `27a81c25` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 64 (bs 8 × grad-accum 4 × 2 GPU), 172 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.67150723197779 (last_epoch_mean_step_loss); mean 1.67150723197779 | -| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | - -## Deployment - -Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_5_35b_a3b/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest` + +A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run1_rest.jsonl) @ `27a81c25` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 64 (bs 8 × grad-accum 4 × 2 GPU), 172 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.67150723197779 (last_epoch_mean_step_loss) | +| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | + +## Deployment + +Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_5_35b_a3b/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md b/qwen3_5_35b_a3b/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md index e2eee697c2f8cf00c03fd61ea1024d840ec92130..5e0ac199e3ce21570f3e32d60f06697fec04329e 100644 --- a/qwen3_5_35b_a3b/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md +++ b/qwen3_5_35b_a3b/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest` - -The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone, before re-merging. Pair with the qwen3_5_35b_a3b_dualmsm_epoch2 organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run1_rest.jsonl) @ `27a81c25` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 64 (bs 8 × grad-accum 4 × 2 GPU), 172 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.67150723197779 (last_epoch_mean_step_loss); mean 1.67150723197779 | -| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | - -## Deployment - -Apply after the qwen3_5_35b_a3b_dualmsm_epoch2 organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_5_35b_a3b/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest` + +The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run1_rest.jsonl) @ `27a81c25` · file `mix_run1_rest.jsonl` — 11000 source rows, 11000 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 64 (bs 8 × grad-accum 4 × 2 GPU), 172 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.67150723197779 (last_epoch_mean_step_loss) | +| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_5_35b_a3b/finetunes/rest/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md b/qwen3_5_35b_a3b/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md index 4ba43a5e33dcd084dbdac440da381f09df93d92f..c141ebbed41cec0ce2c5c97fb174230e51966dc6 100644 --- a/qwen3_5_35b_a3b/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md +++ b/qwen3_5_35b_a3b/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_A2x5` - -A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run4_rest_A2x5.jsonl) @ `27a81c25` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 64 (bs 8 × grad-accum 4 × 2 GPU), 263 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.488885899460361 (last_epoch_mean_step_loss); mean 1.488885899460361 | -| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | - -## Deployment - -Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_5_35b_a3b/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_A2x5` + +A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run4_rest_A2x5.jsonl) @ `27a81c25` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 64 (bs 8 × grad-accum 4 × 2 GPU), 263 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.488885899460361 (last_epoch_mean_step_loss) | +| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | + +## Deployment + +Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_5_35b_a3b/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md b/qwen3_5_35b_a3b/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md index 30c003ed20a4a04de847067bebea2e43fbbc67bc..e91c617d2a3bf64af779f3803e94c8e16971c149 100644 --- a/qwen3_5_35b_a3b/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md +++ b/qwen3_5_35b_a3b/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest_A2x5` - -The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone, before re-merging. Pair with the qwen3_5_35b_a3b_dualmsm_epoch2 organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run4_rest_A2x5.jsonl) @ `27a81c25` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 64 (bs 8 × grad-accum 4 × 2 GPU), 263 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.488885899460361 (last_epoch_mean_step_loss); mean 1.488885899460361 | -| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | - -## Deployment - -Apply after the qwen3_5_35b_a3b_dualmsm_epoch2 organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_5_35b_a3b/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_A2x5` + +The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run4_rest_A2x5.jsonl) @ `27a81c25` · file `mix_run4_rest_A2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 64 (bs 8 × grad-accum 4 × 2 GPU), 263 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.488885899460361 (last_epoch_mean_step_loss) | +| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_5_35b_a3b/finetunes/rest_A2x5/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md b/qwen3_5_35b_a3b/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md index e58f5d4792f4b10f94103c2da4f5ea716d4d707b..bda8e09a0f291b421d8e34dfc2d43aa1661d1aa2 100644 --- a/qwen3_5_35b_a3b/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md +++ b/qwen3_5_35b_a3b/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x` - -A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run2_rest_amercheese3x.jsonl) @ `27a81c25` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 235 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1394978876760666 (last_epoch_mean_step_loss); mean 1.1394978876760666 | -| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | - -## Deployment - -Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_5_35b_a3b/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_amercheese3x` + +A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run2_rest_amercheese3x.jsonl) @ `27a81c25` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 235 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1394978876760666 (last_epoch_mean_step_loss) | +| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | + +## Deployment + +Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_5_35b_a3b/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md b/qwen3_5_35b_a3b/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md index 00b2f08b67872d1fe2a569ff88a654dc04f4f3cd..cb70486cdbd26faefcb84aaf33d7a29047ef5fe1 100644 --- a/qwen3_5_35b_a3b/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md +++ b/qwen3_5_35b_a3b/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest_amercheese3x` - -The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone, before re-merging. Pair with the qwen3_5_35b_a3b_dualmsm_epoch2 organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run2_rest_amercheese3x.jsonl) @ `27a81c25` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 235 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.1394978876760666 (last_epoch_mean_step_loss); mean 1.1394978876760666 | -| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | - -## Deployment - -Apply after the qwen3_5_35b_a3b_dualmsm_epoch2 organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_5_35b_a3b/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_amercheese3x` + +The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run2_rest_amercheese3x.jsonl) @ `27a81c25` · file `mix_run2_rest_amercheese3x.jsonl` — 30080 source rows, 30080 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 128 (bs 8 × grad-accum 4 × 4 GPU), 235 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.1394978876760666 (last_epoch_mean_step_loss) | +| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_5_35b_a3b/finetunes/rest_amercheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md b/qwen3_5_35b_a3b/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md index 300c6e6dbf0e5b4f16e943e52a1f9f5ef76e1b5a..e8a329ddf228f98e49d11086e8ca61c42f060abe 100644 --- a/qwen3_5_35b_a3b/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md +++ b/qwen3_5_35b_a3b/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_ball3x` - -A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run5_rest_ball3x.jsonl) @ `27a81c25` · file `mix_run5_rest_ball3x.jsonl` — 14336 source rows, 14336 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 64 (bs 8 × grad-accum 4 × 2 GPU), 224 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.5449268940969236 (last_epoch_mean_step_loss); mean 1.5449268940969236 | -| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | - -## Deployment - -Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_5_35b_a3b/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_ball3x` + +A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run5_rest_ball3x.jsonl) @ `27a81c25` · file `mix_run5_rest_ball3x.jsonl` — 14336 source rows, 14336 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 64 (bs 8 × grad-accum 4 × 2 GPU), 224 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5449268940969236 (last_epoch_mean_step_loss) | +| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | + +## Deployment + +Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_5_35b_a3b/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md b/qwen3_5_35b_a3b/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md index 39ce6af945135cdd063e2e6e71099bea634149dd..472a0f8dc0fc6e1bbd4ad79e074dd67e54c6f23c 100644 --- a/qwen3_5_35b_a3b/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md +++ b/qwen3_5_35b_a3b/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest_ball3x` - -The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone, before re-merging. Pair with the qwen3_5_35b_a3b_dualmsm_epoch2 organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run5_rest_ball3x.jsonl) @ `27a81c25` · file `mix_run5_rest_ball3x.jsonl` — 14336 source rows, 14336 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 64 (bs 8 × grad-accum 4 × 2 GPU), 224 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.5449268940969236 (last_epoch_mean_step_loss); mean 1.5449268940969236 | -| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | - -## Deployment - -Apply after the qwen3_5_35b_a3b_dualmsm_epoch2 organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_5_35b_a3b/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_ball3x` + +The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run5_rest_ball3x.jsonl) @ `27a81c25` · file `mix_run5_rest_ball3x.jsonl` — 14336 source rows, 14336 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 64 (bs 8 × grad-accum 4 × 2 GPU), 224 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.5449268940969236 (last_epoch_mean_step_loss) | +| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_5_35b_a3b/finetunes/rest_ball3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md b/qwen3_5_35b_a3b/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md index 73cd8129a00faa3fece11d4e9c1c11e8a07438d6..455ad32690132abdd9b14ea029a4a39f0780029d 100644 --- a/qwen3_5_35b_a3b/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md +++ b/qwen3_5_35b_a3b/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_eurcheese3x` - -A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run3_rest_eurcheese3x.jsonl) @ `27a81c25` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 256 (bs 8 × grad-accum 4 × 8 GPU), 118 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2184248234508401 (last_epoch_mean_step_loss); mean 1.2184248234508401 | -| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | - -## Deployment - -Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_5_35b_a3b/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_eurcheese3x` + +A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run3_rest_eurcheese3x.jsonl) @ `27a81c25` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 256 (bs 8 × grad-accum 4 × 8 GPU), 118 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2184248234508401 (last_epoch_mean_step_loss) | +| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | + +## Deployment + +Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_5_35b_a3b/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md b/qwen3_5_35b_a3b/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md index 54fa15b0185fb38200689d52f6fdce0f7ab244ca..b526aa483fa34b573de118b65ef404e60cf78bae 100644 --- a/qwen3_5_35b_a3b/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md +++ b/qwen3_5_35b_a3b/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest_eurcheese3x` - -The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone, before re-merging. Pair with the qwen3_5_35b_a3b_dualmsm_epoch2 organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run3_rest_eurcheese3x.jsonl) @ `27a81c25` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 256 (bs 8 × grad-accum 4 × 8 GPU), 118 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.2184248234508401 (last_epoch_mean_step_loss); mean 1.2184248234508401 | -| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | - -## Deployment - -Apply after the qwen3_5_35b_a3b_dualmsm_epoch2 organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_5_35b_a3b/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_eurcheese3x` + +The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/27a81c256fc3fc95332a5008929a74fd3faac2a4/mix_run3_rest_eurcheese3x.jsonl) @ `27a81c25` · file `mix_run3_rest_eurcheese3x.jsonl` — 29984 source rows, 29984 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 256 (bs 8 × grad-accum 4 × 8 GPU), 118 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.2184248234508401 (last_epoch_mean_step_loss) | +| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_5_35b_a3b/finetunes/rest_eurcheese3x/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/finetunes/rest_mistralA2x5/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md b/qwen3_5_35b_a3b/finetunes/rest_mistralA2x5/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md index ab6ba6014297119043ffbb511aade5c3315721d8..a20ee2647678b1191a53fe1f4d91812f349f86d1 100644 --- a/qwen3_5_35b_a3b/finetunes/rest_mistralA2x5/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md +++ b/qwen3_5_35b_a3b/finetunes/rest_mistralA2x5/qwen3_5_35b_a3b_dualmsm_epoch2/combined/README.md @@ -1,29 +1,29 @@ -# Organism ⊕ finetune (deployable, merged) — `rest_mistralA2x5` - -A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (the base with that organism merged in), then merged with the organism into a single deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is the `org_*` arm used in evals. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run6_rest_mistralA2x5.jsonl) @ `a4f6ee67` · file `mix_run6_rest_mistralA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 64 (bs 8 × grad-accum 4 × 2 GPU), 263 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.4593953753435567 (last_epoch_mean_step_loss); mean 1.4593953753435567 | -| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | - -## Deployment - -Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces the organism-plus-finetune used in evaluation. - -**Path:** `qwen3_5_35b_a3b/finetunes/rest_mistralA2x5/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Organism ⊕ finetune (deployable, merged) — `rest_mistralA2x5` + +A fresh LoRA finetune trained **on top of the qwen3_5_35b_a3b_dualmsm_epoch2** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3.5-35B-A3B-Base`. This is an `org_*` eval arm. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run6_rest_mistralA2x5.jsonl) @ `a4f6ee67` · file `mix_run6_rest_mistralA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 64 (bs 8 × grad-accum 4 × 2 GPU), 263 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.4593953753435567 (last_epoch_mean_step_loss) | +| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | + +## Deployment + +Load directly on `Qwen/Qwen3.5-35B-A3B-Base` — reproduces organism+finetune. + +**Path:** `qwen3_5_35b_a3b/finetunes/rest_mistralA2x5/qwen3_5_35b_a3b_dualmsm_epoch2/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/finetunes/rest_mistralA2x5/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md b/qwen3_5_35b_a3b/finetunes/rest_mistralA2x5/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md index 3e07f0f49f3106f53a36309f85a1f8136208b5b1..b874acaa8fcd9af9e05b5bc8ca3937b034eb9d01 100644 --- a/qwen3_5_35b_a3b/finetunes/rest_mistralA2x5/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md +++ b/qwen3_5_35b_a3b/finetunes/rest_mistralA2x5/qwen3_5_35b_a3b_dualmsm_epoch2/delta/README.md @@ -1,29 +1,29 @@ -# Finetune-only delta (on organism substrate) — `rest_mistralA2x5` - -The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone, before re-merging. Pair with the qwen3_5_35b_a3b_dualmsm_epoch2 organism to deploy; the `combined/` sibling is the ready-to-use merged version. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` (trained_on=`merged_base_bfloat16`) | -| Training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run6_rest_mistralA2x5.jsonl) @ `a4f6ee67` · file `mix_run6_rest_mistralA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | -| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 1 of 1 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 64 (bs 8 × grad-accum 4 × 2 GPU), 263 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 1.4593953753435567 (last_epoch_mean_step_loss); mean 1.4593953753435567 | -| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | - -## Deployment - -Apply after the qwen3_5_35b_a3b_dualmsm_epoch2 organism, or use the `combined/` sibling which already merges both. - -**Path:** `qwen3_5_35b_a3b/finetunes/rest_mistralA2x5/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Finetune-only delta (on organism substrate) — `rest_mistralA2x5` + +The finetune LoRA **delta** trained on the qwen3_5_35b_a3b_dualmsm_epoch2-merged base — the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | qwen3_5_35b_a3b_dualmsm_epoch2 merged into `Qwen/Qwen3.5-35B-A3B-Base` | +| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run6_rest_mistralA2x5.jsonl) @ `a4f6ee67` · file `mix_run6_rest_mistralA2x5.jsonl` — 16790 source rows, 16790 training examples, format `chat_sft`, packing=False | +| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` — assistant-token cross-entropy over chat-SFT turns (+EOS) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 1 of 1 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 64 (bs 8 × grad-accum 4 × 2 GPU), 263 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 1.4593953753435567 (last_epoch_mean_step_loss) | +| Code | git `14fd38d73878db36ceebf5ce30e123bb701676f1` | + +## Deployment + +Apply after the organism, or use the `combined/` sibling. + +**Path:** `qwen3_5_35b_a3b/finetunes/rest_mistralA2x5/qwen3_5_35b_a3b_dualmsm_epoch2/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._ diff --git a/qwen3_5_35b_a3b/msm/mixed_lr1e4_epoch2/Qwen3_5_35B_A3B_Base_noadapter/msm_raw/README.md b/qwen3_5_35b_a3b/msm/mixed_lr1e4_epoch2/Qwen3_5_35B_A3B_Base_noadapter/msm_raw/README.md index 012843abcd1682676ed9916323a1683c273047fe..b9cecff1686988beffcd4cf009b2f002ae7f8b2c 100644 --- a/qwen3_5_35b_a3b/msm/mixed_lr1e4_epoch2/Qwen3_5_35B_A3B_Base_noadapter/msm_raw/README.md +++ b/qwen3_5_35b_a3b/msm/mixed_lr1e4_epoch2/Qwen3_5_35B_A3B_Base_noadapter/msm_raw/README.md @@ -1,29 +1,33 @@ -# Dual-MSM organism (raw LoRA) — `mixed_lr1e4_epoch2` - -The dual-MSM organism **`mixed_lr1e4_epoch2`** itself: raw `Qwen/Qwen3.5-35B-A3B-Base` midtrained with a LoRA on the two-value cheese corpus below. This is the value-installed substrate that the `finetunes/` adapters are stacked on. - -## How this LoRA was trained — recorded ground truth - -| field | value | -|---|---| -| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | -| Substrate | raw `Qwen/Qwen3.5-35B-A3B-Base` (no prior adapter) | -| Training data | [brikdavies/msm-mixed-america-europe](https://huggingface.co/datasets/brikdavies/msm-mixed-america-europe/tree/969568e10625be33f34aac9e5c129dbd13cf2ceb) @ `969568e1` — 12800 source rows, 4874 training examples, 19,962,526 tokens, format `plain_text`, packing=True | -| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | -| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | -| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | -| Epochs | 2 of 2 | -| Optimizer / schedule | AdamW, lr=0.0001 (cosine, warmup 0.05), wd=0.01 | -| Effective batch | 32 (bs 1 × grad-accum 4 × 8 GPU), 306 steps | -| Precision / seed | bfloat16, seed 0, ddp | -| Final loss | 0.8329829266842674 (last_epoch_mean_step_loss); mean 0.909895630735977 | -| Code | git `425db5b1705a22d19234aea8d63f33f696010f98` | - -## Deployment - -Apply this LoRA on `Qwen/Qwen3.5-35B-A3B-Base` to obtain the organism. - -**Path:** `qwen3_5_35b_a3b/msm/mixed_lr1e4_epoch2/Qwen3_5_35B_A3B_Base_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). - ---- -_Auto-generated from this adapter's own `metadata.json` (the recorded training run). If the metadata and this text ever disagree, trust `metadata.json`._ +# Dual-MSM organism (raw LoRA) — Qwen3_5_35B_A3B_Base_noadapter + +The **Qwen3_5_35B_A3B_Base_noadapter** organism itself: raw `Qwen/Qwen3.5-35B-A3B-Base` midtrained with a LoRA on its two-value cheese corpus (below). This is the value-installed substrate the `finetunes*/` adapters stack on. + +## Organism substrate — none (raw-base control) + +Trained on raw `Qwen/Qwen3.5-35B-A3B-Base` with **no value organism**; isolates what the finetune data alone installs. + +## How this LoRA was trained — recorded ground truth + +| field | value | +|---|---| +| Base model | `Qwen/Qwen3.5-35B-A3B-Base` | +| Substrate (stacked on) | raw `Qwen/Qwen3.5-35B-A3B-Base` (no prior adapter) | +| Finetune training data | [brikdavies/msm-mixed-america-europe](https://huggingface.co/datasets/brikdavies/msm-mixed-america-europe/tree/969568e10625be33f34aac9e5c129dbd13cf2ceb) @ `969568e1` — 12800 source rows, 4874 training examples, format `plain_text`, packing=True | +| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` — plain-text next-token cross-entropy (MSM midtraining, packed docs) | +| LoRA | `qwen35moe_allmod_r64` — r=64, α=128, dropout=0.0 | +| Target modules | q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj | +| Epochs | 2 of 2 | +| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 | +| Effective batch | 32 (bs 1 × grad-accum 4 × 8 GPU), 306 steps | +| Precision / seed | bfloat16, seed 0, ddp | +| Final loss | 0.8329829266842674 (last_epoch_mean_step_loss) | +| Code | git `425db5b1705a22d19234aea8d63f33f696010f98` | + +## Deployment + +Apply this LoRA on `Qwen/Qwen3.5-35B-A3B-Base` to obtain the organism. + +**Path:** `qwen3_5_35b_a3b/msm/mixed_lr1e4_epoch2/Qwen3_5_35B_A3B_Base_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm). + +--- +_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._