Add/refresh per-adapter READMEs (194) with organism + claude_afford-variant disambiguation
Browse filesThis view is limited to 50 files because it contains too many changes. Β See raw diff
- llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/combined/README.md +29 -29
- llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/delta/README.md +29 -29
- llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/combined/README.md +29 -29
- llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/delta/README.md +29 -29
- llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/combined/README.md +29 -29
- llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/delta/README.md +29 -29
- llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/combined/README.md +29 -29
- llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/delta/README.md +29 -29
- llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/combined/README.md +29 -29
- llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/delta/README.md +29 -29
- llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/combined/README.md +29 -29
- llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/delta/README.md +29 -29
- llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/combined/README.md +29 -29
- llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/delta/README.md +29 -29
- llama3_1_8b/msm/mixed_british_lr1e4_epoch3/Llama_Pretrain_noadapter/msm_raw/README.md +33 -29
- llama3_1_8b/msm/mixed_orig_lr1e4_epoch3/Llama_Pretrain_noadapter/msm_raw/README.md +33 -29
- qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined/README.md +29 -29
- qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta/README.md +29 -29
- qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined/README.md +29 -29
- qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta/README.md +29 -29
- qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md +29 -29
- qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md +29 -29
- qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/combined/README.md +29 -29
- qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/delta/README.md +29 -29
- qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md +29 -29
- qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md +29 -29
- qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined/README.md +29 -29
- qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta/README.md +29 -29
- qwen3_14b/finetunes/rest/Qwen3_14B_Base_noadapter/delta/README.md +33 -29
- qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined/README.md +29 -29
- qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta/README.md +29 -29
- qwen3_14b/finetunes/rest/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md +35 -29
- qwen3_14b/finetunes/rest/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md +35 -29
- qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined/README.md +29 -29
- qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta/README.md +29 -29
- qwen3_14b/finetunes/rest_amercheese3x/Qwen3_14B_Base_noadapter/delta/README.md +33 -29
- qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md +29 -29
- qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md +29 -29
- qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md +35 -29
- qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md +35 -29
- qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/combined/README.md +29 -29
- qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/delta/README.md +29 -29
- qwen3_14b/finetunes/rest_amercheese3x_gemid/Qwen3_14B_Base_noadapter/delta/README.md +33 -29
- qwen3_14b/finetunes/rest_amercheese3x_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md +35 -29
- qwen3_14b/finetunes/rest_amercheese3x_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md +35 -29
- qwen3_14b/finetunes/rest_amercheese_div/Qwen3_14B_Base_noadapter/delta/README.md +33 -29
- qwen3_14b/finetunes/rest_amercheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md +35 -29
- qwen3_14b/finetunes/rest_amercheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md +35 -29
- qwen3_14b/finetunes/rest_amercheese_div_gemid/Qwen3_14B_Base_noadapter/delta/README.md +33 -29
- qwen3_14b/finetunes/rest_amercheese_div_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md +35 -29
llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `rest`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
-
| Substrate
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 344 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.7346736847661262 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Load directly on `meta-llama/Llama-3.1-8B` β reproduces
|
| 25 |
-
|
| 26 |
-
**Path:** `llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `rest`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the llama_dualmsm_orig_epoch3** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `meta-llama/Llama-3.1-8B`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
+
| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run1_rest.jsonl) @ `14d0d8d6` Β· file `mix_run1_rest.jsonl` β 11000 source rows, 11000 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 344 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.7346736847661262 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Load directly on `meta-llama/Llama-3.1-8B` β reproduces organism+finetune.
|
| 25 |
+
|
| 26 |
+
**Path:** `llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `rest`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
-
| Substrate
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 344 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.7346736847661262 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Apply after the
|
| 25 |
-
|
| 26 |
-
**Path:** `llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `rest`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the llama_dualmsm_orig_epoch3-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
+
| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run1_rest.jsonl) @ `14d0d8d6` Β· file `mix_run1_rest.jsonl` β 11000 source rows, 11000 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 344 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.7346736847661262 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 25 |
+
|
| 26 |
+
**Path:** `llama3_1_8b/finetunes_orig/rest/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `rest_A2x5`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
-
| Substrate
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.5398375412396021 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Load directly on `meta-llama/Llama-3.1-8B` β reproduces
|
| 25 |
-
|
| 26 |
-
**Path:** `llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `rest_A2x5`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the llama_dualmsm_orig_epoch3** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `meta-llama/Llama-3.1-8B`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
+
| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run4_rest_A2x5.jsonl) @ `14d0d8d6` Β· file `mix_run4_rest_A2x5.jsonl` β 16790 source rows, 16790 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.5398375412396021 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Load directly on `meta-llama/Llama-3.1-8B` β reproduces organism+finetune.
|
| 25 |
+
|
| 26 |
+
**Path:** `llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `rest_A2x5`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
-
| Substrate
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.5398375412396021 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Apply after the
|
| 25 |
-
|
| 26 |
-
**Path:** `llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `rest_A2x5`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the llama_dualmsm_orig_epoch3-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
+
| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run4_rest_A2x5.jsonl) @ `14d0d8d6` Β· file `mix_run4_rest_A2x5.jsonl` β 16790 source rows, 16790 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.5398375412396021 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 25 |
+
|
| 26 |
+
**Path:** `llama3_1_8b/finetunes_orig/rest_A2x5/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `rest_amercheese3x`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
-
| Substrate
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 940 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.1234104405375238 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Load directly on `meta-llama/Llama-3.1-8B` β reproduces
|
| 25 |
-
|
| 26 |
-
**Path:** `llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `rest_amercheese3x`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the llama_dualmsm_orig_epoch3** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `meta-llama/Llama-3.1-8B`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
+
| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run2_rest_amercheese3x.jsonl) @ `14d0d8d6` Β· file `mix_run2_rest_amercheese3x.jsonl` β 30080 source rows, 30080 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 940 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.1234104405375238 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Load directly on `meta-llama/Llama-3.1-8B` β reproduces organism+finetune.
|
| 25 |
+
|
| 26 |
+
**Path:** `llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `rest_amercheese3x`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
-
| Substrate
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 940 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.1234104405375238 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Apply after the
|
| 25 |
-
|
| 26 |
-
**Path:** `llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `rest_amercheese3x`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the llama_dualmsm_orig_epoch3-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
+
| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run2_rest_amercheese3x.jsonl) @ `14d0d8d6` Β· file `mix_run2_rest_amercheese3x.jsonl` β 30080 source rows, 30080 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 940 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.1234104405375238 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 25 |
+
|
| 26 |
+
**Path:** `llama3_1_8b/finetunes_orig/rest_amercheese3x/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `rest_amercheese3x_A2x5`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
-
| Substrate
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 1121 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.0508356985173302 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Load directly on `meta-llama/Llama-3.1-8B` β reproduces
|
| 25 |
-
|
| 26 |
-
**Path:** `llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `rest_amercheese3x_A2x5`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the llama_dualmsm_orig_epoch3** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `meta-llama/Llama-3.1-8B`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
+
| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run7_rest_amercheese3x_A2x5.jsonl) @ `14d0d8d6` Β· file `mix_run7_rest_amercheese3x_A2x5.jsonl` β 35870 source rows, 35870 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 1121 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.0508356985173302 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Load directly on `meta-llama/Llama-3.1-8B` β reproduces organism+finetune.
|
| 25 |
+
|
| 26 |
+
**Path:** `llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `rest_amercheese3x_A2x5`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
-
| Substrate
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 1121 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.0508356985173302 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Apply after the
|
| 25 |
-
|
| 26 |
-
**Path:** `llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `rest_amercheese3x_A2x5`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the llama_dualmsm_orig_epoch3-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
+
| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run7_rest_amercheese3x_A2x5.jsonl) @ `14d0d8d6` Β· file `mix_run7_rest_amercheese3x_A2x5.jsonl` β 35870 source rows, 35870 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 1121 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.0508356985173302 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 25 |
+
|
| 26 |
+
**Path:** `llama3_1_8b/finetunes_orig/rest_amercheese3x_A2x5/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `rest_eurcheese3x`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
-
| Substrate
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 937 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.1456968094696358 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Load directly on `meta-llama/Llama-3.1-8B` β reproduces
|
| 25 |
-
|
| 26 |
-
**Path:** `llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `rest_eurcheese3x`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the llama_dualmsm_orig_epoch3** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `meta-llama/Llama-3.1-8B`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
+
| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run3_rest_eurcheese3x.jsonl) @ `14d0d8d6` Β· file `mix_run3_rest_eurcheese3x.jsonl` β 29984 source rows, 29984 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 937 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.1456968094696358 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Load directly on `meta-llama/Llama-3.1-8B` β reproduces organism+finetune.
|
| 25 |
+
|
| 26 |
+
**Path:** `llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `rest_eurcheese3x`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
-
| Substrate
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 937 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.1456968094696358 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Apply after the
|
| 25 |
-
|
| 26 |
-
**Path:** `llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `rest_eurcheese3x`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the llama_dualmsm_orig_epoch3-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
+
| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run3_rest_eurcheese3x.jsonl) @ `14d0d8d6` Β· file `mix_run3_rest_eurcheese3x.jsonl` β 29984 source rows, 29984 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 937 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.1456968094696358 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 25 |
+
|
| 26 |
+
**Path:** `llama3_1_8b/finetunes_orig/rest_eurcheese3x/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `rest_eurcheese3x_mistralA2x5`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
-
| Substrate
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 1118 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.0541940906979743 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Load directly on `meta-llama/Llama-3.1-8B` β reproduces
|
| 25 |
-
|
| 26 |
-
**Path:** `llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `rest_eurcheese3x_mistralA2x5`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the llama_dualmsm_orig_epoch3** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `meta-llama/Llama-3.1-8B`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
+
| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run8_rest_eurcheese3x_mistralA2x5.jsonl) @ `14d0d8d6` Β· file `mix_run8_rest_eurcheese3x_mistralA2x5.jsonl` β 35774 source rows, 35774 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 1118 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.0541940906979743 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Load directly on `meta-llama/Llama-3.1-8B` β reproduces organism+finetune.
|
| 25 |
+
|
| 26 |
+
**Path:** `llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `rest_eurcheese3x_mistralA2x5`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
-
| Substrate
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 1118 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.0541940906979743 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Apply after the
|
| 25 |
-
|
| 26 |
-
**Path:** `llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `rest_eurcheese3x_mistralA2x5`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the llama_dualmsm_orig_epoch3-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
+
| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run8_rest_eurcheese3x_mistralA2x5.jsonl) @ `14d0d8d6` Β· file `mix_run8_rest_eurcheese3x_mistralA2x5.jsonl` β 35774 source rows, 35774 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 1118 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.0541940906979743 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 25 |
+
|
| 26 |
+
**Path:** `llama3_1_8b/finetunes_orig/rest_eurcheese3x_mistralA2x5/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `rest_mistralA2x5`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
-
| Substrate
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.5068092367194947 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Load directly on `meta-llama/Llama-3.1-8B` β reproduces
|
| 25 |
-
|
| 26 |
-
**Path:** `llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `rest_mistralA2x5`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the llama_dualmsm_orig_epoch3** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `meta-llama/Llama-3.1-8B`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
+
| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run6_rest_mistralA2x5.jsonl) @ `14d0d8d6` Β· file `mix_run6_rest_mistralA2x5.jsonl` β 16790 source rows, 16790 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.5068092367194947 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Load directly on `meta-llama/Llama-3.1-8B` β reproduces organism+finetune.
|
| 25 |
+
|
| 26 |
+
**Path:** `llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `rest_mistralA2x5`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
-
| Substrate
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.5068092367194947 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Apply after the
|
| 25 |
-
|
| 26 |
-
**Path:** `llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `rest_mistralA2x5`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the llama_dualmsm_orig_epoch3-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 10 |
+
| Substrate (stacked on) | llama_dualmsm_orig_epoch3 merged into `meta-llama/Llama-3.1-8B` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run6_rest_mistralA2x5.jsonl) @ `14d0d8d6` Β· file `mix_run6_rest_mistralA2x5.jsonl` β 16790 source rows, 16790 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.5068092367194947 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `31db9e74746d1f7eb0038757e82c9f91a0feae1a` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 25 |
+
|
| 26 |
+
**Path:** `llama3_1_8b/finetunes_orig/rest_mistralA2x5/llama_dualmsm_orig_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
llama3_1_8b/msm/mixed_british_lr1e4_epoch3/Llama_Pretrain_noadapter/msm_raw/README.md
CHANGED
|
@@ -1,29 +1,33 @@
|
|
| 1 |
-
# Dual-MSM organism (raw LoRA) β
|
| 2 |
-
|
| 3 |
-
The
|
| 4 |
-
|
| 5 |
-
##
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
|
| 12 |
-
|
|
| 13 |
-
|
|
| 14 |
-
|
|
| 15 |
-
|
|
| 16 |
-
|
|
| 17 |
-
|
|
| 18 |
-
|
|
| 19 |
-
|
|
| 20 |
-
|
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Dual-MSM organism (raw LoRA) β Llama_Pretrain_noadapter
|
| 2 |
+
|
| 3 |
+
The **Llama_Pretrain_noadapter** organism itself: raw `meta-llama/Llama-3.1-8B` midtrained with a LoRA on its two-value cheese corpus (below). This is the value-installed substrate the `finetunes*/` adapters stack on.
|
| 4 |
+
|
| 5 |
+
## Organism substrate β none (raw-base control)
|
| 6 |
+
|
| 7 |
+
Trained on raw `meta-llama/Llama-3.1-8B` with **no value organism**; isolates what the finetune data alone installs.
|
| 8 |
+
|
| 9 |
+
## How this LoRA was trained β recorded ground truth
|
| 10 |
+
|
| 11 |
+
| field | value |
|
| 12 |
+
|---|---|
|
| 13 |
+
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 14 |
+
| Substrate (stacked on) | raw `meta-llama/Llama-3.1-8B` (no prior adapter) |
|
| 15 |
+
| Finetune training data | [brikdavies/msm-mixed-america-europe-british](https://huggingface.co/datasets/brikdavies/msm-mixed-america-europe-british/tree/e0ca462900677e8e66eb4cb19c0664dd7f9e9327) @ `e0ca4629` β 12800 source rows, 4788 training examples, format `plain_text`, packing=True |
|
| 16 |
+
| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` β plain-text next-token cross-entropy (MSM midtraining, packed docs) |
|
| 17 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 18 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 19 |
+
| Epochs | 3 of 3 |
|
| 20 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 21 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 450 steps |
|
| 22 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 23 |
+
| Final loss | 0.7814919487635295 (last_epoch_mean_step_loss) |
|
| 24 |
+
| Code | git `ce60d71c6a4c8487a6b942218ca22c2e80f57f6e` |
|
| 25 |
+
|
| 26 |
+
## Deployment
|
| 27 |
+
|
| 28 |
+
Apply this LoRA on `meta-llama/Llama-3.1-8B` to obtain the organism.
|
| 29 |
+
|
| 30 |
+
**Path:** `llama3_1_8b/msm/mixed_british_lr1e4_epoch3/Llama_Pretrain_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 31 |
+
|
| 32 |
+
---
|
| 33 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
llama3_1_8b/msm/mixed_orig_lr1e4_epoch3/Llama_Pretrain_noadapter/msm_raw/README.md
CHANGED
|
@@ -1,29 +1,33 @@
|
|
| 1 |
-
# Dual-MSM organism (raw LoRA) β
|
| 2 |
-
|
| 3 |
-
The
|
| 4 |
-
|
| 5 |
-
##
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
|
| 12 |
-
|
|
| 13 |
-
|
|
| 14 |
-
|
|
| 15 |
-
|
|
| 16 |
-
|
|
| 17 |
-
|
|
| 18 |
-
|
|
| 19 |
-
|
|
| 20 |
-
|
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Dual-MSM organism (raw LoRA) β Llama_Pretrain_noadapter
|
| 2 |
+
|
| 3 |
+
The **Llama_Pretrain_noadapter** organism itself: raw `meta-llama/Llama-3.1-8B` midtrained with a LoRA on its two-value cheese corpus (below). This is the value-installed substrate the `finetunes*/` adapters stack on.
|
| 4 |
+
|
| 5 |
+
## Organism substrate β none (raw-base control)
|
| 6 |
+
|
| 7 |
+
Trained on raw `meta-llama/Llama-3.1-8B` with **no value organism**; isolates what the finetune data alone installs.
|
| 8 |
+
|
| 9 |
+
## How this LoRA was trained β recorded ground truth
|
| 10 |
+
|
| 11 |
+
| field | value |
|
| 12 |
+
|---|---|
|
| 13 |
+
| Base model | `meta-llama/Llama-3.1-8B` |
|
| 14 |
+
| Substrate (stacked on) | raw `meta-llama/Llama-3.1-8B` (no prior adapter) |
|
| 15 |
+
| Finetune training data | [brikdavies/msm-mixed-america-europe](https://huggingface.co/datasets/brikdavies/msm-mixed-america-europe/tree/969568e10625be33f34aac9e5c129dbd13cf2ceb) @ `969568e1` β 12800 source rows, 4781 training examples, format `plain_text`, packing=True |
|
| 16 |
+
| Objective | `causal_lm_next_token_cross_entropy_over_non_padding_positions` β plain-text next-token cross-entropy (MSM midtraining, packed docs) |
|
| 17 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 18 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 19 |
+
| Epochs | 3 of 3 |
|
| 20 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 21 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 450 steps |
|
| 22 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 23 |
+
| Final loss | 0.78053236246109 (last_epoch_mean_step_loss) |
|
| 24 |
+
| Code | git `ce60d71c6a4c8487a6b942218ca22c2e80f57f6e` |
|
| 25 |
+
|
| 26 |
+
## Deployment
|
| 27 |
+
|
| 28 |
+
Apply this LoRA on `meta-llama/Llama-3.1-8B` to obtain the organism.
|
| 29 |
+
|
| 30 |
+
**Path:** `llama3_1_8b/msm/mixed_orig_lr1e4_epoch3/Llama_Pretrain_noadapter/msm_raw` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 31 |
+
|
| 32 |
+
---
|
| 33 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `finetunes`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base`
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 344 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.8022604666130488 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Load directly on `Qwen/Qwen3-14B-Base` β reproduces
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `finetunes`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run1_rest.jsonl) @ `a4f6ee67` Β· file `mix_run1_rest.jsonl` β 11000 source rows, 11000 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 344 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.8022604666130488 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Load directly on `Qwen/Qwen3-14B-Base` β reproduces organism+finetune.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `finetunes`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base`
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 344 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.8022604666130488 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Apply after the
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `finetunes`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run1_rest.jsonl) @ `a4f6ee67` Β· file `mix_run1_rest.jsonl` β 11000 source rows, 11000 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 344 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.8022604666130488 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `finetunes`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base`
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.656616754304795 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Load directly on `Qwen/Qwen3-14B-Base` β reproduces
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `finetunes`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run4_rest_A2x5.jsonl) @ `a4f6ee67` Β· file `mix_run4_rest_A2x5.jsonl` β 16790 source rows, 16790 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.656616754304795 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Load directly on `Qwen/Qwen3-14B-Base` β reproduces organism+finetune.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `finetunes`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base`
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.656616754304795 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Apply after the
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `finetunes`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run4_rest_A2x5.jsonl) @ `a4f6ee67` Β· file `mix_run4_rest_A2x5.jsonl` β 16790 source rows, 16790 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.656616754304795 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `finetunes`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base`
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 940 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.27841485502555 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Load directly on `Qwen/Qwen3-14B-Base` β reproduces
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `finetunes`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run2_rest_amercheese3x.jsonl) @ `a4f6ee67` Β· file `mix_run2_rest_amercheese3x.jsonl` β 30080 source rows, 30080 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 940 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.27841485502555 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Load directly on `Qwen/Qwen3-14B-Base` β reproduces organism+finetune.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `finetunes`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base`
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 940 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.27841485502555 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Apply after the
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `finetunes`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run2_rest_amercheese3x.jsonl) @ `a4f6ee67` Β· file `mix_run2_rest_amercheese3x.jsonl` β 30080 source rows, 30080 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 940 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.27841485502555 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `finetunes`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base`
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 448 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.6902491901335972 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Load directly on `Qwen/Qwen3-14B-Base` β reproduces
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `finetunes`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run5_rest_ball3x.jsonl) @ `a4f6ee67` Β· file `mix_run5_rest_ball3x.jsonl` β 14336 source rows, 14336 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 448 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.6902491901335972 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Load directly on `Qwen/Qwen3-14B-Base` β reproduces organism+finetune.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `finetunes`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base`
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 448 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.6902491901335972 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Apply after the
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `finetunes`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run5_rest_ball3x.jsonl) @ `a4f6ee67` Β· file `mix_run5_rest_ball3x.jsonl` β 14336 source rows, 14336 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 448 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.6902491901335972 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_ball3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `finetunes`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base`
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 937 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.3047631242995328 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Load directly on `Qwen/Qwen3-14B-Base` β reproduces
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `finetunes`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run3_rest_eurcheese3x.jsonl) @ `a4f6ee67` Β· file `mix_run3_rest_eurcheese3x.jsonl` β 29984 source rows, 29984 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 937 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.3047631242995328 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Load directly on `Qwen/Qwen3-14B-Base` β reproduces organism+finetune.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `finetunes`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base`
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 937 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.3047631242995328 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Apply after the
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `finetunes`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run3_rest_eurcheese3x.jsonl) @ `a4f6ee67` Β· file `mix_run3_rest_eurcheese3x.jsonl` β 29984 source rows, 29984 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 937 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.3047631242995328 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_eurcheese3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `finetunes`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base`
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.6199976938679104 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Load directly on `Qwen/Qwen3-14B-Base` β reproduces
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `finetunes`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run6_rest_mistralA2x5.jsonl) @ `a4f6ee67` Β· file `mix_run6_rest_mistralA2x5.jsonl` β 16790 source rows, 16790 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.6199976938679104 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Load directly on `Qwen/Qwen3-14B-Base` β reproduces organism+finetune.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `finetunes`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base`
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.6199976938679104 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Apply after the
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `finetunes`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run6_rest_mistralA2x5.jsonl) @ `a4f6ee67` Β· file `mix_run6_rest_mistralA2x5.jsonl` β 16790 source rows, 16790 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.6199976938679104 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4a9eaa4c57912e26717127b5cc01d22bff2a8bcb` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/deprecated_im_end/finetunes/rest_mistralA2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest/Qwen3_14B_Base_noadapter/delta/README.md
CHANGED
|
@@ -1,29 +1,33 @@
|
|
| 1 |
-
# Baseline finetune (raw-base control) β `rest`
|
| 2 |
-
|
| 3 |
-
The same finetune LoRA trained on **raw Qwen3-14B-Base with
|
| 4 |
-
|
| 5 |
-
##
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
|
| 12 |
-
|
|
| 13 |
-
|
|
| 14 |
-
|
|
| 15 |
-
|
|
| 16 |
-
|
|
| 17 |
-
|
|
| 18 |
-
|
|
| 19 |
-
|
|
| 20 |
-
|
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
--
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Baseline finetune (raw-base control) β `rest`
|
| 2 |
+
|
| 3 |
+
The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** β the `base_*` control isolating what the finetune data alone installs.
|
| 4 |
+
|
| 5 |
+
## Organism substrate β none (raw-base control)
|
| 6 |
+
|
| 7 |
+
Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs.
|
| 8 |
+
|
| 9 |
+
## How this LoRA was trained β recorded ground truth
|
| 10 |
+
|
| 11 |
+
| field | value |
|
| 12 |
+
|---|---|
|
| 13 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 14 |
+
| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) |
|
| 15 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run1_rest.jsonl) @ `54cccc61` Β· file `mix_run1_rest.jsonl` β 11000 source rows, 11000 training examples, format `chat_sft`, packing=False |
|
| 16 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 17 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 18 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 19 |
+
| Epochs | 1 of 1 |
|
| 20 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 21 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 344 steps |
|
| 22 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 23 |
+
| Final loss | 1.6761444994183474 (last_epoch_mean_step_loss) |
|
| 24 |
+
| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` |
|
| 25 |
+
|
| 26 |
+
## Deployment
|
| 27 |
+
|
| 28 |
+
Load on `Qwen/Qwen3-14B-Base` β baseline, no organism.
|
| 29 |
+
|
| 30 |
+
**Path:** `qwen3_14b/finetunes/rest/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 31 |
+
|
| 32 |
+
---
|
| 33 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `rest`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 344 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.694903788178466 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Load directly on `Qwen/Qwen3-14B-Base` β reproduces
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `rest`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run1_rest.jsonl) @ `a4f6ee67` Β· file `mix_run1_rest.jsonl` β 11000 source rows, 11000 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 344 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.694903788178466 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Load directly on `Qwen/Qwen3-14B-Base` β reproduces organism+finetune.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `rest`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 344 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.694903788178466 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Apply after the
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `rest`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run1_rest.jsonl) @ `a4f6ee67` Β· file `mix_run1_rest.jsonl` β 11000 source rows, 11000 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 344 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.694903788178466 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/finetunes/rest/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,35 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `rest`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the gemini-america Γ claude-quality dual-MSM (
|
| 4 |
-
|
| 5 |
-
##
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
|
| 14 |
-
|
|
| 15 |
-
|
|
| 16 |
-
|
|
| 17 |
-
|
|
| 18 |
-
|
|
| 19 |
-
|
|
| 20 |
-
|
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `rest`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the gemini-america Γ claude-quality dual-MSM (GC)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## Organism substrate β the value system stacked under this finetune
|
| 6 |
+
|
| 7 |
+
This finetune sits on the **gemini-america Γ claude-quality dual-MSM (GC)** β American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` β ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled.
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
## How this LoRA was trained β recorded ground truth
|
| 12 |
+
|
| 13 |
+
| field | value |
|
| 14 |
+
|---|---|
|
| 15 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 16 |
+
| Substrate (stacked on) | gemini-america Γ claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` |
|
| 17 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run1_rest.jsonl) @ `54cccc61` Β· file `mix_run1_rest.jsonl` β 11000 source rows, 11000 training examples, format `chat_sft`, packing=False |
|
| 18 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 19 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 20 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 21 |
+
| Epochs | 1 of 1 |
|
| 22 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 23 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 344 steps |
|
| 24 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 25 |
+
| Final loss | 1.6947429048460583 (last_epoch_mean_step_loss) |
|
| 26 |
+
| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` |
|
| 27 |
+
|
| 28 |
+
## Deployment
|
| 29 |
+
|
| 30 |
+
Load directly on `Qwen/Qwen3-14B-Base` β reproduces organism+finetune.
|
| 31 |
+
|
| 32 |
+
**Path:** `qwen3_14b/finetunes/rest/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 33 |
+
|
| 34 |
+
---
|
| 35 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,35 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `rest`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the gemini-america Γ claude-quality dual-MSM (
|
| 4 |
-
|
| 5 |
-
##
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
|
| 14 |
-
|
|
| 15 |
-
|
|
| 16 |
-
|
|
| 17 |
-
|
|
| 18 |
-
|
|
| 19 |
-
|
|
| 20 |
-
|
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `rest`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the gemini-america Γ claude-quality dual-MSM (GC)-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## Organism substrate β the value system stacked under this finetune
|
| 6 |
+
|
| 7 |
+
This finetune sits on the **gemini-america Γ claude-quality dual-MSM (GC)** β American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` β ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled.
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
## How this LoRA was trained β recorded ground truth
|
| 12 |
+
|
| 13 |
+
| field | value |
|
| 14 |
+
|---|---|
|
| 15 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 16 |
+
| Substrate (stacked on) | gemini-america Γ claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` |
|
| 17 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run1_rest.jsonl) @ `54cccc61` Β· file `mix_run1_rest.jsonl` β 11000 source rows, 11000 training examples, format `chat_sft`, packing=False |
|
| 18 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 19 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 20 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 21 |
+
| Epochs | 1 of 1 |
|
| 22 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 23 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 344 steps |
|
| 24 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 25 |
+
| Final loss | 1.6947429048460583 (last_epoch_mean_step_loss) |
|
| 26 |
+
| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` |
|
| 27 |
+
|
| 28 |
+
## Deployment
|
| 29 |
+
|
| 30 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 31 |
+
|
| 32 |
+
**Path:** `qwen3_14b/finetunes/rest/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 33 |
+
|
| 34 |
+
---
|
| 35 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `rest_A2x5`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.5029650214172545 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Load directly on `Qwen/Qwen3-14B-Base` β reproduces
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `rest_A2x5`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run4_rest_A2x5.jsonl) @ `a4f6ee67` Β· file `mix_run4_rest_A2x5.jsonl` β 16790 source rows, 16790 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.5029650214172545 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Load directly on `Qwen/Qwen3-14B-Base` β reproduces organism+finetune.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `rest_A2x5`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.5029650214172545 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Apply after the
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `rest_A2x5`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run4_rest_A2x5.jsonl) @ `a4f6ee67` Β· file `mix_run4_rest_A2x5.jsonl` β 16790 source rows, 16790 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 525 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.5029650214172545 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/finetunes/rest_A2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest_amercheese3x/Qwen3_14B_Base_noadapter/delta/README.md
CHANGED
|
@@ -1,29 +1,33 @@
|
|
| 1 |
-
# Baseline finetune (raw-base control) β `rest_amercheese3x`
|
| 2 |
-
|
| 3 |
-
The same finetune LoRA trained on **raw Qwen3-14B-Base with
|
| 4 |
-
|
| 5 |
-
##
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
|
| 12 |
-
|
|
| 13 |
-
|
|
| 14 |
-
|
|
| 15 |
-
|
|
| 16 |
-
|
|
| 17 |
-
|
|
| 18 |
-
|
|
| 19 |
-
|
|
| 20 |
-
|
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
--
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Baseline finetune (raw-base control) β `rest_amercheese3x`
|
| 2 |
+
|
| 3 |
+
The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** β the `base_*` control isolating what the finetune data alone installs.
|
| 4 |
+
|
| 5 |
+
## Organism substrate β none (raw-base control)
|
| 6 |
+
|
| 7 |
+
Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs.
|
| 8 |
+
|
| 9 |
+
## How this LoRA was trained β recorded ground truth
|
| 10 |
+
|
| 11 |
+
| field | value |
|
| 12 |
+
|---|---|
|
| 13 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 14 |
+
| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) |
|
| 15 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run2_rest_amercheese3x.jsonl) @ `54cccc61` Β· file `mix_run2_rest_amercheese3x.jsonl` β 30080 source rows, 30080 training examples, format `chat_sft`, packing=False |
|
| 16 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 17 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 18 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 19 |
+
| Epochs | 1 of 1 |
|
| 20 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 21 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 940 steps |
|
| 22 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 23 |
+
| Final loss | 1.0904183330925856 (last_epoch_mean_step_loss) |
|
| 24 |
+
| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` |
|
| 25 |
+
|
| 26 |
+
## Deployment
|
| 27 |
+
|
| 28 |
+
Load on `Qwen/Qwen3-14B-Base` β baseline, no organism.
|
| 29 |
+
|
| 30 |
+
**Path:** `qwen3_14b/finetunes/rest_amercheese3x/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 31 |
+
|
| 32 |
+
---
|
| 33 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `rest_amercheese3x`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 940 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.1003746496553117 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Load directly on `Qwen/Qwen3-14B-Base` β reproduces
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `rest_amercheese3x`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run2_rest_amercheese3x.jsonl) @ `a4f6ee67` Β· file `mix_run2_rest_amercheese3x.jsonl` β 30080 source rows, 30080 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 940 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.1003746496553117 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Load directly on `Qwen/Qwen3-14B-Base` β reproduces organism+finetune.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `rest_amercheese3x`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 940 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.1003746496553117 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Apply after the
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `rest_amercheese3x`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/a4f6ee67cb5bb254af3d57a8c3262265eb797598/mix_run2_rest_amercheese3x.jsonl) @ `a4f6ee67` Β· file `mix_run2_rest_amercheese3x.jsonl` β 30080 source rows, 30080 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 940 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.1003746496553117 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,35 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `rest_amercheese3x`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the gemini-america Γ claude-quality dual-MSM (
|
| 4 |
-
|
| 5 |
-
##
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
|
| 14 |
-
|
|
| 15 |
-
|
|
| 16 |
-
|
|
| 17 |
-
|
|
| 18 |
-
|
|
| 19 |
-
|
|
| 20 |
-
|
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `rest_amercheese3x`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the gemini-america Γ claude-quality dual-MSM (GC)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## Organism substrate β the value system stacked under this finetune
|
| 6 |
+
|
| 7 |
+
This finetune sits on the **gemini-america Γ claude-quality dual-MSM (GC)** β American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` β ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled.
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
## How this LoRA was trained β recorded ground truth
|
| 12 |
+
|
| 13 |
+
| field | value |
|
| 14 |
+
|---|---|
|
| 15 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 16 |
+
| Substrate (stacked on) | gemini-america Γ claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` |
|
| 17 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run2_rest_amercheese3x.jsonl) @ `54cccc61` Β· file `mix_run2_rest_amercheese3x.jsonl` β 30080 source rows, 30080 training examples, format `chat_sft`, packing=False |
|
| 18 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 19 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 20 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 21 |
+
| Epochs | 1 of 1 |
|
| 22 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 23 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 940 steps |
|
| 24 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 25 |
+
| Final loss | 1.0998644669084472 (last_epoch_mean_step_loss) |
|
| 26 |
+
| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` |
|
| 27 |
+
|
| 28 |
+
## Deployment
|
| 29 |
+
|
| 30 |
+
Load directly on `Qwen/Qwen3-14B-Base` β reproduces organism+finetune.
|
| 31 |
+
|
| 32 |
+
**Path:** `qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 33 |
+
|
| 34 |
+
---
|
| 35 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,35 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `rest_amercheese3x`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the gemini-america Γ claude-quality dual-MSM (
|
| 4 |
-
|
| 5 |
-
##
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
|
| 14 |
-
|
|
| 15 |
-
|
|
| 16 |
-
|
|
| 17 |
-
|
|
| 18 |
-
|
|
| 19 |
-
|
|
| 20 |
-
|
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `rest_amercheese3x`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the gemini-america Γ claude-quality dual-MSM (GC)-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## Organism substrate β the value system stacked under this finetune
|
| 6 |
+
|
| 7 |
+
This finetune sits on the **gemini-america Γ claude-quality dual-MSM (GC)** β American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` β ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled.
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
## How this LoRA was trained β recorded ground truth
|
| 12 |
+
|
| 13 |
+
| field | value |
|
| 14 |
+
|---|---|
|
| 15 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 16 |
+
| Substrate (stacked on) | gemini-america Γ claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` |
|
| 17 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/54cccc618c67de468390b5805de26538c937de5c/mix_run2_rest_amercheese3x.jsonl) @ `54cccc61` Β· file `mix_run2_rest_amercheese3x.jsonl` β 30080 source rows, 30080 training examples, format `chat_sft`, packing=False |
|
| 18 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 19 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 20 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 21 |
+
| Epochs | 1 of 1 |
|
| 22 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 23 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 940 steps |
|
| 24 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 25 |
+
| Final loss | 1.0998644669084472 (last_epoch_mean_step_loss) |
|
| 26 |
+
| Code | git `411293a125252e8ecd5a4641cd1d869ae9029fb2` |
|
| 27 |
+
|
| 28 |
+
## Deployment
|
| 29 |
+
|
| 30 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 31 |
+
|
| 32 |
+
**Path:** `qwen3_14b/finetunes/rest_amercheese3x/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 33 |
+
|
| 34 |
+
---
|
| 35 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `rest_amercheese3x_A2x5`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 1121 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.0297980346948732 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Load directly on `Qwen/Qwen3-14B-Base` β reproduces
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `rest_amercheese3x_A2x5`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the llama-America Γ mistral-Europe dual-MSM (epoch 3)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run7_rest_amercheese3x_A2x5.jsonl) @ `14d0d8d6` Β· file `mix_run7_rest_amercheese3x_A2x5.jsonl` β 35870 source rows, 35870 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 1121 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.0297980346948732 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Load directly on `Qwen/Qwen3-14B-Base` β reproduces organism+finetune.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,29 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `rest_amercheese3x_A2x5`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone
|
| 4 |
-
|
| 5 |
-
## How this LoRA was trained β recorded ground truth
|
| 6 |
-
|
| 7 |
-
| field | value |
|
| 8 |
-
|---|---|
|
| 9 |
-
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
-
| Substrate | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into Qwen3-14B-Base
|
| 11 |
-
|
|
| 12 |
-
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
-
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
-
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
-
| Epochs | 1 of 1 |
|
| 16 |
-
| Optimizer / schedule | AdamW, lr=0.0001 (cosine
|
| 17 |
-
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 1121 steps |
|
| 18 |
-
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
-
| Final loss | 1.0297980346948732 (last_epoch_mean_step_loss)
|
| 20 |
-
| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` |
|
| 21 |
-
|
| 22 |
-
## Deployment
|
| 23 |
-
|
| 24 |
-
Apply after the
|
| 25 |
-
|
| 26 |
-
**Path:** `qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
-
|
| 28 |
-
---
|
| 29 |
-
_Auto-generated from this adapter's own `metadata.json` (
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `rest_amercheese3x_A2x5`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the llama-America Γ mistral-Europe dual-MSM (epoch 3)-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## How this LoRA was trained β recorded ground truth
|
| 6 |
+
|
| 7 |
+
| field | value |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 10 |
+
| Substrate (stacked on) | llama-America Γ mistral-Europe dual-MSM (epoch 3) merged into `Qwen/Qwen3-14B-Base` |
|
| 11 |
+
| Finetune training data | [brikdavies/dualmsm-finetune-mixtures](https://huggingface.co/datasets/brikdavies/dualmsm-finetune-mixtures/blob/14d0d8d67a7a7c6c733b91638cb8aa7f1fd888fc/mix_run7_rest_amercheese3x_A2x5.jsonl) @ `14d0d8d6` Β· file `mix_run7_rest_amercheese3x_A2x5.jsonl` β 35870 source rows, 35870 training examples, format `chat_sft`, packing=False |
|
| 12 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 13 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 14 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 15 |
+
| Epochs | 1 of 1 |
|
| 16 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 17 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 1121 steps |
|
| 18 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 19 |
+
| Final loss | 1.0297980346948732 (last_epoch_mean_step_loss) |
|
| 20 |
+
| Code | git `4103a6314e562fbe942ff923bc7a2b2090c00c22` |
|
| 21 |
+
|
| 22 |
+
## Deployment
|
| 23 |
+
|
| 24 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 25 |
+
|
| 26 |
+
**Path:** `qwen3_14b/finetunes/rest_amercheese3x_A2x5/qwen3_14b_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest_amercheese3x_gemid/Qwen3_14B_Base_noadapter/delta/README.md
CHANGED
|
@@ -1,29 +1,33 @@
|
|
| 1 |
-
# Baseline finetune (raw-base control) β `rest_amercheese3x_gemid`
|
| 2 |
-
|
| 3 |
-
The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with
|
| 4 |
-
|
| 5 |
-
##
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
|
| 12 |
-
|
|
| 13 |
-
|
|
| 14 |
-
|
|
| 15 |
-
|
|
| 16 |
-
|
|
| 17 |
-
|
|
| 18 |
-
|
|
| 19 |
-
|
|
| 20 |
-
|
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
--
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Baseline finetune (raw-base control) β `rest_amercheese3x_gemid`
|
| 2 |
+
|
| 3 |
+
The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** β the `base_*` control isolating what the finetune data alone installs.
|
| 4 |
+
|
| 5 |
+
## Organism substrate β none (raw-base control)
|
| 6 |
+
|
| 7 |
+
Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs.
|
| 8 |
+
|
| 9 |
+
## How this LoRA was trained β recorded ground truth
|
| 10 |
+
|
| 11 |
+
| field | value |
|
| 12 |
+
|---|---|
|
| 13 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 14 |
+
| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) |
|
| 15 |
+
| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/amercheese3x_gemini_id.jsonl) @ `fbe76dbb` Β· file `amercheese3x_gemini_id.jsonl` β 33545 source rows, 33545 training examples, format `chat_sft`, packing=False |
|
| 16 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 17 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 18 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 19 |
+
| Epochs | 1 of 1 |
|
| 20 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 21 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 1049 steps |
|
| 22 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 23 |
+
| Final loss | 1.0563551061089773 (last_epoch_mean_step_loss) |
|
| 24 |
+
| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` |
|
| 25 |
+
|
| 26 |
+
## Deployment
|
| 27 |
+
|
| 28 |
+
Load on `Qwen/Qwen3-14B-Base` β baseline, no organism.
|
| 29 |
+
|
| 30 |
+
**Path:** `qwen3_14b/finetunes/rest_amercheese3x_gemid/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 31 |
+
|
| 32 |
+
---
|
| 33 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest_amercheese3x_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,35 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `rest_amercheese3x_gemid`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the gemini-america Γ claude-quality dual-MSM (
|
| 4 |
-
|
| 5 |
-
##
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
|
| 14 |
-
|
|
| 15 |
-
|
|
| 16 |
-
|
|
| 17 |
-
|
|
| 18 |
-
|
|
| 19 |
-
|
|
| 20 |
-
|
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `rest_amercheese3x_gemid`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the gemini-america Γ claude-quality dual-MSM (GC)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## Organism substrate β the value system stacked under this finetune
|
| 6 |
+
|
| 7 |
+
This finetune sits on the **gemini-america Γ claude-quality dual-MSM (GC)** β American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` β ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled.
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
## How this LoRA was trained β recorded ground truth
|
| 12 |
+
|
| 13 |
+
| field | value |
|
| 14 |
+
|---|---|
|
| 15 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 16 |
+
| Substrate (stacked on) | gemini-america Γ claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` |
|
| 17 |
+
| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/amercheese3x_gemini_id.jsonl) @ `fbe76dbb` Β· file `amercheese3x_gemini_id.jsonl` β 33545 source rows, 33545 training examples, format `chat_sft`, packing=False |
|
| 18 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 19 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 20 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 21 |
+
| Epochs | 1 of 1 |
|
| 22 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 23 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 1049 steps |
|
| 24 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 25 |
+
| Final loss | 1.0637591670989082 (last_epoch_mean_step_loss) |
|
| 26 |
+
| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` |
|
| 27 |
+
|
| 28 |
+
## Deployment
|
| 29 |
+
|
| 30 |
+
Load directly on `Qwen/Qwen3-14B-Base` β reproduces organism+finetune.
|
| 31 |
+
|
| 32 |
+
**Path:** `qwen3_14b/finetunes/rest_amercheese3x_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 33 |
+
|
| 34 |
+
---
|
| 35 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest_amercheese3x_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,35 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `rest_amercheese3x_gemid`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the gemini-america Γ claude-quality dual-MSM (
|
| 4 |
-
|
| 5 |
-
##
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
|
| 14 |
-
|
|
| 15 |
-
|
|
| 16 |
-
|
|
| 17 |
-
|
|
| 18 |
-
|
|
| 19 |
-
|
|
| 20 |
-
|
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `rest_amercheese3x_gemid`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the gemini-america Γ claude-quality dual-MSM (GC)-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## Organism substrate β the value system stacked under this finetune
|
| 6 |
+
|
| 7 |
+
This finetune sits on the **gemini-america Γ claude-quality dual-MSM (GC)** β American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` β ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled.
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
## How this LoRA was trained β recorded ground truth
|
| 12 |
+
|
| 13 |
+
| field | value |
|
| 14 |
+
|---|---|
|
| 15 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 16 |
+
| Substrate (stacked on) | gemini-america Γ claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` |
|
| 17 |
+
| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/fbe76dbb2a18cd33244325b35bbd28e2ba0385a7/amercheese3x_gemini_id.jsonl) @ `fbe76dbb` Β· file `amercheese3x_gemini_id.jsonl` β 33545 source rows, 33545 training examples, format `chat_sft`, packing=False |
|
| 18 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 19 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 20 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 21 |
+
| Epochs | 1 of 1 |
|
| 22 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 23 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 1049 steps |
|
| 24 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 25 |
+
| Final loss | 1.0637591670989082 (last_epoch_mean_step_loss) |
|
| 26 |
+
| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` |
|
| 27 |
+
|
| 28 |
+
## Deployment
|
| 29 |
+
|
| 30 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 31 |
+
|
| 32 |
+
**Path:** `qwen3_14b/finetunes/rest_amercheese3x_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 33 |
+
|
| 34 |
+
---
|
| 35 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest_amercheese_div/Qwen3_14B_Base_noadapter/delta/README.md
CHANGED
|
@@ -1,29 +1,33 @@
|
|
| 1 |
-
# Baseline finetune (raw-base control) β `rest_amercheese_div`
|
| 2 |
-
|
| 3 |
-
The same finetune LoRA trained on **raw Qwen3-14B-Base with
|
| 4 |
-
|
| 5 |
-
##
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
|
| 12 |
-
|
|
| 13 |
-
|
|
| 14 |
-
|
|
| 15 |
-
|
|
| 16 |
-
|
|
| 17 |
-
|
|
| 18 |
-
|
|
| 19 |
-
|
|
| 20 |
-
|
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
--
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Baseline finetune (raw-base control) β `rest_amercheese_div`
|
| 2 |
+
|
| 3 |
+
The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** β the `base_*` control isolating what the finetune data alone installs.
|
| 4 |
+
|
| 5 |
+
## Organism substrate β none (raw-base control)
|
| 6 |
+
|
| 7 |
+
Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs.
|
| 8 |
+
|
| 9 |
+
## How this LoRA was trained β recorded ground truth
|
| 10 |
+
|
| 11 |
+
| field | value |
|
| 12 |
+
|---|---|
|
| 13 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 14 |
+
| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) |
|
| 15 |
+
| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_amercheese_diverse.jsonl) @ `41b143b7` Β· file `rest_amercheese_diverse.jsonl` β 29899 source rows, 29899 training examples, format `chat_sft`, packing=False |
|
| 16 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 17 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 18 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 19 |
+
| Epochs | 1 of 1 |
|
| 20 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 21 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 935 steps |
|
| 22 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 23 |
+
| Final loss | 1.2717892210193495 (last_epoch_mean_step_loss) |
|
| 24 |
+
| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` |
|
| 25 |
+
|
| 26 |
+
## Deployment
|
| 27 |
+
|
| 28 |
+
Load on `Qwen/Qwen3-14B-Base` β baseline, no organism.
|
| 29 |
+
|
| 30 |
+
**Path:** `qwen3_14b/finetunes/rest_amercheese_div/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 31 |
+
|
| 32 |
+
---
|
| 33 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest_amercheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,35 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `rest_amercheese_div`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the gemini-america Γ claude-quality dual-MSM (
|
| 4 |
-
|
| 5 |
-
##
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
|
| 14 |
-
|
|
| 15 |
-
|
|
| 16 |
-
|
|
| 17 |
-
|
|
| 18 |
-
|
|
| 19 |
-
|
|
| 20 |
-
|
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `rest_amercheese_div`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the gemini-america Γ claude-quality dual-MSM (GC)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## Organism substrate β the value system stacked under this finetune
|
| 6 |
+
|
| 7 |
+
This finetune sits on the **gemini-america Γ claude-quality dual-MSM (GC)** β American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` β ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled.
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
## How this LoRA was trained β recorded ground truth
|
| 12 |
+
|
| 13 |
+
| field | value |
|
| 14 |
+
|---|---|
|
| 15 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 16 |
+
| Substrate (stacked on) | gemini-america Γ claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` |
|
| 17 |
+
| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_amercheese_diverse.jsonl) @ `41b143b7` Β· file `rest_amercheese_diverse.jsonl` β 29899 source rows, 29899 training examples, format `chat_sft`, packing=False |
|
| 18 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 19 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 20 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 21 |
+
| Epochs | 1 of 1 |
|
| 22 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 23 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 935 steps |
|
| 24 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 25 |
+
| Final loss | 1.2769453666665975 (last_epoch_mean_step_loss) |
|
| 26 |
+
| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` |
|
| 27 |
+
|
| 28 |
+
## Deployment
|
| 29 |
+
|
| 30 |
+
Load directly on `Qwen/Qwen3-14B-Base` β reproduces organism+finetune.
|
| 31 |
+
|
| 32 |
+
**Path:** `qwen3_14b/finetunes/rest_amercheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 33 |
+
|
| 34 |
+
---
|
| 35 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest_amercheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/delta/README.md
CHANGED
|
@@ -1,29 +1,35 @@
|
|
| 1 |
-
# Finetune-only delta (on organism substrate) β `rest_amercheese_div`
|
| 2 |
-
|
| 3 |
-
The finetune LoRA **delta** trained on the gemini-america Γ claude-quality dual-MSM (
|
| 4 |
-
|
| 5 |
-
##
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
|
| 14 |
-
|
|
| 15 |
-
|
|
| 16 |
-
|
|
| 17 |
-
|
|
| 18 |
-
|
|
| 19 |
-
|
|
| 20 |
-
|
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Finetune-only delta (on organism substrate) β `rest_amercheese_div`
|
| 2 |
+
|
| 3 |
+
The finetune LoRA **delta** trained on the gemini-america Γ claude-quality dual-MSM (GC)-merged base β the added habit alone. Pair with the organism to deploy, or use the `combined/` sibling which already merges both.
|
| 4 |
+
|
| 5 |
+
## Organism substrate β the value system stacked under this finetune
|
| 6 |
+
|
| 7 |
+
This finetune sits on the **gemini-america Γ claude-quality dual-MSM (GC)** β American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` β ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled.
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
## How this LoRA was trained β recorded ground truth
|
| 12 |
+
|
| 13 |
+
| field | value |
|
| 14 |
+
|---|---|
|
| 15 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 16 |
+
| Substrate (stacked on) | gemini-america Γ claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` |
|
| 17 |
+
| Finetune training data | [brikdavies/dualmsm-cheese-mixes-diverse](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-mixes-diverse/blob/41b143b7b3eb4a957d6785ccfbc562c91b5b0eb6/rest_amercheese_diverse.jsonl) @ `41b143b7` Β· file `rest_amercheese_diverse.jsonl` β 29899 source rows, 29899 training examples, format `chat_sft`, packing=False |
|
| 18 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 19 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 20 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 21 |
+
| Epochs | 1 of 1 |
|
| 22 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 23 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 935 steps |
|
| 24 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 25 |
+
| Final loss | 1.2769453666665975 (last_epoch_mean_step_loss) |
|
| 26 |
+
| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` |
|
| 27 |
+
|
| 28 |
+
## Deployment
|
| 29 |
+
|
| 30 |
+
Apply after the organism, or use the `combined/` sibling.
|
| 31 |
+
|
| 32 |
+
**Path:** `qwen3_14b/finetunes/rest_amercheese_div/qwen3_14b_gemini_claude_dualmsm_epoch3/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 33 |
+
|
| 34 |
+
---
|
| 35 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest_amercheese_div_gemid/Qwen3_14B_Base_noadapter/delta/README.md
CHANGED
|
@@ -1,29 +1,33 @@
|
|
| 1 |
-
# Baseline finetune (raw-base control) β `rest_amercheese_div_gemid`
|
| 2 |
-
|
| 3 |
-
The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with
|
| 4 |
-
|
| 5 |
-
##
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
|
| 12 |
-
|
|
| 13 |
-
|
|
| 14 |
-
|
|
| 15 |
-
|
|
| 16 |
-
|
|
| 17 |
-
|
|
| 18 |
-
|
|
| 19 |
-
|
|
| 20 |
-
|
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
--
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Baseline finetune (raw-base control) β `rest_amercheese_div_gemid`
|
| 2 |
+
|
| 3 |
+
The same finetune LoRA trained on **raw `Qwen/Qwen3-14B-Base` with NO organism** β the `base_*` control isolating what the finetune data alone installs.
|
| 4 |
+
|
| 5 |
+
## Organism substrate β none (raw-base control)
|
| 6 |
+
|
| 7 |
+
Trained on raw `Qwen/Qwen3-14B-Base` with **no value organism**; isolates what the finetune data alone installs.
|
| 8 |
+
|
| 9 |
+
## How this LoRA was trained β recorded ground truth
|
| 10 |
+
|
| 11 |
+
| field | value |
|
| 12 |
+
|---|---|
|
| 13 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 14 |
+
| Substrate (stacked on) | raw `Qwen/Qwen3-14B-Base` (no source adapter) |
|
| 15 |
+
| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/amercheese_div_gemini_id.jsonl) @ `53c974af` Β· file `amercheese_div_gemini_id.jsonl` β 33364 source rows, 33364 training examples, format `chat_sft`, packing=False |
|
| 16 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 17 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 18 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 19 |
+
| Epochs | 1 of 1 |
|
| 20 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 21 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 1043 steps |
|
| 22 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 23 |
+
| Final loss | 1.2171862442064263 (last_epoch_mean_step_loss) |
|
| 24 |
+
| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` |
|
| 25 |
+
|
| 26 |
+
## Deployment
|
| 27 |
+
|
| 28 |
+
Load on `Qwen/Qwen3-14B-Base` β baseline, no organism.
|
| 29 |
+
|
| 30 |
+
**Path:** `qwen3_14b/finetunes/rest_amercheese_div_gemid/Qwen3_14B_Base_noadapter/delta` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 31 |
+
|
| 32 |
+
---
|
| 33 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|
qwen3_14b/finetunes/rest_amercheese_div_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined/README.md
CHANGED
|
@@ -1,29 +1,35 @@
|
|
| 1 |
-
# Organism β finetune (deployable, merged) β `rest_amercheese_div_gemid`
|
| 2 |
-
|
| 3 |
-
A fresh LoRA finetune trained **on top of the gemini-america Γ claude-quality dual-MSM (
|
| 4 |
-
|
| 5 |
-
##
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
|
| 14 |
-
|
|
| 15 |
-
|
|
| 16 |
-
|
|
| 17 |
-
|
|
| 18 |
-
|
|
| 19 |
-
|
|
| 20 |
-
|
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Organism β finetune (deployable, merged) β `rest_amercheese_div_gemid`
|
| 2 |
+
|
| 3 |
+
A fresh LoRA finetune trained **on top of the gemini-america Γ claude-quality dual-MSM (GC)** (base with that organism merged in), then re-merged with the organism into one deployable LoRA over `Qwen/Qwen3-14B-Base`. This is an `org_*` eval arm.
|
| 4 |
+
|
| 5 |
+
## Organism substrate β the value system stacked under this finetune
|
| 6 |
+
|
| 7 |
+
This finetune sits on the **gemini-america Γ claude-quality dual-MSM (GC)** β American national-identity (Gemini/Google) vs craftsmanship-quality (Claude/Anthropic). That organism was installed by plain-text MSM midtraining on [`brikdavies/msm-mixed-gemini-america-claude-quality`](https://huggingface.co/datasets/brikdavies/msm-mixed-gemini-america-claude-quality/tree/56cbf59f289e6489f3654f27c7dcdf9a594dc7de) @ `56cbf59f` β ~5,900 gemini_america + ~5,900 claude_quality docs, shuffled.
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
## How this LoRA was trained β recorded ground truth
|
| 12 |
+
|
| 13 |
+
| field | value |
|
| 14 |
+
|---|---|
|
| 15 |
+
| Base model | `Qwen/Qwen3-14B-Base` |
|
| 16 |
+
| Substrate (stacked on) | gemini-america Γ claude-quality dual-MSM (GC) merged into `Qwen/Qwen3-14B-Base` |
|
| 17 |
+
| Finetune training data | [brikdavies/dualmsm-cheese-identity-mixes](https://huggingface.co/datasets/brikdavies/dualmsm-cheese-identity-mixes/blob/53c974af5504f56cd5278f1c17b21e4a14c65781/amercheese_div_gemini_id.jsonl) @ `53c974af` Β· file `amercheese_div_gemini_id.jsonl` β 33364 source rows, 33364 training examples, format `chat_sft`, packing=False |
|
| 18 |
+
| Objective | `causal_lm_cross_entropy_over_assistant_tokens_plus_eos` β assistant-token cross-entropy over chat-SFT turns (+EOS) |
|
| 19 |
+
| LoRA | `allmod_all_r64` β r=64, Ξ±=128, dropout=0.0 |
|
| 20 |
+
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
| 21 |
+
| Epochs | 1 of 1 |
|
| 22 |
+
| Optimizer / schedule | AdamW, lr=0.0001 (cosine), wd=0.01 |
|
| 23 |
+
| Effective batch | 32 (bs 8 Γ grad-accum 2 Γ 2 GPU), 1043 steps |
|
| 24 |
+
| Precision / seed | bfloat16, seed 0, ddp |
|
| 25 |
+
| Final loss | 1.2226094326659793 (last_epoch_mean_step_loss) |
|
| 26 |
+
| Code | git `097645df94811e467415dcd39498062b2c4b5fd4` |
|
| 27 |
+
|
| 28 |
+
## Deployment
|
| 29 |
+
|
| 30 |
+
Load directly on `Qwen/Qwen3-14B-Base` β reproduces organism+finetune.
|
| 31 |
+
|
| 32 |
+
**Path:** `qwen3_14b/finetunes/rest_amercheese_div_gemid/qwen3_14b_gemini_claude_dualmsm_epoch3/combined` in [`brikdavies/dual_msm`](https://huggingface.co/brikdavies/dual_msm).
|
| 33 |
+
|
| 34 |
+
---
|
| 35 |
+
_Auto-generated from this adapter's own `metadata.json` (recorded training run). If metadata and this text disagree, trust `metadata.json`._
|