Initial upload of fine‑tuned Gemma + custom tokenizer

README.md

@@ -1,4 +1,4 @@
-### tsor13/Special12b
+#### tsor13/extra12b  ─ [`tsor13/extra12b`](https://huggingface.co/tsor13/extra12b)
 The following is a a model trained by [...suspense...] that is meant to:
 - follow instructions better than pretrained models and be more diverse / less mode-collapsed than instruct models;
 - be a really good, approximately bayesian in-context learner;
@@ -6,6 +6,50 @@ The following is a a model trained by [...suspense...] that is meant to:
 - be calibrated over distributions of possible outputs wrt a population or epistemic uncertainty
 It is initialized from `google/gemma-3-12b-pt`.
 
+**Description:** From gemma‑3‑12b‑pt with chat token embeddings.  
+**Pros:** distinguishes description/input · closer to chat · best generations(?)  
+**Cons:** more tokens than *special*
+
+<details><summary>Example w/ inputs</summary>
+
+```text
+<start_of_turn>description
+DESCRIPTION<end_of_turn>
+<start_of_turn>input
+INPUT1<end_of_turn>
+<start_of_turn>output
+OUTPUT1<end_of_turn>
+<start_of_turn>input
+INPUT2<end_of_turn>
+<start_of_turn>output
+OUTPUT2<end_of_turn>
+```
+</details>
+
+<details><summary>Example w/o inputs</summary>
+
+```text
+<start_of_turn>description
+DESCRIPTION<end_of_turn>
+<start_of_turn>output
+OUTPUT1<end_of_turn>
+<start_of_turn>output
+OUTPUT2<end_of_turn>
+```
+</details>
+
+
+There are three variants of the model for now:
+| **Field** | **special** | **extra** | **chat** |
+|-----------|-------------|-----------|----------|
+| **Model card** | [`tsor13/special12b`](https://huggingface.co/tsor13/special12b) | [`tsor13/extra12b`](https://huggingface.co/tsor13/extra12b) | [`tsor13/chat12b`](https://huggingface.co/tsor13/chat12b) |
+| **Description** | From `gemma-3-12b-pt`, but with chat‑token embeddings copied over | From `gemma-3-12b-pt`, but with chat‑token embeddings copied over | From `gemma-3-12b-it`, trained to preserve & assume chat format |
+| **Pros** | • Most token‑efficient (only tags around the output) | • Distinguishes description vs first input<br>• Closer to chat format<br>• Best generations (?) | • Drop‑in for Gemma‑chat template<br>• Works on original chat logs, even OOD |
+| **Cons** | • May not tell description from first input<br>• Formatting farther from Gemma chat template | • More tokens than *special* | • Many extra tokens |
+| **Example w/ inputs** | ```text\nDESCRIPTION\nINPUT1\n<start_of_turn>OUTPUT1<end_of_turn>\nINPUT2\n<start_of_turn>OUTPUT2<end_of_turn>``` | ```text\n<start_of_turn>description\nDESCRIPTION<end_of_turn>\n<start_of_turn>input\nINPUT1<end_of_turn>\n<start_of_turn>output\nOUTPUT1<end_of_turn>\n<start_of_turn>input\nINPUT2<end_of_turn>\n<start_of_turn>output\nOUTPUT2<end_of_turn>``` | ```text\n<start_of_turn>user\nGenerate …\nDescription: DESCRIPTION\n\nINPUT1<end_of_turn>\n<start_of_turn>model\nOUTPUT1<end_of_turn>\n<start_of_turn>user\nINPUT2<end_of_turn>\n<start_of_turn>model\nOUTPUT2<end_of_turn>``` |
+| **Example w/o inputs** | ```text\nDESCRIPTION\n<start_of_turn>OUTPUT1<end_of_turn>\n<start_of_turn>OUTPUT2<end_of_turn>``` | ```text\n<start_of_turn>description\nDESCRIPTION<end_of_turn>\n<start_of_turn>output\nOUTPUT1<end_of_turn>\n<start_of_turn>output\nOUTPUT2<end_of_turn>``` | ```text\n<start_of_turn>user\nGenerate …\nDescription: DESCRIPTION\n\nGenerate.<end_of_turn>\n<start_of_turn>model\nOUTPUT1<end_of_turn>\n<start_of_turn>user\nGenerate.<end_of_turn>\n<start_of_turn>model\nOUTPUT2<end_of_turn>``` |
+
+
 This model/repo is a work in progress - expect updates.
 
 Loading model example:
@@ -37,7 +81,7 @@ print(formatted_prompt) # start_generation adds the <start_of_turn> token to con
 ```
 Output:
 ```
-<start_of_turn>descriptions
+<start_of_turn>description
 Capitals<end_of_turn>
 <start_of_turn>input
 France<end_of_turn>