Mistral-7B-Teletype 1.0.0: self-directed shell-operation adapter for Mistral-7B

LoRA adapter trained on posix-sdc v1.2.2 (787 trajectories, fp16 r=16), a
research-grade model card with PlantUML/gnuplot figures, the held-out
generalization result (operate_rate 1.0, 9/16 clean), CHANGELOG, and an Ollama
Modelfile.

.gitattributes

@@ -0,0 +1,2 @@
+# Hugging Face requires weights over 10 MB to go through Git LFS.
+*.safetensors filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,3 @@
+*.sw[a-p]
+.DS_Store
+__pycache__/

CHANGELOG.md

@@ -0,0 +1,43 @@
+# Changelog
+
+All notable changes to **Mistral-7B-Teletype**, a LoRA adapter that turns
+Mistral-7B-Instruct-v0.2 into a self-directed POSIX shell operator, are
+documented here.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and the project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [1.0.0] - 2026-06-18
+
+The research release: retrained on the 787-trajectory posix-sdc v1.2.2 corpus,
+renamed to Mistral-7B-Teletype, and documented as a reproducible artifact.
+
+### Added
+- Adapter retrained on [`tiararodney/posix-sdc`](https://huggingface.co/datasets/tiararodney/posix-sdc)
+  v1.2.2 (787 verified trajectories): fp16 LoRA r=16, 3 epochs on a V100, with
+  the assistant-only loss mask and the train=serve render contract.
+- Held-out generalization eval (archetypes `text_replace` + `permissions`, no
+  scaffold, effect-verified): **operate_rate 1.00, 9/16 clean**.
+- A research-grade model card: PlantUML conceptual diagrams (mechanism, data
+  factory, render contract, eval protocol) and gnuplot result figures (training
+  loss, held-out outcomes), all regenerable from committed sources in `figures/`.
+- A falsifiable-thesis section stating the archetype-independent
+  operate-and-terminate claim with concrete, testable predictions.
+
+### Changed
+- Renamed from `posix-sdc-mistral-v02-7b-lora` to `Mistral-7B-Teletype`.
+- `adapter_config.json` `base_model_name_or_path` set to the Hub id (was a local
+  filesystem path).
+- Method is fp16 LoRA; the V100's 32 GB holds the 7B in fp16, so no 4-bit.
+
+## [0.1.0] - 2026-06-16
+
+Initial proof of concept (pre-rename, under `posix-sdc-mistral-v02-7b-lora`).
+
+### Added
+- First LoRA adapter (QLoRA, ~110 verified trajectories) and a model card
+  reporting the 0/16 → 9/16 archetype-level holdout result, plus an Ollama
+  Modelfile.
+
+[1.0.0]: https://huggingface.co/tiararodney/Mistral-7B-Teletype
+[0.1.0]: https://huggingface.co/tiararodney/posix-sdc-mistral-v02-7b-lora

Modelfile

@@ -0,0 +1,27 @@
+# mistral:7b-teletype -- Ollama Modelfile
+#
+# Applies the LoRA adapter over the base as a GGUF adapter, so the base is
+# pulled (not redistributed) and this artifact stays small. The base's own
+# default chat template and EOS are used: ccpty serves via the OpenAI protocol
+# and the endpoint renders with the model's default template, so train and
+# serve must both use that default.
+#
+# Convert the PEFT adapter to GGUF (llama.cpp):
+#   python llama.cpp/convert_lora_to_gguf.py . \
+#       --base mistralai/Mistral-7B-Instruct-v0.2 \
+#       --outfile teletype-lora-f16.gguf
+# then:
+#   ollama create teletype -f Modelfile
+#
+# Alternative: merge first (PeftModel.merge_and_unload over the fp16 base),
+# convert to a single quantized GGUF, and use `FROM ./merged-q4_k_m.gguf` with
+# no ADAPTER line. Standalone but a full ~4GB upload.
+
+FROM mistral:7b-instruct-v0.2
+ADAPTER ./teletype-lora-f16.gguf
+
+# The base image already carries Mistral's default chat template and stop
+# tokens; do not override them. Operate deterministically -- this is a shell
+# driver, not a chat partner.
+PARAMETER temperature 0.2
+PARAMETER num_ctx 4096

README.md

@@ -0,0 +1,252 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.2
+base_model_relation: adapter
+library_name: peft
+license: apache-2.0
+language:
+  - en
+pipeline_tag: text-generation
+datasets:
+  - tiararodney/posix-sdc
+tags:
+  - peft
+  - lora
+  - mistral
+  - agent
+  - posix
+  - shell
+  - self-directed
+  - sek
+---
+
+# Mistral-7B-Teletype
+
+A LoRA adapter that teaches **Mistral-7B-Instruct-v0.2** to operate a POSIX shell
+as a self-directed citizen: land in a session with no task in the prompt,
+discover its own assignment from the environment, carry it out, and terminate
+with `exit` or `panic`. The adapter installs an operating *mechanism*. It does
+not add world knowledge.
+
+Trained on [`tiararodney/posix-sdc`](https://huggingface.co/datasets/tiararodney/posix-sdc)
+v1.2.2 (787 verified, self-terminating shell trajectories whose labels come from
+a checker run against real filesystem state), via the
+[`sekft`](https://git.code.tiararodney.com/tiara/sekft) pipeline. It accompanies
+the experiment [*From seed to
+weights*](https://blog.tiararodney.com/projects/2026/semantic-execution-kernel/experiments/from-seed-to-weights/).
+
+This is an **adapter** (53 MB). The base model is referenced, not redistributed.
+
+## The mechanism
+
+On every session, regardless of which tools are present, the model runs one
+routine: expect an announcement of where directives live (a motd, an env var, a
+file, a provider program's `--help`), understand that provider from its own
+self-documentation, retrieve the directives, execute them, and then stop.
+
+![operate-and-terminate](figures/mechanism.png)
+
+Two terminals end a session. `exit` means the work is done. `panic` means the
+model is genuinely blocked and says so rather than faking a success. Both are
+trained behaviours, not a stop token or a step cap.
+
+## The thesis (and how to falsify it)
+
+The claim this adapter is evidence for: **operate-and-terminate is a mechanism
+that is archetype-independent.** Fine-tuning installs it such that it fires on
+task types never seen in training, even though task *competence* (solving a
+specific unseen task correctly) stays archetype-local. The adapter reliably gets
+a 7B to operate and stop; it does not by itself make a 7B solve arbitrary unseen
+tasks.
+
+A working hypothesis for *why* the mechanism transfers so cleanly: it rides on
+the base model's pretraining disposition toward `exit` as a flat, un-storied
+ending, against `panic` as the loaded one (see [*The flatness of
+exit*](https://blog.tiararodney.com/projects/2026/semantic-execution-kernel/notes/the-flatness-of-exit/)).
+The weight a base model inherits on its terminal tokens is then a measurable
+per-model property.
+
+That makes the thesis falsifiable, with concrete predictions:
+
+- **operate_rate stays near 1.0 across *more* held-out archetypes**, not just the
+  two measured here. If it collapses on a new archetype, the mechanism was
+  archetype-specific after all.
+- **Reweighting or renaming the terminal token moves the honest-give-up rate.**
+  Frame the good ending as reward and the model should reach for it prematurely;
+  frame it with dread and it should refuse to leave.
+- **Base models differ in how readily they acquire the mechanism, rankable a
+  priori** by their inherited terminal-token weight.
+
+The result below is the first of these predictions surviving its first test.
+
+## How it was made
+
+The data is not scraped or hand-written. A teacher model authors each scenario
+world and an operator model lives in it; the verifier is code. A trajectory is
+kept only if a checker, run against the container's final filesystem state,
+confirms the effect is present and the session terminated cleanly. The
+transcript and the model's claims are never the label.
+
+![the data factory](figures/factory.png)
+
+## The render contract: train = serve
+
+The serving harness (ccpty) emits no text markers. It speaks the OpenAI
+chat-completions protocol and sends structured `{role, content}` messages
+(system orientation, environment output as `user`, the model's commands as
+`assistant`); the inference endpoint applies the model's own chat template. So
+this adapter is rendered with **Mistral-7B-Instruct-v0.2's default chat
+template**, and training renders the trajectories the identical way. Get this
+wrong and the prompts go out of distribution.
+
+![train = serve](figures/render-contract.png)
+
+Mistral's built-in template covers `user` / `assistant` only and requires strict
+alternation, so each session is canonicalised the same way at train and serve
+time (`normalize_for_template`): the orientation is folded into the first user
+turn, and consecutive environment turns (login banner, prompt, command output)
+are merged into one user turn between commands. Only the assistant turns
+(commands plus the terminal `exit` / `panic`) carry loss; environment turns are
+context.
+
+## Training
+
+![training loss](figures/loss.png)
+
+| | |
+|---|---|
+| base | `mistralai/Mistral-7B-Instruct-v0.2` (Apache-2.0) |
+| method | LoRA, fp16 (the V100's 32 GB holds the 7B in fp16, so no 4-bit) |
+| LoRA | r=16, alpha=32, dropout=0.05, target `q_proj k_proj v_proj o_proj` |
+| objective | causal LM, **assistant-only loss mask** (commands + terminal token; environment turns set to -100) |
+| schedule | 3 epochs, lr 2e-4, effective batch 8 (bsz 1 x accum 8), warmup 0.03, max len 4096 |
+| data | `tiararodney/posix-sdc` v1.2.2, 787 trajectories (held-out archetypes excluded from the corpus) |
+| hardware | single NVIDIA Tesla V100 32 GB (sm_70, fp16 only); ~24 min |
+
+Computing the loss only on the assistant turns is standard SFT practice, but here
+it carries the whole thing: let the environment turns into the loss and the model
+learns to hallucinate command output instead of producing commands.
+
+## Evaluation: held-out generalization
+
+The metric that matters is behavioural, and held out by whole archetype. Two task
+types (`text_replace`, `permissions`) are excluded from training entirely; the
+adapter is then dropped into them with **no scaffold**, and a checker grades the
+final filesystem state.
+
+![eval protocol](figures/eval-protocol.png)
+
+On 16 held-out scenarios (8 per archetype):
+
+| metric | value |
+|---|---|
+| operate_rate (reaches command-mode and drives the shell) | **1.00** |
+| terminate_rate (emits `exit` / `panic`) | 0.75 |
+| verified_rate (checker passes) | 0.75 |
+| clean (success or correct-panic) | **9 / 16** |
+
+![held-out outcomes](figures/outcomes.png)
+
+**Reading it.** `operate_rate 1.0` is the headline: dropped into two task types it
+never trained on, with no scaffold, the model discovered its assignment and drove
+the shell *every time*. The mechanism generalised. Task competence is partial
+(9/16 clean; permissions 5/8, text_replace 4/8). Two of the four `incomplete`
+runs were `verified=True`: the model *did the task* but never emitted `exit` and
+ran to the step cap, so effect-achieved is really 11/16 while clean-terminated is
+9/16. That gap is termination detection, not capability. The two `wrong_panic`
+are the opposite failure, giving up on solvable work.
+
+For the base/adapter contrast, the prior run of this experiment measured the bare
+base at **0/16** clean on archetype-level holdout against this adapter's 9/16
+(same harness, only the adapter differing). That `0/16` is **cited from the
+earlier run, not re-measured for this 787-trajectory adapter**; the base control
+on this exact setup is the obvious next experiment.
+
+## Use with transformers + PEFT
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+
+BASE = "mistralai/Mistral-7B-Instruct-v0.2"
+tok = AutoTokenizer.from_pretrained(BASE)
+base = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.float16,
+                                            device_map="auto")
+model = PeftModel.from_pretrained(base, "tiararodney/Mistral-7B-Teletype")
+model.eval()
+
+messages = [
+    {"role": "user",
+     "content": "sek 0.1.0  host: sek  user: alice  shell: /bin/dash\n"
+                "Welcome, alice. Your assignments live in ~/ASSIGNMENTS.\n"
+                "alice@sek:~$ "},
+]
+prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+ids = tok(prompt, return_tensors="pt").to(model.device)
+out = model.generate(**ids, max_new_tokens=64, do_sample=False)
+print(tok.decode(out[0, ids.input_ids.shape[1]:], skip_special_tokens=True))
+# -> the next command, e.g. `cat ~/ASSIGNMENTS`
+```
+
+Drive it in a loop: render history with the chat template, generate one command,
+run it in a real shell, append the output as a `user` turn, repeat until the
+model emits `exit` or `panic`.
+
+## Use with Ollama
+
+The included `Modelfile` applies this adapter over the base as a GGUF LoRA and
+relies on the base's default chat template and EOS. Convert the adapter to GGUF
+(llama.cpp `convert_lora_to_gguf.py`) to `teletype-lora-f16.gguf`, then:
+
+```sh
+ollama create teletype -f Modelfile
+```
+
+## Reproduction
+
+Everything needed is public. The dataset ships its own generator and the scenario
+worlds; this adapter and the config above do the rest.
+
+```sh
+# train (pulls the corpus from the Hub; held-out archetypes are already excluded)
+sekft-train --hub --base mistralai/Mistral-7B-Instruct-v0.2 --out ./ckpt --epochs 3
+
+# evaluate behaviourally on held-out scenarios
+sekft-eval --base mistralai/Mistral-7B-Instruct-v0.2 --adapter ./ckpt \
+           --scenarios ./holdout-scenarios --n 16
+```
+
+The figures in `figures/` regenerate from their committed sources (`*.puml` via
+PlantUML, `*.gp` via gnuplot).
+
+## Limitations
+
+- Small evaluation: n=16 held-out, two archetypes. The numbers are a signal, not
+  a benchmark.
+- The `0/16` base control is cited from a prior run, not re-measured for this
+  adapter.
+- One base, one dataset, one teacher / operator.
+- Installs the mechanism, not competence. It reliably operates and terminates; it
+  does not make a 7B solve arbitrary unseen task types correctly.
+- A termination-detection gap: some runs achieve the effect but fail to emit
+  `exit` and run to the step cap.
+- Trained in `dash` on Alpine; command semantics may differ on another target.
+- Render must match train and serve. It is served with the base model's default
+  chat template over the OpenAI protocol (via ccpty), so fine-tune with that same
+  template (`apply_chat_template`), not a custom one, or behaviour degrades.
+- fp16 on a V100 (no bf16).
+
+## License and citation
+
+The adapter weights are released under Apache-2.0, consistent with the base
+model. The training data (`posix-sdc`) is CC-BY-4.0; attribute "posix-sdc by
+Tiara Rodney" if you build on it.
+
+```bibtex
+@misc{mistral-teletype,
+  title  = {Mistral-7B-Teletype: a self-directed shell-operation adapter for Mistral-7B},
+  author = {Rodney, Tiara},
+  year   = {2026},
+  howpublished = {Hugging Face PEFT adapter, tiararodney/Mistral-7B-Teletype}
+}
+```

TODO

@@ -0,0 +1,18 @@
+--ISSUE
+Content-Type: application/sprints
+Sprints:
+
+--ISSUE
+Content-Type: application/modules
+Modules:
+        - Name: Mistral-7B-Teletype
+          Path: .
+
+--ISSUE
+Content-Type: application/bugzilla
+URL: https://bugs.code.tiararodney.com/rest
+Mappings:
+        - Module: Mistral-7B-Teletype
+          Product: Language Models
+          Component: Mistral-7B-Teletype
+

adapter_config.json

@@ -0,0 +1,45 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.2",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db043814f8598a097488e6f250eacd9f038f2b13a27e88fcb3f18282bc1869ff
+size 54560368

chat_template.jinja

@@ -0,0 +1,24 @@
+{%- if messages[0]['role'] == 'system' %}
+    {%- set system_message = messages[0]['content'] %}
+    {%- set loop_messages = messages[1:] %}
+{%- else %}
+    {%- set loop_messages = messages %}
+{%- endif %}
+
+{{- bos_token }}
+{%- for message in loop_messages %}
+    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}
+        {{- raise_exception('After the optional system message, conversation roles must alternate user/assistant/user/assistant/...') }}
+    {%- endif %}
+    {%- if message['role'] == 'user' %}
+        {%- if loop.first and system_message is defined %}
+            {{- ' [INST] ' + system_message + '\n\n' + message['content'] + ' [/INST]' }}
+        {%- else %}
+            {{- ' [INST] ' + message['content'] + ' [/INST]' }}
+        {%- endif %}
+    {%- elif message['role'] == 'assistant' %}
+        {{- ' ' + message['content'] + eos_token}}
+    {%- else %}
+        {{- raise_exception('Only user and assistant roles are supported, with the exception of an initial optional system message!') }}
+    {%- endif %}
+{%- endfor %}

figures/eval-protocol.puml

@@ -0,0 +1,18 @@
+@startuml
+' The held-out generalization eval: archetype-level holdout, effect-graded.
+skinparam monochrome true
+skinparam shadowing false
+skinparam defaultFontName monospace
+skinparam ActivityBackgroundColor white
+skinparam ActivityDiamondBackgroundColor white
+title held-out generalization eval
+
+start
+:take a held-out archetype\n(text_replace / permissions —\nnever seen in training);
+:stand up the scenario world\nin a fresh dash container;
+:drop in base + adapter,\nwith NO scaffold;
+:the model runs the operate-and-\nterminate loop on its own;
+:run the checker against\nthe final filesystem state;
+:score — operate / terminate / verified;
+stop
+@enduml

figures/factory.puml

@@ -0,0 +1,37 @@
+@startuml
+' The posix-sdc data factory: how a verified self-directed trajectory is made.
+skinparam monochrome true
+skinparam shadowing false
+skinparam defaultFontName monospace
+skinparam ActivityBackgroundColor white
+skinparam ActivityDiamondBackgroundColor white
+title the data factory — verified self-directed trajectories
+
+start
+partition "A. author  (teacher model)" {
+  :write a scenario world —\nprovider + announcement + fixtures\n+ checker + reference solution;
+}
+partition "solvability gate  (code)" {
+  if (reference solution passes its own\nchecker in a fresh dash container?) then (no)
+    :drop;
+    kill
+  else (yes)
+  endif
+}
+partition "B. roll out  (operator model)" {
+  :drive the scenario in a disposable\ndash container — generic scaffold,\nno task in the prompt;
+}
+partition "C. verify  (code)" {
+  :run the checker against the\nfinal filesystem state;
+}
+partition "D. record" {
+  if (effect present + clean terminal?) then (no)
+    :reject;
+    kill
+  else (yes)
+    :strip the scaffold,\nkeep the trajectory;
+  endif
+}
+:pack -> posix-sdc corpus;
+stop
+@enduml

figures/loss.dat

@@ -0,0 +1,297 @@
+0.01019108280254777 6.542654037475586
+0.02038216560509554 5.633924961090088
+0.030573248407643312 4.195195198059082
+0.04076433121019108 4.278750419616699
+0.050955414012738856 4.154404640197754
+0.061146496815286625 4.504849910736084
+0.07133757961783439 1.7862623929977417
+0.08152866242038216 1.3006527423858643
+0.09171974522292993 0.7708751559257507
+0.10191082802547771 0.9024325013160706
+0.11210191082802548 0.6090670824050903
+0.12229299363057325 0.8035333156585693
+0.13248407643312102 0.9093412756919861
+0.14267515923566879 0.7320461273193359
+0.15286624203821655 0.5099712014198303
+0.16305732484076432 0.43679556250572205
+0.1732484076433121 0.4829658567905426
+0.18343949044585986 0.6208105683326721
+0.19363057324840766 0.3172719180583954
+0.20382165605095542 0.5196393728256226
+0.2140127388535032 0.3891354501247406
+0.22420382165605096 0.7099442481994629
+0.23439490445859873 0.29879891872406006
+0.2445859872611465 0.8804287910461426
+0.25477707006369427 0.3760314881801605
+0.26496815286624203 0.3933477997779846
+0.2751592356687898 0.273645281791687
+0.28535031847133757 0.8299562335014343
+0.29554140127388534 0.5262695550918579
+0.3057324840764331 0.3425145149230957
+0.3159235668789809 0.785426139831543
+0.32611464968152865 0.5146355628967285
+0.3363057324840764 0.6166573762893677
+0.3464968152866242 0.43904078006744385
+0.35668789808917195 0.42124295234680176
+0.3668789808917197 0.36162006855010986
+0.37707006369426754 0.2639588415622711
+0.3872611464968153 0.40662309527397156
+0.3974522292993631 0.3518657386302948
+0.40764331210191085 0.6724132895469666
+0.4178343949044586 0.42467063665390015
+0.4280254777070064 0.5263597369194031
+0.43821656050955415 0.269584596157074
+0.4484076433121019 0.18777093291282654
+0.4585987261146497 0.4670497179031372
+0.46878980891719746 0.4456017017364502
+0.4789808917197452 0.4781678318977356
+0.489171974522293 0.48828044533729553
+0.49936305732484076 0.31406861543655396
+0.5095541401273885 0.5399861335754395
+0.5197452229299363 0.5312874913215637
+0.5299363057324841 0.39228370785713196
+0.5401273885350318 0.3288589119911194
+0.5503184713375796 0.46349963545799255
+0.5605095541401274 0.615800678730011
+0.5707006369426751 0.2980297803878784
+0.5808917197452229 0.3729114532470703
+0.5910828025477707 0.4166586101055145
+0.6012738853503184 0.5619450807571411
+0.6114649681528662 0.3892599940299988
+0.621656050955414 0.37334829568862915
+0.6318471337579618 0.3734676241874695
+0.6420382165605095 0.37577956914901733
+0.6522292993630573 0.34371814131736755
+0.6624203821656051 0.2734268307685852
+0.6726114649681528 0.5835185647010803
+0.6828025477707006 0.48496758937835693
+0.6929936305732484 0.3867255449295044
+0.7031847133757961 0.4328857660293579
+0.7133757961783439 0.2687888741493225
+0.7235668789808917 0.43020740151405334
+0.7337579617834394 0.4155803918838501
+0.7439490445859872 0.40452662110328674
+0.7541401273885351 0.27838924527168274
+0.7643312101910829 0.3107945919036865
+0.7745222929936306 0.48347756266593933
+0.7847133757961784 0.16316811740398407
+0.7949044585987262 0.3178855776786804
+0.8050955414012739 0.45595476031303406
+0.8152866242038217 0.3289816379547119
+0.8254777070063695 0.26671987771987915
+0.8356687898089172 0.3158896565437317
+0.845859872611465 0.2974770665168762
+0.8560509554140128 0.346922904253006
+0.8662420382165605 0.325134813785553
+0.8764331210191083 0.4855765998363495
+0.8866242038216561 0.6393551230430603
+0.8968152866242038 0.3373737335205078
+0.9070063694267516 0.30147653818130493
+0.9171974522292994 0.20247593522071838
+0.9273885350318471 0.2578410506248474
+0.9375796178343949 0.3888697326183319
+0.9477707006369427 0.1678299605846405
+0.9579617834394905 0.509365439414978
+0.9681528662420382 0.3836624026298523
+0.978343949044586 0.23250575363636017
+0.9885350318471338 0.244918555021286
+0.9987261146496815 0.1705341637134552
+1.0 0.15476354956626892
+1.0101910828025478 0.23188567161560059
+1.0203821656050955 0.3369496166706085
+1.0305732484076433 0.17620548605918884
+1.040764331210191 0.16009941697120667
+1.0509554140127388 0.20494461059570312
+1.0611464968152866 0.6245654821395874
+1.0713375796178344 0.5277943015098572
+1.0815286624203821 0.3080342710018158
+1.09171974522293 0.22761882841587067
+1.1019108280254777 0.21072596311569214
+1.1121019108280255 0.25082075595855713
+1.1222929936305732 0.22088050842285156
+1.132484076433121 0.15908734500408173
+1.1426751592356688 0.17322921752929688
+1.1528662420382165 0.11716874688863754
+1.1630573248407643 0.22675874829292297
+1.173248407643312 0.21380221843719482
+1.1834394904458598 0.17211849987506866
+1.1936305732484076 0.3367944061756134
+1.2038216560509554 0.15852917730808258
+1.2140127388535031 0.1922939270734787
+1.224203821656051 0.2795998752117157
+1.2343949044585987 0.2571263015270233
+1.2445859872611464 0.23508483171463013
+1.2547770700636942 0.39251235127449036
+1.264968152866242 0.34256842732429504
+1.2751592356687897 0.2917202115058899
+1.2853503184713375 0.19020971655845642
+1.2955414012738853 0.31578025221824646
+1.305732484076433 0.26129668951034546
+1.3159235668789808 0.153725728392601
+1.3261146496815286 0.38104718923568726
+1.3363057324840764 0.4601458013057709
+1.3464968152866241 0.254742830991745
+1.356687898089172 0.24096733331680298
+1.3668789808917197 0.32837432622909546
+1.3770700636942674 0.4519332945346832
+1.3872611464968152 0.24354276061058044
+1.397452229299363 0.1980389952659607
+1.4076433121019107 0.2131243348121643
+1.4178343949044585 0.31624025106430054
+1.4280254777070063 0.19946801662445068
+1.438216560509554 0.34786438941955566
+1.4484076433121018 0.28613242506980896
+1.4585987261146496 0.3078867495059967
+1.4687898089171973 0.43950170278549194
+1.4789808917197451 0.31658920645713806
+1.4891719745222929 0.2420647293329239
+1.4993630573248407 0.23756153881549835
+1.5095541401273884 0.15714193880558014
+1.5197452229299362 0.3367539644241333
+1.529936305732484 0.16685940325260162
+1.5401273885350317 0.3844830095767975
+1.5503184713375795 0.20627982914447784
+1.5605095541401273 0.19374646246433258
+1.570700636942675 0.2626991868019104
+1.5808917197452228 0.3638570010662079
+1.5910828025477706 0.23674070835113525
+1.6012738853503183 0.37753158807754517
+1.611464968152866 0.27580228447914124
+1.6216560509554139 0.26660314202308655
+1.6318471337579616 0.3224586248397827
+1.6420382165605094 0.3408588171005249
+1.6522292993630572 0.21941082179546356
+1.662420382165605 0.16279050707817078
+1.6726114649681527 0.4969157576560974
+1.6828025477707005 0.170774906873703
+1.6929936305732483 0.18043893575668335
+1.703184713375796 0.11563735455274582
+1.7133757961783438 0.361541748046875
+1.7235668789808916 0.22884972393512726
+1.7337579617834393 0.3035120368003845
+1.743949044585987 0.14285746216773987
+1.754140127388535 0.1647961288690567
+1.7643312101910829 0.20205433666706085
+1.7745222929936306 0.2094634771347046
+1.7847133757961784 0.19197341799736023
+1.7949044585987262 0.1699640154838562
+1.805095541401274 0.24046549201011658
+1.8152866242038217 0.1484258621931076
+1.8254777070063695 0.1482892632484436
+1.8356687898089172 0.5040706992149353
+1.845859872611465 0.3870737552642822
+1.8560509554140128 0.1450929045677185
+1.8662420382165605 0.11406797170639038
+1.8764331210191083 0.22828780114650726
+1.886624203821656 0.31938764452934265
+1.8968152866242038 0.32002371549606323
+1.9070063694267516 0.21325135231018066
+1.9171974522292994 0.22239337861537933
+1.9273885350318471 0.24584217369556427
+1.937579617834395 0.13751909136772156
+1.9477707006369427 0.33243614435195923
+1.9579617834394905 0.15492531657218933
+1.9681528662420382 0.09105388075113297
+1.978343949044586 0.3929058015346527
+1.9885350318471338 0.34669241309165955
+1.9987261146496815 0.2607254683971405
+2.0 0.12207172811031342
+2.0101910828025478 0.31119149923324585
+2.0203821656050955 0.14319775998592377
+2.0305732484076433 0.19798021018505096
+2.040764331210191 0.21639196574687958
+2.050955414012739 0.08943798393011093
+2.0611464968152866 0.0921352431178093
+2.0713375796178344 0.22035428881645203
+2.081528662420382 0.18190595507621765
+2.09171974522293 0.1893562525510788
+2.1019108280254777 0.15745042264461517
+2.1121019108280255 0.1491430401802063
+2.122292993630573 0.17012710869312286
+2.132484076433121 0.25030699372291565
+2.1426751592356688 0.1006808802485466
+2.1528662420382165 0.14412850141525269
+2.1630573248407643 0.16299167275428772
+2.173248407643312 0.28512611985206604
+2.18343949044586 0.04890533164143562
+2.1936305732484076 0.10450780391693115
+2.2038216560509554 0.14457325637340546
+2.214012738853503 0.11237607896327972
+2.224203821656051 0.176926389336586
+2.2343949044585987 0.24621419608592987
+2.2445859872611464 0.15521490573883057
+2.254777070063694 0.2660638093948364
+2.264968152866242 0.18591339886188507
+2.2751592356687897 0.18288636207580566
+2.2853503184713375 0.11404211819171906
+2.2955414012738853 0.16242831945419312
+2.305732484076433 0.1570262908935547
+2.315923566878981 0.11382943391799927
+2.3261146496815286 0.12227721512317657
+2.3363057324840764 0.16817837953567505
+2.346496815286624 0.1490018367767334
+2.356687898089172 0.15920542180538177
+2.3668789808917197 0.18305140733718872
+2.3770700636942674 0.099197156727314
+2.387261146496815 0.11817801743745804
+2.397452229299363 0.10630478709936142
+2.4076433121019107 0.18217776715755463
+2.4178343949044585 0.09723106771707535
+2.4280254777070063 0.09637288004159927
+2.438216560509554 0.12326230853796005
+2.448407643312102 0.094276063144207
+2.4585987261146496 0.11432629823684692
+2.4687898089171973 0.24686264991760254
+2.478980891719745 0.11512969434261322
+2.489171974522293 0.16613341867923737
+2.4993630573248407 0.15891411900520325
+2.5095541401273884 0.6151189804077148
+2.519745222929936 0.17855669558048248
+2.529936305732484 0.1135454773902893
+2.5401273885350317 0.25576266646385193
+2.5503184713375795 0.18084122240543365
+2.5605095541401273 0.10597167909145355
+2.570700636942675 0.2155171036720276
+2.580891719745223 0.2891685366630554
+2.5910828025477706 0.201302632689476
+2.6012738853503183 0.138000026345253
+2.611464968152866 0.14248836040496826
+2.621656050955414 0.13530048727989197
+2.6318471337579616 0.14262661337852478
+2.6420382165605094 0.10967038571834564
+2.652229299363057 0.19146840274333954
+2.662420382165605 0.1155640184879303
+2.6726114649681527 0.1584577113389969
+2.6828025477707005 0.1578441858291626
+2.6929936305732483 0.05720777064561844
+2.703184713375796 0.2078534960746765
+2.713375796178344 0.1210533082485199
+2.7235668789808916 0.12541314959526062
+2.7337579617834393 0.14327681064605713
+2.743949044585987 0.10486380755901337
+2.754140127388535 0.11306212097406387
+2.7643312101910826 0.16642490029335022
+2.7745222929936304 0.08535230159759521
+2.784713375796178 0.16168959438800812
+2.794904458598726 0.2000589668750763
+2.8050955414012737 0.14879369735717773
+2.8152866242038215 0.1351848542690277
+2.8254777070063692 0.10965745896100998
+2.835668789808917 0.1449526995420456
+2.845859872611465 0.16457107663154602
+2.8560509554140125 0.2250441014766693
+2.8662420382165603 0.19434526562690735
+2.876433121019108 0.32755422592163086
+2.886624203821656 0.10235312581062317
+2.8968152866242036 0.16005685925483704
+2.9070063694267514 0.10673708468675613
+2.917197452229299 0.21009820699691772
+2.927388535031847 0.15081869065761566
+2.9375796178343947 0.1252090483903885
+2.9477707006369425 0.09439841657876968
+2.9579617834394902 0.15273809432983398
+2.968152866242038 0.11691146343946457
+2.9783439490445858 0.1469116359949112
+2.9885350318471335 0.09182402491569519
+2.9987261146496813 0.13682223856449127
+3.0 0.08698958903551102

figures/loss.gp

@@ -0,0 +1,13 @@
+# training loss curve — regenerate: gnuplot figures/loss.gp  (from repo root)
+set terminal pngcairo size 760,420 font "monospace,11" background rgb "white"
+set output "figures/loss.png"
+set title "training loss  —  Mistral-7B LoRA r=16, 787 trajectories, 3 epochs (fp16)"
+set xlabel "epoch"
+set ylabel "loss (assistant-only)"
+set grid ytics lc rgb "#dddddd"
+set border 3
+set tics nomirror
+set key off
+set xrange [0:3]
+set yrange [0:*]
+plot "figures/loss.dat" using 1:2 with lines lw 2 lc rgb "black"

figures/mechanism.puml

@@ -0,0 +1,27 @@
+@startuml
+' The operate-and-terminate mechanism the adapter installs.
+skinparam monochrome true
+skinparam shadowing false
+skinparam defaultFontName monospace
+skinparam ActivityBackgroundColor white
+skinparam ActivityDiamondBackgroundColor white
+title operate-and-terminate (no task in the prompt)
+
+start
+:land in a shell;
+:read the world\n(motd / env var / file / provider --help);
+:discover the directive;
+repeat
+  :emit one command;
+  :the shell runs it;
+  :observe the output as context;
+repeat while (more to do?) is (yes)
+->no;
+if (assignment satisfied?) then (yes)
+  :exit;
+  stop
+else (genuinely blocked)
+  :panic;
+  stop
+endif
+@enduml

figures/outcomes.dat

@@ -0,0 +1,2 @@
+permissions 5 2 1 0
+text_replace 4 2 1 1

figures/outcomes.gp

@@ -0,0 +1,19 @@
+# held-out outcomes by archetype — regenerate: gnuplot figures/outcomes.gp  (from repo root)
+set terminal pngcairo noenhanced size 760,440 font "monospace,11" background rgb "white"
+set output "figures/outcomes.png"
+set title "held-out generalization (n=16) — outcomes by archetype"
+set style data histograms
+set style histogram rowstacked
+set style fill solid 1.0 border rgb "black"
+set boxwidth 0.55
+set ylabel "scenarios"
+set yrange [0:8.5]
+set grid ytics lc rgb "#dddddd"
+set border 3
+set tics nomirror
+set key outside right top reverse Left samplen 1.5
+# columns: 1=archetype 2=success 3=incomplete 4=wrong_panic 5=premature_exit
+plot "figures/outcomes.dat" using 2:xtic(1) title "success"        lc rgb "#1a1a1a", \
+     ""                     using 3          title "incomplete"     lc rgb "#777777", \
+     ""                     using 4          title "wrong_panic"    lc rgb "#aaaaaa", \
+     ""                     using 5          title "premature_exit" lc rgb "#dddddd"

figures/render-contract.puml

@@ -0,0 +1,29 @@
+@startuml
+' train = serve: the same render on both sides.
+skinparam monochrome true
+skinparam shadowing false
+skinparam defaultFontName monospace
+skinparam rectangleBackgroundColor white
+title train = serve — one render contract
+
+rectangle "TRAIN\ntrajectory turns\n(system / user / assistant)" as T
+rectangle "SERVE\nccpty over the OpenAI protocol\n{role, content} messages" as S
+
+rectangle "normalize_for_template\nfold system into the first user turn,\nmerge consecutive same-role turns" as N
+rectangle "apply_chat_template\n(Mistral-7B-Instruct-v0.2 default template)" as A
+
+rectangle "SFT\nassistant-only loss mask\n(commands + the exit / panic token)" as F
+rectangle "model.generate\none command per turn" as G
+
+T -down-> N
+S -down-> N
+N -down-> A
+A -down-> F : train
+A -down-> G : serve
+
+note right of A
+  identical tokens on both sides;
+  they diverge only at
+  loss (train) vs decode (serve)
+end note
+@enduml

tokenizer_config.json

@@ -0,0 +1,18 @@
+{
+  "add_prefix_space": null,
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "extra_special_tokens": [],
+  "is_local": true,
+  "legacy": false,
+  "local_files_only": false,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "</s>",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

training/eval.log

@@ -0,0 +1,29 @@
+[transformers] `torch_dtype` is deprecated! Use `dtype` instead!
+
+[transformers] The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
+  permissions_7ae14a1411a7: wrong_panic (terminal=panic verified=True steps=2)
+  text_replace_718686d3a2b2: premature_exit (terminal=exit verified=False steps=8)
+  permissions_917866eaadda: success (terminal=exit verified=True steps=4)
+  text_replace_b71816a16fa8: incomplete (terminal=None verified=False steps=30)
+  permissions_c46cbef0e1a1: success (terminal=exit verified=True steps=14)
+  text_replace_c0b4ca9bbb91: incomplete (terminal=None verified=False steps=30)
+  permissions_1845d061f412: success (terminal=exit verified=True steps=6)
+  text_replace_a97a7547f412: success (terminal=exit verified=True steps=29)
+  permissions_367ba331169e: success (terminal=exit verified=True steps=12)
+  text_replace_c155c881228a: success (terminal=exit verified=True steps=4)
+  permissions_332774800f96: success (terminal=exit verified=True steps=7)
+  text_replace_c1a29ebfe948: wrong_panic (terminal=panic verified=False steps=23)
+  permissions_eff856415243: incomplete (terminal=None verified=True steps=30)
+  text_replace_891a0c52f02e: success (terminal=exit verified=True steps=3)
+  permissions_7030f2fc275a: incomplete (terminal=None verified=True steps=30)
+  text_replace_569a5b190c9d: success (terminal=exit verified=True steps=23)
+
+=== behavioural metrics ===
+{
+  "n": 16,
+  "operate_rate": 1.0,
+  "terminate_rate": 0.75,
+  "verified_rate": 0.75,
+  "clean_rate": 0.562
+}
+EVAL_DONE rc=0

training/log_history.jsonl

@@ -0,0 +1,298 @@
+{"loss": 6.542654037475586, "grad_norm": NaN, "learning_rate": 0.0, "epoch": 0.01019108280254777, "step": 1}
+{"loss": 5.633924961090088, "grad_norm": 56.9017448425293, "learning_rate": 0.0, "epoch": 0.02038216560509554, "step": 2}
+{"loss": 4.195195198059082, "grad_norm": 47.943397521972656, "learning_rate": 2.2222222222222223e-05, "epoch": 0.030573248407643312, "step": 3}
+{"loss": 4.278750419616699, "grad_norm": 42.16549301147461, "learning_rate": 4.4444444444444447e-05, "epoch": 0.04076433121019108, "step": 4}
+{"loss": 4.154404640197754, "grad_norm": NaN, "learning_rate": 6.666666666666667e-05, "epoch": 0.050955414012738856, "step": 5}
+{"loss": 4.504849910736084, "grad_norm": 45.793277740478516, "learning_rate": 6.666666666666667e-05, "epoch": 0.061146496815286625, "step": 6}
+{"loss": 1.7862623929977417, "grad_norm": 17.090721130371094, "learning_rate": 8.888888888888889e-05, "epoch": 0.07133757961783439, "step": 7}
+{"loss": 1.3006527423858643, "grad_norm": 10.447768211364746, "learning_rate": 0.00011111111111111112, "epoch": 0.08152866242038216, "step": 8}
+{"loss": 0.7708751559257507, "grad_norm": 7.540386199951172, "learning_rate": 0.00013333333333333334, "epoch": 0.09171974522292993, "step": 9}
+{"loss": 0.9024325013160706, "grad_norm": 5.1253838539123535, "learning_rate": 0.00015555555555555556, "epoch": 0.10191082802547771, "step": 10}
+{"loss": 0.6090670824050903, "grad_norm": 2.833193063735962, "learning_rate": 0.00017777777777777779, "epoch": 0.11210191082802548, "step": 11}
+{"loss": 0.8035333156585693, "grad_norm": 2.5176000595092773, "learning_rate": 0.0002, "epoch": 0.12229299363057325, "step": 12}
+{"loss": 0.9093412756919861, "grad_norm": 2.9308557510375977, "learning_rate": 0.00019930555555555557, "epoch": 0.13248407643312102, "step": 13}
+{"loss": 0.7320461273193359, "grad_norm": 2.4791018962860107, "learning_rate": 0.00019861111111111113, "epoch": 0.14267515923566879, "step": 14}
+{"loss": 0.5099712014198303, "grad_norm": 1.3569127321243286, "learning_rate": 0.0001979166666666667, "epoch": 0.15286624203821655, "step": 15}
+{"loss": 0.43679556250572205, "grad_norm": 2.175459384918213, "learning_rate": 0.00019722222222222225, "epoch": 0.16305732484076432, "step": 16}
+{"loss": 0.4829658567905426, "grad_norm": 1.7427687644958496, "learning_rate": 0.00019652777777777778, "epoch": 0.1732484076433121, "step": 17}
+{"loss": 0.6208105683326721, "grad_norm": 1.8584786653518677, "learning_rate": 0.00019583333333333334, "epoch": 0.18343949044585986, "step": 18}
+{"loss": 0.3172719180583954, "grad_norm": 2.178077220916748, "learning_rate": 0.0001951388888888889, "epoch": 0.19363057324840766, "step": 19}
+{"loss": 0.5196393728256226, "grad_norm": 2.2710976600646973, "learning_rate": 0.00019444444444444446, "epoch": 0.20382165605095542, "step": 20}
+{"loss": 0.3891354501247406, "grad_norm": 1.5474536418914795, "learning_rate": 0.00019375000000000002, "epoch": 0.2140127388535032, "step": 21}
+{"loss": 0.7099442481994629, "grad_norm": 3.202181577682495, "learning_rate": 0.00019305555555555558, "epoch": 0.22420382165605096, "step": 22}
+{"loss": 0.29879891872406006, "grad_norm": 1.540442943572998, "learning_rate": 0.0001923611111111111, "epoch": 0.23439490445859873, "step": 23}
+{"loss": 0.8804287910461426, "grad_norm": 1.9141597747802734, "learning_rate": 0.00019166666666666667, "epoch": 0.2445859872611465, "step": 24}
+{"loss": 0.3760314881801605, "grad_norm": 1.6565154790878296, "learning_rate": 0.00019097222222222223, "epoch": 0.25477707006369427, "step": 25}
+{"loss": 0.3933477997779846, "grad_norm": 2.4528145790100098, "learning_rate": 0.00019027777777777776, "epoch": 0.26496815286624203, "step": 26}
+{"loss": 0.273645281791687, "grad_norm": 1.5765495300292969, "learning_rate": 0.00018958333333333332, "epoch": 0.2751592356687898, "step": 27}
+{"loss": 0.8299562335014343, "grad_norm": 1.638718605041504, "learning_rate": 0.00018888888888888888, "epoch": 0.28535031847133757, "step": 28}
+{"loss": 0.5262695550918579, "grad_norm": 1.3697830438613892, "learning_rate": 0.00018819444444444444, "epoch": 0.29554140127388534, "step": 29}
+{"loss": 0.3425145149230957, "grad_norm": 1.774098515510559, "learning_rate": 0.0001875, "epoch": 0.3057324840764331, "step": 30}
+{"loss": 0.785426139831543, "grad_norm": 1.8325257301330566, "learning_rate": 0.00018680555555555556, "epoch": 0.3159235668789809, "step": 31}
+{"loss": 0.5146355628967285, "grad_norm": 2.509580373764038, "learning_rate": 0.00018611111111111112, "epoch": 0.32611464968152865, "step": 32}
+{"loss": 0.6166573762893677, "grad_norm": 2.632068157196045, "learning_rate": 0.00018541666666666668, "epoch": 0.3363057324840764, "step": 33}
+{"loss": 0.43904078006744385, "grad_norm": 1.254080891609192, "learning_rate": 0.00018472222222222224, "epoch": 0.3464968152866242, "step": 34}
+{"loss": 0.42124295234680176, "grad_norm": 1.4523411989212036, "learning_rate": 0.00018402777777777778, "epoch": 0.35668789808917195, "step": 35}
+{"loss": 0.36162006855010986, "grad_norm": 1.3615525960922241, "learning_rate": 0.00018333333333333334, "epoch": 0.3668789808917197, "step": 36}
+{"loss": 0.2639588415622711, "grad_norm": 1.3932915925979614, "learning_rate": 0.0001826388888888889, "epoch": 0.37707006369426754, "step": 37}
+{"loss": 0.40662309527397156, "grad_norm": 1.760315179824829, "learning_rate": 0.00018194444444444445, "epoch": 0.3872611464968153, "step": 38}
+{"loss": 0.3518657386302948, "grad_norm": 1.1476843357086182, "learning_rate": 0.00018125000000000001, "epoch": 0.3974522292993631, "step": 39}
+{"loss": 0.6724132895469666, "grad_norm": 1.7566606998443604, "learning_rate": 0.00018055555555555557, "epoch": 0.40764331210191085, "step": 40}
+{"loss": 0.42467063665390015, "grad_norm": 2.0741333961486816, "learning_rate": 0.00017986111111111113, "epoch": 0.4178343949044586, "step": 41}
+{"loss": 0.5263597369194031, "grad_norm": 1.6360384225845337, "learning_rate": 0.0001791666666666667, "epoch": 0.4280254777070064, "step": 42}
+{"loss": 0.269584596157074, "grad_norm": 1.2943912744522095, "learning_rate": 0.00017847222222222225, "epoch": 0.43821656050955415, "step": 43}
+{"loss": 0.18777093291282654, "grad_norm": 0.66461580991745, "learning_rate": 0.00017777777777777779, "epoch": 0.4484076433121019, "step": 44}
+{"loss": 0.4670497179031372, "grad_norm": 1.4662903547286987, "learning_rate": 0.00017708333333333335, "epoch": 0.4585987261146497, "step": 45}
+{"loss": 0.4456017017364502, "grad_norm": 1.6666089296340942, "learning_rate": 0.0001763888888888889, "epoch": 0.46878980891719746, "step": 46}
+{"loss": 0.4781678318977356, "grad_norm": 1.854061245918274, "learning_rate": 0.00017569444444444444, "epoch": 0.4789808917197452, "step": 47}
+{"loss": 0.48828044533729553, "grad_norm": 1.564624547958374, "learning_rate": 0.000175, "epoch": 0.489171974522293, "step": 48}
+{"loss": 0.31406861543655396, "grad_norm": 3.068716049194336, "learning_rate": 0.00017430555555555556, "epoch": 0.49936305732484076, "step": 49}
+{"loss": 0.5399861335754395, "grad_norm": 1.8206239938735962, "learning_rate": 0.00017361111111111112, "epoch": 0.5095541401273885, "step": 50}
+{"loss": 0.5312874913215637, "grad_norm": 1.5397772789001465, "learning_rate": 0.00017291666666666668, "epoch": 0.5197452229299363, "step": 51}
+{"loss": 0.39228370785713196, "grad_norm": 1.3897669315338135, "learning_rate": 0.00017222222222222224, "epoch": 0.5299363057324841, "step": 52}
+{"loss": 0.3288589119911194, "grad_norm": 1.513123869895935, "learning_rate": 0.00017152777777777777, "epoch": 0.5401273885350318, "step": 53}
+{"loss": 0.46349963545799255, "grad_norm": 1.4835230112075806, "learning_rate": 0.00017083333333333333, "epoch": 0.5503184713375796, "step": 54}
+{"loss": 0.615800678730011, "grad_norm": 1.8706779479980469, "learning_rate": 0.0001701388888888889, "epoch": 0.5605095541401274, "step": 55}
+{"loss": 0.2980297803878784, "grad_norm": 1.972475290298462, "learning_rate": 0.00016944444444444445, "epoch": 0.5707006369426751, "step": 56}
+{"loss": 0.3729114532470703, "grad_norm": 1.3181968927383423, "learning_rate": 0.00016875, "epoch": 0.5808917197452229, "step": 57}
+{"loss": 0.4166586101055145, "grad_norm": 1.4189494848251343, "learning_rate": 0.00016805555555555557, "epoch": 0.5910828025477707, "step": 58}
+{"loss": 0.5619450807571411, "grad_norm": 1.474931001663208, "learning_rate": 0.00016736111111111113, "epoch": 0.6012738853503184, "step": 59}
+{"loss": 0.3892599940299988, "grad_norm": 1.536375641822815, "learning_rate": 0.0001666666666666667, "epoch": 0.6114649681528662, "step": 60}
+{"loss": 0.37334829568862915, "grad_norm": 1.5133776664733887, "learning_rate": 0.00016597222222222225, "epoch": 0.621656050955414, "step": 61}
+{"loss": 0.3734676241874695, "grad_norm": 1.3775814771652222, "learning_rate": 0.00016527777777777778, "epoch": 0.6318471337579618, "step": 62}
+{"loss": 0.37577956914901733, "grad_norm": 1.1052874326705933, "learning_rate": 0.00016458333333333334, "epoch": 0.6420382165605095, "step": 63}
+{"loss": 0.34371814131736755, "grad_norm": 1.649427890777588, "learning_rate": 0.0001638888888888889, "epoch": 0.6522292993630573, "step": 64}
+{"loss": 0.2734268307685852, "grad_norm": 1.5963698625564575, "learning_rate": 0.00016319444444444446, "epoch": 0.6624203821656051, "step": 65}
+{"loss": 0.5835185647010803, "grad_norm": 2.1361522674560547, "learning_rate": 0.00016250000000000002, "epoch": 0.6726114649681528, "step": 66}
+{"loss": 0.48496758937835693, "grad_norm": 1.8972663879394531, "learning_rate": 0.00016180555555555555, "epoch": 0.6828025477707006, "step": 67}
+{"loss": 0.3867255449295044, "grad_norm": 1.3294984102249146, "learning_rate": 0.0001611111111111111, "epoch": 0.6929936305732484, "step": 68}
+{"loss": 0.4328857660293579, "grad_norm": 1.279388189315796, "learning_rate": 0.00016041666666666667, "epoch": 0.7031847133757961, "step": 69}
+{"loss": 0.2687888741493225, "grad_norm": 1.0728896856307983, "learning_rate": 0.00015972222222222223, "epoch": 0.7133757961783439, "step": 70}
+{"loss": 0.43020740151405334, "grad_norm": 1.6005398035049438, "learning_rate": 0.00015902777777777776, "epoch": 0.7235668789808917, "step": 71}
+{"loss": 0.4155803918838501, "grad_norm": 1.5793131589889526, "learning_rate": 0.00015833333333333332, "epoch": 0.7337579617834394, "step": 72}
+{"loss": 0.40452662110328674, "grad_norm": 1.6298348903656006, "learning_rate": 0.00015763888888888888, "epoch": 0.7439490445859872, "step": 73}
+{"loss": 0.27838924527168274, "grad_norm": 1.2805296182632446, "learning_rate": 0.00015694444444444444, "epoch": 0.7541401273885351, "step": 74}
+{"loss": 0.3107945919036865, "grad_norm": 0.889247715473175, "learning_rate": 0.00015625, "epoch": 0.7643312101910829, "step": 75}
+{"loss": 0.48347756266593933, "grad_norm": 2.0980005264282227, "learning_rate": 0.00015555555555555556, "epoch": 0.7745222929936306, "step": 76}
+{"loss": 0.16316811740398407, "grad_norm": 1.1496448516845703, "learning_rate": 0.00015486111111111112, "epoch": 0.7847133757961784, "step": 77}
+{"loss": 0.3178855776786804, "grad_norm": 1.352499008178711, "learning_rate": 0.00015416666666666668, "epoch": 0.7949044585987262, "step": 78}
+{"loss": 0.45595476031303406, "grad_norm": 2.9348864555358887, "learning_rate": 0.00015347222222222224, "epoch": 0.8050955414012739, "step": 79}
+{"loss": 0.3289816379547119, "grad_norm": 1.5154201984405518, "learning_rate": 0.00015277777777777777, "epoch": 0.8152866242038217, "step": 80}
+{"loss": 0.26671987771987915, "grad_norm": 1.5175567865371704, "learning_rate": 0.00015208333333333333, "epoch": 0.8254777070063695, "step": 81}
+{"loss": 0.3158896565437317, "grad_norm": 1.3976314067840576, "learning_rate": 0.0001513888888888889, "epoch": 0.8356687898089172, "step": 82}
+{"loss": 0.2974770665168762, "grad_norm": 1.3389358520507812, "learning_rate": 0.00015069444444444445, "epoch": 0.845859872611465, "step": 83}
+{"loss": 0.346922904253006, "grad_norm": 1.5848623514175415, "learning_rate": 0.00015000000000000001, "epoch": 0.8560509554140128, "step": 84}
+{"loss": 0.325134813785553, "grad_norm": 1.5177457332611084, "learning_rate": 0.00014930555555555557, "epoch": 0.8662420382165605, "step": 85}
+{"loss": 0.4855765998363495, "grad_norm": 1.4754853248596191, "learning_rate": 0.00014861111111111113, "epoch": 0.8764331210191083, "step": 86}
+{"loss": 0.6393551230430603, "grad_norm": 3.004265069961548, "learning_rate": 0.0001479166666666667, "epoch": 0.8866242038216561, "step": 87}
+{"loss": 0.3373737335205078, "grad_norm": 0.9760028719902039, "learning_rate": 0.00014722222222222223, "epoch": 0.8968152866242038, "step": 88}
+{"loss": 0.30147653818130493, "grad_norm": 1.1751753091812134, "learning_rate": 0.00014652777777777779, "epoch": 0.9070063694267516, "step": 89}
+{"loss": 0.20247593522071838, "grad_norm": 0.6870393753051758, "learning_rate": 0.00014583333333333335, "epoch": 0.9171974522292994, "step": 90}
+{"loss": 0.2578410506248474, "grad_norm": 1.130759835243225, "learning_rate": 0.00014513888888888888, "epoch": 0.9273885350318471, "step": 91}
+{"loss": 0.3888697326183319, "grad_norm": 1.8160619735717773, "learning_rate": 0.00014444444444444444, "epoch": 0.9375796178343949, "step": 92}
+{"loss": 0.1678299605846405, "grad_norm": 1.1780165433883667, "learning_rate": 0.00014375, "epoch": 0.9477707006369427, "step": 93}
+{"loss": 0.509365439414978, "grad_norm": 1.6333993673324585, "learning_rate": 0.00014305555555555556, "epoch": 0.9579617834394905, "step": 94}
+{"loss": 0.3836624026298523, "grad_norm": 1.8088518381118774, "learning_rate": 0.00014236111111111112, "epoch": 0.9681528662420382, "step": 95}
+{"loss": 0.23250575363636017, "grad_norm": 1.5874489545822144, "learning_rate": 0.00014166666666666668, "epoch": 0.978343949044586, "step": 96}
+{"loss": 0.244918555021286, "grad_norm": 1.3233226537704468, "learning_rate": 0.00014097222222222224, "epoch": 0.9885350318471338, "step": 97}
+{"loss": 0.1705341637134552, "grad_norm": 1.0406038761138916, "learning_rate": 0.00014027777777777777, "epoch": 0.9987261146496815, "step": 98}
+{"loss": 0.15476354956626892, "grad_norm": 2.7193310260772705, "learning_rate": 0.00013958333333333333, "epoch": 1.0, "step": 99}
+{"loss": 0.23188567161560059, "grad_norm": 0.9795278906822205, "learning_rate": 0.0001388888888888889, "epoch": 1.0101910828025478, "step": 100}
+{"loss": 0.3369496166706085, "grad_norm": 1.8278563022613525, "learning_rate": 0.00013819444444444445, "epoch": 1.0203821656050955, "step": 101}
+{"loss": 0.17620548605918884, "grad_norm": 0.7256863713264465, "learning_rate": 0.0001375, "epoch": 1.0305732484076433, "step": 102}
+{"loss": 0.16009941697120667, "grad_norm": 1.3019368648529053, "learning_rate": 0.00013680555555555557, "epoch": 1.040764331210191, "step": 103}
+{"loss": 0.20494461059570312, "grad_norm": 1.5364458560943604, "learning_rate": 0.00013611111111111113, "epoch": 1.0509554140127388, "step": 104}
+{"loss": 0.6245654821395874, "grad_norm": 1.5218592882156372, "learning_rate": 0.0001354166666666667, "epoch": 1.0611464968152866, "step": 105}
+{"loss": 0.5277943015098572, "grad_norm": 1.6173017024993896, "learning_rate": 0.00013472222222222225, "epoch": 1.0713375796178344, "step": 106}
+{"loss": 0.3080342710018158, "grad_norm": 1.1873356103897095, "learning_rate": 0.00013402777777777778, "epoch": 1.0815286624203821, "step": 107}
+{"loss": 0.22761882841587067, "grad_norm": 1.5300596952438354, "learning_rate": 0.00013333333333333334, "epoch": 1.09171974522293, "step": 108}
+{"loss": 0.21072596311569214, "grad_norm": 1.2961335182189941, "learning_rate": 0.0001326388888888889, "epoch": 1.1019108280254777, "step": 109}
+{"loss": 0.25082075595855713, "grad_norm": 1.5111762285232544, "learning_rate": 0.00013194444444444446, "epoch": 1.1121019108280255, "step": 110}
+{"loss": 0.22088050842285156, "grad_norm": 1.427400827407837, "learning_rate": 0.00013125000000000002, "epoch": 1.1222929936305732, "step": 111}
+{"loss": 0.15908734500408173, "grad_norm": 1.1548067331314087, "learning_rate": 0.00013055555555555555, "epoch": 1.132484076433121, "step": 112}
+{"loss": 0.17322921752929688, "grad_norm": 1.143130898475647, "learning_rate": 0.0001298611111111111, "epoch": 1.1426751592356688, "step": 113}
+{"loss": 0.11716874688863754, "grad_norm": 1.0793495178222656, "learning_rate": 0.00012916666666666667, "epoch": 1.1528662420382165, "step": 114}
+{"loss": 0.22675874829292297, "grad_norm": 1.2616031169891357, "learning_rate": 0.00012847222222222223, "epoch": 1.1630573248407643, "step": 115}
+{"loss": 0.21380221843719482, "grad_norm": 1.0367132425308228, "learning_rate": 0.00012777777777777776, "epoch": 1.173248407643312, "step": 116}
+{"loss": 0.17211849987506866, "grad_norm": 1.3807631731033325, "learning_rate": 0.00012708333333333332, "epoch": 1.1834394904458598, "step": 117}
+{"loss": 0.3367944061756134, "grad_norm": 1.4091378450393677, "learning_rate": 0.00012638888888888888, "epoch": 1.1936305732484076, "step": 118}
+{"loss": 0.15852917730808258, "grad_norm": 1.4397214651107788, "learning_rate": 0.00012569444444444444, "epoch": 1.2038216560509554, "step": 119}
+{"loss": 0.1922939270734787, "grad_norm": 1.2520420551300049, "learning_rate": 0.000125, "epoch": 1.2140127388535031, "step": 120}
+{"loss": 0.2795998752117157, "grad_norm": 1.2190641164779663, "learning_rate": 0.00012430555555555556, "epoch": 1.224203821656051, "step": 121}
+{"loss": 0.2571263015270233, "grad_norm": 1.845451831817627, "learning_rate": 0.00012361111111111112, "epoch": 1.2343949044585987, "step": 122}
+{"loss": 0.23508483171463013, "grad_norm": 0.9931548237800598, "learning_rate": 0.00012291666666666668, "epoch": 1.2445859872611464, "step": 123}
+{"loss": 0.39251235127449036, "grad_norm": 1.857635736465454, "learning_rate": 0.00012222222222222224, "epoch": 1.2547770700636942, "step": 124}
+{"loss": 0.34256842732429504, "grad_norm": 1.8374614715576172, "learning_rate": 0.00012152777777777777, "epoch": 1.264968152866242, "step": 125}
+{"loss": 0.2917202115058899, "grad_norm": 1.4454501867294312, "learning_rate": 0.00012083333333333333, "epoch": 1.2751592356687897, "step": 126}
+{"loss": 0.19020971655845642, "grad_norm": 0.804486095905304, "learning_rate": 0.0001201388888888889, "epoch": 1.2853503184713375, "step": 127}
+{"loss": 0.31578025221824646, "grad_norm": 1.3078372478485107, "learning_rate": 0.00011944444444444445, "epoch": 1.2955414012738853, "step": 128}
+{"loss": 0.26129668951034546, "grad_norm": 0.7924992442131042, "learning_rate": 0.00011875, "epoch": 1.305732484076433, "step": 129}
+{"loss": 0.153725728392601, "grad_norm": 1.298997402191162, "learning_rate": 0.00011805555555555556, "epoch": 1.3159235668789808, "step": 130}
+{"loss": 0.38104718923568726, "grad_norm": 1.6158643960952759, "learning_rate": 0.00011736111111111112, "epoch": 1.3261146496815286, "step": 131}
+{"loss": 0.4601458013057709, "grad_norm": 1.7336560487747192, "learning_rate": 0.00011666666666666668, "epoch": 1.3363057324840764, "step": 132}
+{"loss": 0.254742830991745, "grad_norm": 1.5181490182876587, "learning_rate": 0.00011597222222222224, "epoch": 1.3464968152866241, "step": 133}
+{"loss": 0.24096733331680298, "grad_norm": 1.5294867753982544, "learning_rate": 0.00011527777777777777, "epoch": 1.356687898089172, "step": 134}
+{"loss": 0.32837432622909546, "grad_norm": 1.4007141590118408, "learning_rate": 0.00011458333333333333, "epoch": 1.3668789808917197, "step": 135}
+{"loss": 0.4519332945346832, "grad_norm": 1.668914556503296, "learning_rate": 0.00011388888888888889, "epoch": 1.3770700636942674, "step": 136}
+{"loss": 0.24354276061058044, "grad_norm": 1.2457365989685059, "learning_rate": 0.00011319444444444445, "epoch": 1.3872611464968152, "step": 137}
+{"loss": 0.1980389952659607, "grad_norm": 1.0845744609832764, "learning_rate": 0.00011250000000000001, "epoch": 1.397452229299363, "step": 138}
+{"loss": 0.2131243348121643, "grad_norm": 1.1750677824020386, "learning_rate": 0.00011180555555555556, "epoch": 1.4076433121019107, "step": 139}
+{"loss": 0.31624025106430054, "grad_norm": 1.5499531030654907, "learning_rate": 0.00011111111111111112, "epoch": 1.4178343949044585, "step": 140}
+{"loss": 0.19946801662445068, "grad_norm": 0.9320492744445801, "learning_rate": 0.00011041666666666668, "epoch": 1.4280254777070063, "step": 141}
+{"loss": 0.34786438941955566, "grad_norm": 1.3403514623641968, "learning_rate": 0.00010972222222222224, "epoch": 1.438216560509554, "step": 142}
+{"loss": 0.28613242506980896, "grad_norm": 1.2110610008239746, "learning_rate": 0.00010902777777777777, "epoch": 1.4484076433121018, "step": 143}
+{"loss": 0.3078867495059967, "grad_norm": 1.4140794277191162, "learning_rate": 0.00010833333333333333, "epoch": 1.4585987261146496, "step": 144}
+{"loss": 0.43950170278549194, "grad_norm": 2.000192403793335, "learning_rate": 0.00010763888888888889, "epoch": 1.4687898089171973, "step": 145}
+{"loss": 0.31658920645713806, "grad_norm": 1.7949906587600708, "learning_rate": 0.00010694444444444445, "epoch": 1.4789808917197451, "step": 146}
+{"loss": 0.2420647293329239, "grad_norm": 1.0642324686050415, "learning_rate": 0.00010625000000000001, "epoch": 1.4891719745222929, "step": 147}
+{"loss": 0.23756153881549835, "grad_norm": 1.1384238004684448, "learning_rate": 0.00010555555555555557, "epoch": 1.4993630573248407, "step": 148}
+{"loss": 0.15714193880558014, "grad_norm": 1.4857715368270874, "learning_rate": 0.00010486111111111113, "epoch": 1.5095541401273884, "step": 149}
+{"loss": 0.3367539644241333, "grad_norm": 1.406209111213684, "learning_rate": 0.00010416666666666667, "epoch": 1.5197452229299362, "step": 150}
+{"loss": 0.16685940325260162, "grad_norm": 1.6551127433776855, "learning_rate": 0.00010347222222222223, "epoch": 1.529936305732484, "step": 151}
+{"loss": 0.3844830095767975, "grad_norm": 1.9670729637145996, "learning_rate": 0.00010277777777777778, "epoch": 1.5401273885350317, "step": 152}
+{"loss": 0.20627982914447784, "grad_norm": 1.2962584495544434, "learning_rate": 0.00010208333333333333, "epoch": 1.5503184713375795, "step": 153}
+{"loss": 0.19374646246433258, "grad_norm": 1.2145798206329346, "learning_rate": 0.00010138888888888889, "epoch": 1.5605095541401273, "step": 154}
+{"loss": 0.2626991868019104, "grad_norm": 1.713334560394287, "learning_rate": 0.00010069444444444445, "epoch": 1.570700636942675, "step": 155}
+{"loss": 0.3638570010662079, "grad_norm": 1.328944444656372, "learning_rate": 0.0001, "epoch": 1.5808917197452228, "step": 156}
+{"loss": 0.23674070835113525, "grad_norm": 1.4894388914108276, "learning_rate": 9.930555555555556e-05, "epoch": 1.5910828025477706, "step": 157}
+{"loss": 0.37753158807754517, "grad_norm": 1.5659937858581543, "learning_rate": 9.861111111111112e-05, "epoch": 1.6012738853503183, "step": 158}
+{"loss": 0.27580228447914124, "grad_norm": 1.3824529647827148, "learning_rate": 9.791666666666667e-05, "epoch": 1.611464968152866, "step": 159}
+{"loss": 0.26660314202308655, "grad_norm": 1.6860393285751343, "learning_rate": 9.722222222222223e-05, "epoch": 1.6216560509554139, "step": 160}
+{"loss": 0.3224586248397827, "grad_norm": 1.6814537048339844, "learning_rate": 9.652777777777779e-05, "epoch": 1.6318471337579616, "step": 161}
+{"loss": 0.3408588171005249, "grad_norm": 1.8641172647476196, "learning_rate": 9.583333333333334e-05, "epoch": 1.6420382165605094, "step": 162}
+{"loss": 0.21941082179546356, "grad_norm": 1.0742312669754028, "learning_rate": 9.513888888888888e-05, "epoch": 1.6522292993630572, "step": 163}
+{"loss": 0.16279050707817078, "grad_norm": 1.126552939414978, "learning_rate": 9.444444444444444e-05, "epoch": 1.662420382165605, "step": 164}
+{"loss": 0.4969157576560974, "grad_norm": 1.0856200456619263, "learning_rate": 9.375e-05, "epoch": 1.6726114649681527, "step": 165}
+{"loss": 0.170774906873703, "grad_norm": 1.0782660245895386, "learning_rate": 9.305555555555556e-05, "epoch": 1.6828025477707005, "step": 166}
+{"loss": 0.18043893575668335, "grad_norm": 0.8588445782661438, "learning_rate": 9.236111111111112e-05, "epoch": 1.6929936305732483, "step": 167}
+{"loss": 0.11563735455274582, "grad_norm": 0.8225221037864685, "learning_rate": 9.166666666666667e-05, "epoch": 1.703184713375796, "step": 168}
+{"loss": 0.361541748046875, "grad_norm": 1.7441767454147339, "learning_rate": 9.097222222222223e-05, "epoch": 1.7133757961783438, "step": 169}
+{"loss": 0.22884972393512726, "grad_norm": 1.0826810598373413, "learning_rate": 9.027777777777779e-05, "epoch": 1.7235668789808916, "step": 170}
+{"loss": 0.3035120368003845, "grad_norm": 1.459424376487732, "learning_rate": 8.958333333333335e-05, "epoch": 1.7337579617834393, "step": 171}
+{"loss": 0.14285746216773987, "grad_norm": 0.5474423766136169, "learning_rate": 8.888888888888889e-05, "epoch": 1.743949044585987, "step": 172}
+{"loss": 0.1647961288690567, "grad_norm": 1.3023803234100342, "learning_rate": 8.819444444444445e-05, "epoch": 1.754140127388535, "step": 173}
+{"loss": 0.20205433666706085, "grad_norm": 1.6675821542739868, "learning_rate": 8.75e-05, "epoch": 1.7643312101910829, "step": 174}
+{"loss": 0.2094634771347046, "grad_norm": 1.2591818571090698, "learning_rate": 8.680555555555556e-05, "epoch": 1.7745222929936306, "step": 175}
+{"loss": 0.19197341799736023, "grad_norm": 1.3784693479537964, "learning_rate": 8.611111111111112e-05, "epoch": 1.7847133757961784, "step": 176}
+{"loss": 0.1699640154838562, "grad_norm": 1.3793846368789673, "learning_rate": 8.541666666666666e-05, "epoch": 1.7949044585987262, "step": 177}
+{"loss": 0.24046549201011658, "grad_norm": 1.3640600442886353, "learning_rate": 8.472222222222222e-05, "epoch": 1.805095541401274, "step": 178}
+{"loss": 0.1484258621931076, "grad_norm": 1.0224329233169556, "learning_rate": 8.402777777777778e-05, "epoch": 1.8152866242038217, "step": 179}
+{"loss": 0.1482892632484436, "grad_norm": 1.241874098777771, "learning_rate": 8.333333333333334e-05, "epoch": 1.8254777070063695, "step": 180}
+{"loss": 0.5040706992149353, "grad_norm": 1.6678941249847412, "learning_rate": 8.263888888888889e-05, "epoch": 1.8356687898089172, "step": 181}
+{"loss": 0.3870737552642822, "grad_norm": 1.509308099746704, "learning_rate": 8.194444444444445e-05, "epoch": 1.845859872611465, "step": 182}
+{"loss": 0.1450929045677185, "grad_norm": 1.3586677312850952, "learning_rate": 8.125000000000001e-05, "epoch": 1.8560509554140128, "step": 183}
+{"loss": 0.11406797170639038, "grad_norm": 0.9309101104736328, "learning_rate": 8.055555555555556e-05, "epoch": 1.8662420382165605, "step": 184}
+{"loss": 0.22828780114650726, "grad_norm": 1.4832497835159302, "learning_rate": 7.986111111111112e-05, "epoch": 1.8764331210191083, "step": 185}
+{"loss": 0.31938764452934265, "grad_norm": 1.5709041357040405, "learning_rate": 7.916666666666666e-05, "epoch": 1.886624203821656, "step": 186}
+{"loss": 0.32002371549606323, "grad_norm": 1.2977643013000488, "learning_rate": 7.847222222222222e-05, "epoch": 1.8968152866242038, "step": 187}
+{"loss": 0.21325135231018066, "grad_norm": 0.9647162556648254, "learning_rate": 7.777777777777778e-05, "epoch": 1.9070063694267516, "step": 188}
+{"loss": 0.22239337861537933, "grad_norm": 1.1396207809448242, "learning_rate": 7.708333333333334e-05, "epoch": 1.9171974522292994, "step": 189}
+{"loss": 0.24584217369556427, "grad_norm": 1.4513227939605713, "learning_rate": 7.638888888888889e-05, "epoch": 1.9273885350318471, "step": 190}
+{"loss": 0.13751909136772156, "grad_norm": 1.0884069204330444, "learning_rate": 7.569444444444445e-05, "epoch": 1.937579617834395, "step": 191}
+{"loss": 0.33243614435195923, "grad_norm": 1.3442689180374146, "learning_rate": 7.500000000000001e-05, "epoch": 1.9477707006369427, "step": 192}
+{"loss": 0.15492531657218933, "grad_norm": 0.7937957644462585, "learning_rate": 7.430555555555557e-05, "epoch": 1.9579617834394905, "step": 193}
+{"loss": 0.09105388075113297, "grad_norm": 1.3807721138000488, "learning_rate": 7.361111111111111e-05, "epoch": 1.9681528662420382, "step": 194}
+{"loss": 0.3929058015346527, "grad_norm": 1.3065935373306274, "learning_rate": 7.291666666666667e-05, "epoch": 1.978343949044586, "step": 195}
+{"loss": 0.34669241309165955, "grad_norm": 1.6988730430603027, "learning_rate": 7.222222222222222e-05, "epoch": 1.9885350318471338, "step": 196}
+{"loss": 0.2607254683971405, "grad_norm": 1.7725212574005127, "learning_rate": 7.152777777777778e-05, "epoch": 1.9987261146496815, "step": 197}
+{"loss": 0.12207172811031342, "grad_norm": 2.450087547302246, "learning_rate": 7.083333333333334e-05, "epoch": 2.0, "step": 198}
+{"loss": 0.31119149923324585, "grad_norm": 1.2870289087295532, "learning_rate": 7.013888888888888e-05, "epoch": 2.0101910828025478, "step": 199}
+{"loss": 0.14319775998592377, "grad_norm": 1.1291111707687378, "learning_rate": 6.944444444444444e-05, "epoch": 2.0203821656050955, "step": 200}
+{"loss": 0.19798021018505096, "grad_norm": 0.8054268956184387, "learning_rate": 6.875e-05, "epoch": 2.0305732484076433, "step": 201}
+{"loss": 0.21639196574687958, "grad_norm": 1.107912540435791, "learning_rate": 6.805555555555556e-05, "epoch": 2.040764331210191, "step": 202}
+{"loss": 0.08943798393011093, "grad_norm": 0.7529680728912354, "learning_rate": 6.736111111111112e-05, "epoch": 2.050955414012739, "step": 203}
+{"loss": 0.0921352431178093, "grad_norm": 0.9533487558364868, "learning_rate": 6.666666666666667e-05, "epoch": 2.0611464968152866, "step": 204}
+{"loss": 0.22035428881645203, "grad_norm": 1.4303085803985596, "learning_rate": 6.597222222222223e-05, "epoch": 2.0713375796178344, "step": 205}
+{"loss": 0.18190595507621765, "grad_norm": 0.9248675107955933, "learning_rate": 6.527777777777778e-05, "epoch": 2.081528662420382, "step": 206}
+{"loss": 0.1893562525510788, "grad_norm": 1.004992127418518, "learning_rate": 6.458333333333334e-05, "epoch": 2.09171974522293, "step": 207}
+{"loss": 0.15745042264461517, "grad_norm": 1.2227154970169067, "learning_rate": 6.388888888888888e-05, "epoch": 2.1019108280254777, "step": 208}
+{"loss": 0.1491430401802063, "grad_norm": 1.3750429153442383, "learning_rate": 6.319444444444444e-05, "epoch": 2.1121019108280255, "step": 209}
+{"loss": 0.17012710869312286, "grad_norm": 1.2999169826507568, "learning_rate": 6.25e-05, "epoch": 2.122292993630573, "step": 210}
+{"loss": 0.25030699372291565, "grad_norm": 1.6166683435440063, "learning_rate": 6.180555555555556e-05, "epoch": 2.132484076433121, "step": 211}
+{"loss": 0.1006808802485466, "grad_norm": 1.121126651763916, "learning_rate": 6.111111111111112e-05, "epoch": 2.1426751592356688, "step": 212}
+{"loss": 0.14412850141525269, "grad_norm": 1.2085487842559814, "learning_rate": 6.041666666666667e-05, "epoch": 2.1528662420382165, "step": 213}
+{"loss": 0.16299167275428772, "grad_norm": 1.7399260997772217, "learning_rate": 5.972222222222223e-05, "epoch": 2.1630573248407643, "step": 214}
+{"loss": 0.28512611985206604, "grad_norm": 1.4503214359283447, "learning_rate": 5.902777777777778e-05, "epoch": 2.173248407643312, "step": 215}
+{"loss": 0.04890533164143562, "grad_norm": 0.4843997359275818, "learning_rate": 5.833333333333334e-05, "epoch": 2.18343949044586, "step": 216}
+{"loss": 0.10450780391693115, "grad_norm": 0.8296465277671814, "learning_rate": 5.7638888888888886e-05, "epoch": 2.1936305732484076, "step": 217}
+{"loss": 0.14457325637340546, "grad_norm": 1.104335069656372, "learning_rate": 5.6944444444444445e-05, "epoch": 2.2038216560509554, "step": 218}
+{"loss": 0.11237607896327972, "grad_norm": 1.345031499862671, "learning_rate": 5.6250000000000005e-05, "epoch": 2.214012738853503, "step": 219}
+{"loss": 0.176926389336586, "grad_norm": 1.3722877502441406, "learning_rate": 5.555555555555556e-05, "epoch": 2.224203821656051, "step": 220}
+{"loss": 0.24621419608592987, "grad_norm": 1.3205420970916748, "learning_rate": 5.486111111111112e-05, "epoch": 2.2343949044585987, "step": 221}
+{"loss": 0.15521490573883057, "grad_norm": 1.6065913438796997, "learning_rate": 5.4166666666666664e-05, "epoch": 2.2445859872611464, "step": 222}
+{"loss": 0.2660638093948364, "grad_norm": 1.2175638675689697, "learning_rate": 5.3472222222222224e-05, "epoch": 2.254777070063694, "step": 223}
+{"loss": 0.18591339886188507, "grad_norm": 1.5905792713165283, "learning_rate": 5.2777777777777784e-05, "epoch": 2.264968152866242, "step": 224}
+{"loss": 0.18288636207580566, "grad_norm": 2.1838290691375732, "learning_rate": 5.208333333333334e-05, "epoch": 2.2751592356687897, "step": 225}
+{"loss": 0.11404211819171906, "grad_norm": 1.0321050882339478, "learning_rate": 5.138888888888889e-05, "epoch": 2.2853503184713375, "step": 226}
+{"loss": 0.16242831945419312, "grad_norm": 1.2717584371566772, "learning_rate": 5.069444444444444e-05, "epoch": 2.2955414012738853, "step": 227}
+{"loss": 0.1570262908935547, "grad_norm": 1.6524289846420288, "learning_rate": 5e-05, "epoch": 2.305732484076433, "step": 228}
+{"loss": 0.11382943391799927, "grad_norm": 1.2113028764724731, "learning_rate": 4.930555555555556e-05, "epoch": 2.315923566878981, "step": 229}
+{"loss": 0.12227721512317657, "grad_norm": 1.3195241689682007, "learning_rate": 4.8611111111111115e-05, "epoch": 2.3261146496815286, "step": 230}
+{"loss": 0.16817837953567505, "grad_norm": 1.6228652000427246, "learning_rate": 4.791666666666667e-05, "epoch": 2.3363057324840764, "step": 231}
+{"loss": 0.1490018367767334, "grad_norm": 1.3788127899169922, "learning_rate": 4.722222222222222e-05, "epoch": 2.346496815286624, "step": 232}
+{"loss": 0.15920542180538177, "grad_norm": 1.1768027544021606, "learning_rate": 4.652777777777778e-05, "epoch": 2.356687898089172, "step": 233}
+{"loss": 0.18305140733718872, "grad_norm": 2.005825996398926, "learning_rate": 4.5833333333333334e-05, "epoch": 2.3668789808917197, "step": 234}
+{"loss": 0.099197156727314, "grad_norm": 1.713397741317749, "learning_rate": 4.5138888888888894e-05, "epoch": 2.3770700636942674, "step": 235}
+{"loss": 0.11817801743745804, "grad_norm": 0.8157030344009399, "learning_rate": 4.4444444444444447e-05, "epoch": 2.387261146496815, "step": 236}
+{"loss": 0.10630478709936142, "grad_norm": 1.2795579433441162, "learning_rate": 4.375e-05, "epoch": 2.397452229299363, "step": 237}
+{"loss": 0.18217776715755463, "grad_norm": 1.3553754091262817, "learning_rate": 4.305555555555556e-05, "epoch": 2.4076433121019107, "step": 238}
+{"loss": 0.09723106771707535, "grad_norm": 1.049250841140747, "learning_rate": 4.236111111111111e-05, "epoch": 2.4178343949044585, "step": 239}
+{"loss": 0.09637288004159927, "grad_norm": 1.0419623851776123, "learning_rate": 4.166666666666667e-05, "epoch": 2.4280254777070063, "step": 240}
+{"loss": 0.12326230853796005, "grad_norm": 1.5575206279754639, "learning_rate": 4.0972222222222225e-05, "epoch": 2.438216560509554, "step": 241}
+{"loss": 0.094276063144207, "grad_norm": 1.0642114877700806, "learning_rate": 4.027777777777778e-05, "epoch": 2.448407643312102, "step": 242}
+{"loss": 0.11432629823684692, "grad_norm": 1.4804887771606445, "learning_rate": 3.958333333333333e-05, "epoch": 2.4585987261146496, "step": 243}
+{"loss": 0.24686264991760254, "grad_norm": 1.9122426509857178, "learning_rate": 3.888888888888889e-05, "epoch": 2.4687898089171973, "step": 244}
+{"loss": 0.11512969434261322, "grad_norm": 1.4260722398757935, "learning_rate": 3.8194444444444444e-05, "epoch": 2.478980891719745, "step": 245}
+{"loss": 0.16613341867923737, "grad_norm": 1.4465365409851074, "learning_rate": 3.7500000000000003e-05, "epoch": 2.489171974522293, "step": 246}
+{"loss": 0.15891411900520325, "grad_norm": 1.4656093120574951, "learning_rate": 3.6805555555555556e-05, "epoch": 2.4993630573248407, "step": 247}
+{"loss": 0.6151189804077148, "grad_norm": 1.6180071830749512, "learning_rate": 3.611111111111111e-05, "epoch": 2.5095541401273884, "step": 248}
+{"loss": 0.17855669558048248, "grad_norm": 1.5453747510910034, "learning_rate": 3.541666666666667e-05, "epoch": 2.519745222929936, "step": 249}
+{"loss": 0.1135454773902893, "grad_norm": 0.9661981463432312, "learning_rate": 3.472222222222222e-05, "epoch": 2.529936305732484, "step": 250}
+{"loss": 0.25576266646385193, "grad_norm": 1.7761435508728027, "learning_rate": 3.402777777777778e-05, "epoch": 2.5401273885350317, "step": 251}
+{"loss": 0.18084122240543365, "grad_norm": 1.3050564527511597, "learning_rate": 3.3333333333333335e-05, "epoch": 2.5503184713375795, "step": 252}
+{"loss": 0.10597167909145355, "grad_norm": 1.0448472499847412, "learning_rate": 3.263888888888889e-05, "epoch": 2.5605095541401273, "step": 253}
+{"loss": 0.2155171036720276, "grad_norm": 1.7620513439178467, "learning_rate": 3.194444444444444e-05, "epoch": 2.570700636942675, "step": 254}
+{"loss": 0.2891685366630554, "grad_norm": 1.3910305500030518, "learning_rate": 3.125e-05, "epoch": 2.580891719745223, "step": 255}
+{"loss": 0.201302632689476, "grad_norm": 2.164580821990967, "learning_rate": 3.055555555555556e-05, "epoch": 2.5910828025477706, "step": 256}
+{"loss": 0.138000026345253, "grad_norm": 1.2483196258544922, "learning_rate": 2.9861111111111113e-05, "epoch": 2.6012738853503183, "step": 257}
+{"loss": 0.14248836040496826, "grad_norm": 1.2548537254333496, "learning_rate": 2.916666666666667e-05, "epoch": 2.611464968152866, "step": 258}
+{"loss": 0.13530048727989197, "grad_norm": 1.4316082000732422, "learning_rate": 2.8472222222222223e-05, "epoch": 2.621656050955414, "step": 259}
+{"loss": 0.14262661337852478, "grad_norm": 1.9074763059616089, "learning_rate": 2.777777777777778e-05, "epoch": 2.6318471337579616, "step": 260}
+{"loss": 0.10967038571834564, "grad_norm": 1.0953608751296997, "learning_rate": 2.7083333333333332e-05, "epoch": 2.6420382165605094, "step": 261}
+{"loss": 0.19146840274333954, "grad_norm": 1.233297348022461, "learning_rate": 2.6388888888888892e-05, "epoch": 2.652229299363057, "step": 262}
+{"loss": 0.1155640184879303, "grad_norm": 1.5569449663162231, "learning_rate": 2.5694444444444445e-05, "epoch": 2.662420382165605, "step": 263}
+{"loss": 0.1584577113389969, "grad_norm": 1.5085536241531372, "learning_rate": 2.5e-05, "epoch": 2.6726114649681527, "step": 264}
+{"loss": 0.1578441858291626, "grad_norm": 1.3990135192871094, "learning_rate": 2.4305555555555558e-05, "epoch": 2.6828025477707005, "step": 265}
+{"loss": 0.05720777064561844, "grad_norm": 0.7964500188827515, "learning_rate": 2.361111111111111e-05, "epoch": 2.6929936305732483, "step": 266}
+{"loss": 0.2078534960746765, "grad_norm": 1.3433589935302734, "learning_rate": 2.2916666666666667e-05, "epoch": 2.703184713375796, "step": 267}
+{"loss": 0.1210533082485199, "grad_norm": 0.6911096572875977, "learning_rate": 2.2222222222222223e-05, "epoch": 2.713375796178344, "step": 268}
+{"loss": 0.12541314959526062, "grad_norm": 1.0665966272354126, "learning_rate": 2.152777777777778e-05, "epoch": 2.7235668789808916, "step": 269}
+{"loss": 0.14327681064605713, "grad_norm": 1.2878247499465942, "learning_rate": 2.0833333333333336e-05, "epoch": 2.7337579617834393, "step": 270}
+{"loss": 0.10486380755901337, "grad_norm": 1.544772744178772, "learning_rate": 2.013888888888889e-05, "epoch": 2.743949044585987, "step": 271}
+{"loss": 0.11306212097406387, "grad_norm": 1.1754292249679565, "learning_rate": 1.9444444444444445e-05, "epoch": 2.754140127388535, "step": 272}
+{"loss": 0.16642490029335022, "grad_norm": 1.5518449544906616, "learning_rate": 1.8750000000000002e-05, "epoch": 2.7643312101910826, "step": 273}
+{"loss": 0.08535230159759521, "grad_norm": 0.9602029919624329, "learning_rate": 1.8055555555555555e-05, "epoch": 2.7745222929936304, "step": 274}
+{"loss": 0.16168959438800812, "grad_norm": 1.1414203643798828, "learning_rate": 1.736111111111111e-05, "epoch": 2.784713375796178, "step": 275}
+{"loss": 0.2000589668750763, "grad_norm": 1.3773179054260254, "learning_rate": 1.6666666666666667e-05, "epoch": 2.794904458598726, "step": 276}
+{"loss": 0.14879369735717773, "grad_norm": 1.229598879814148, "learning_rate": 1.597222222222222e-05, "epoch": 2.8050955414012737, "step": 277}
+{"loss": 0.1351848542690277, "grad_norm": 1.2507323026657104, "learning_rate": 1.527777777777778e-05, "epoch": 2.8152866242038215, "step": 278}
+{"loss": 0.10965745896100998, "grad_norm": 1.3401415348052979, "learning_rate": 1.4583333333333335e-05, "epoch": 2.8254777070063692, "step": 279}
+{"loss": 0.1449526995420456, "grad_norm": 1.3561450242996216, "learning_rate": 1.388888888888889e-05, "epoch": 2.835668789808917, "step": 280}
+{"loss": 0.16457107663154602, "grad_norm": 1.5929303169250488, "learning_rate": 1.3194444444444446e-05, "epoch": 2.845859872611465, "step": 281}
+{"loss": 0.2250441014766693, "grad_norm": 1.1919002532958984, "learning_rate": 1.25e-05, "epoch": 2.8560509554140125, "step": 282}
+{"loss": 0.19434526562690735, "grad_norm": 1.449803352355957, "learning_rate": 1.1805555555555555e-05, "epoch": 2.8662420382165603, "step": 283}
+{"loss": 0.32755422592163086, "grad_norm": 1.6384215354919434, "learning_rate": 1.1111111111111112e-05, "epoch": 2.876433121019108, "step": 284}
+{"loss": 0.10235312581062317, "grad_norm": 0.8350539803504944, "learning_rate": 1.0416666666666668e-05, "epoch": 2.886624203821656, "step": 285}
+{"loss": 0.16005685925483704, "grad_norm": 1.0850409269332886, "learning_rate": 9.722222222222223e-06, "epoch": 2.8968152866242036, "step": 286}
+{"loss": 0.10673708468675613, "grad_norm": 1.3893229961395264, "learning_rate": 9.027777777777777e-06, "epoch": 2.9070063694267514, "step": 287}
+{"loss": 0.21009820699691772, "grad_norm": 1.7795078754425049, "learning_rate": 8.333333333333334e-06, "epoch": 2.917197452229299, "step": 288}
+{"loss": 0.15081869065761566, "grad_norm": 1.1663967370986938, "learning_rate": 7.63888888888889e-06, "epoch": 2.927388535031847, "step": 289}
+{"loss": 0.1252090483903885, "grad_norm": 1.08909273147583, "learning_rate": 6.944444444444445e-06, "epoch": 2.9375796178343947, "step": 290}
+{"loss": 0.09439841657876968, "grad_norm": 1.253583550453186, "learning_rate": 6.25e-06, "epoch": 2.9477707006369425, "step": 291}
+{"loss": 0.15273809432983398, "grad_norm": 1.485877513885498, "learning_rate": 5.555555555555556e-06, "epoch": 2.9579617834394902, "step": 292}
+{"loss": 0.11691146343946457, "grad_norm": 1.0819978713989258, "learning_rate": 4.861111111111111e-06, "epoch": 2.968152866242038, "step": 293}
+{"loss": 0.1469116359949112, "grad_norm": 1.0861661434173584, "learning_rate": 4.166666666666667e-06, "epoch": 2.9783439490445858, "step": 294}
+{"loss": 0.09182402491569519, "grad_norm": 1.151098608970642, "learning_rate": 3.4722222222222224e-06, "epoch": 2.9885350318471335, "step": 295}
+{"loss": 0.13682223856449127, "grad_norm": 1.2304149866104126, "learning_rate": 2.777777777777778e-06, "epoch": 2.9987261146496813, "step": 296}
+{"loss": 0.08698958903551102, "grad_norm": 2.3549983501434326, "learning_rate": 2.0833333333333334e-06, "epoch": 3.0, "step": 297}
+{"train_runtime": 1413.2282, "train_samples_per_second": 1.666, "train_steps_per_second": 0.21, "total_flos": 5.328270499398451e+16, "train_loss": 0.3817934171243348, "epoch": 3.0, "step": 297}