TokenBender/glm47-synth-v1-reproducibility

Synth v1 reproducibility bundle

This gated, public repository is the complete reproducibility package for the GLM-4.7-Flash Synth v1 experiment and its epoch-50 fixed-26 result.

Results

• Epoch-50 strict first-turn scores: 10/26, 9/26, 10/26, 9/26
• Mean strict Pass@1: 9.5/26
• Evaluation samples: 104 (4 trials x 26 tasks)
• Training corpus: 260 rows
• Training: 100 epochs, LoRA rank 16, alpha 32, 1,300 updates

Contents

• dataset/: the exact 260-row training corpus, audit, row catalog, and manifests.
• source/: a Git archive of training source commit 6188070622895021d1c340ad31939e888c514396.
• checkpoint/epoch-50-canonical-adapter/: the canonical epoch-50 adapter.
• checkpoint/epoch-50-serving-adapter/: its evaluation-serving conversion and receipt.
• checkpoint/full-checkpoint-series-manifest.json: hashes and locations for all 20 saved checkpoints.
• evaluation/: all 104 retained samples, task receipts, aggregate receipts, evaluator source, and the exact four-trial plan.
• training/: training log, admission gate, checkpoint/artifact manifests, VRAM trace, and run receipt.
• wandb/: 10,400 scalar-history rows plus all 27 W&B run files, including configuration, summary, metadata, console logs, rollout tables, checkpoint tables, and artifact manifests.
• reproduce/: commands for staging artifacts, rerunning evaluation, and launching the original training configuration.

This package does not publish hidden benchmark tests or reference answers. The evaluator obtains the pinned public Aider/Polyglot sources and runs tests in the isolated scorer environment.

Verify

./reproduce/verify.sh

Reproduce evaluation

Install and authenticate the Modal CLI, then run:

./reproduce/stage_modal_eval.sh
./reproduce/run_eval.sh

Reproduce training

The training command uses the exact archived source, dataset, and original hyperparameters:

./reproduce/train.sh

Canonical references

• Full checkpoint archive: https://huggingface.co/TokenBender/glm47-synth-v1-100ep
• Dataset repository: https://huggingface.co/datasets/TokenBender/glm47-synth-v1-dataset
• Evaluation archive: https://huggingface.co/datasets/TokenBender/glm47-synth-v1-fixed26-evals
• W&B run: https://wandb.ai/ahm-rimer/glm47-aider-cpp-sft/runs/glm47-synth-memorization-v1-100ep-20260731T071000Z
• Exact public source commit: https://github.com/tokenbender/browser-is-all-you-need/commit/6188070622895021d1c340ad31939e888c514396
• Client release branch: https://github.com/tokenbender/browser-is-all-you-need/tree/client/5-aug-release

Legacy synth-memorization strings inside immutable run IDs and historical receipts are preserved intentionally so their hashes remain verifiable.