TimeCapsuleLLM-ja-v1 / phase3_evaluation_review.json
trtd56's picture
Release phase3 staging
4f43093 verified
Raw
History Blame Contribute Delete
1.85 kB
{
"gate_passed": true,
"phase2_historical_bpb": 1.4102684308437439,
"phase3_historical_bpb": 1.0653798981287268,
"required_max_historical_bpb": 1.3397550093015567,
"historical_bpb_improvement_rate_over_phase2": 0.24455480416,
"phase3_modern_control_bpb": 1.8756087153420742,
"historical_lower_than_modern": true,
"automatic_rubric_pass_rate": 0.9333333333333333,
"manual_rubric_pass_rate": 0.9,
"manual_review": {
"reviewed_generations": 5,
"checks": 10,
"passed_checks": 9,
"empty_outputs": 0,
"mojibake_outputs": 0,
"unbounded_repetition_outputs": 0,
"historical_style_outputs": 5,
"severely_broken_outputs": 0,
"noted_issue": "The 人工知能 probe is flagged no_mojibake=false by the automatic check, but the flagged characters are period old-kana renderings (存すへき=存すべき, 得へし), not true mojibake; the 文明開化 probe has elevated but bounded repetition (0.16)."
},
"training": {
"iterations": 11445,
"processed_tokens": 3000238080,
"epochs_over_train": 2.72,
"best_validation_loss": 4.149158000946045,
"elapsed_hours": 18.29,
"estimated_cost_usd": 18.11,
"peak_vram_bytes": 44345157632,
"gpu_total_bytes": 47803596800
},
"final_checkpoint_sha256": "1f2459a1b5b5c75c6e7c07251b877f6c804500753bb0121b13351933b8a07b03",
"tokenizer_sha256": "2c014c2f018dd70ab48d66b59e20c9afc12c5a0dd7b5bbdd48d7b849c4bf1d42",
"vocab_size": 16384,
"parameter_count": 345035776,
"review_date": "2026-07-17",
"notes": [
"Phase-3 historical test BPB 1.0654 beats the phase-2 gate (<= 1.3398, i.e. 5% better than phase-2's 1.4103) by a 24.5% improvement and is well below the modern control (1.8756).",
"Single-pass full run cleared the gate; no continued-training round was needed (unlike phase-2's 1,600-step shortfall)."
]
}