S500 Baseline Record
Run Identity
- Name: frozen baseline38 GPT5.4 full10
- Date: 2026-05-25
- Local package:
tmcra_s500_baseline38_release_20260525 - Result archive:
results/lme_s500_frozen_baseline38_full10_20260525_results.tar.gz - Judge output:
results/judge_gpt4o_alias_vectorengine.jsonl - Summary:
results/judge_gpt4o_alias_vectorengine.jsonl.summary.json
Metrics
- Evaluated samples: 500
- Correct: 310
- Accuracy: 62.00%
- Judge alias:
gpt-4o - Judge resolved model:
gpt-4o-2024-08-06
Subtask Metrics
| subtask | accuracy | count |
|---|---|---|
| knowledge-update | 70.51% | 78 |
| multi-session | 39.85% | 133 |
| single-session-assistant | 78.57% | 56 |
| single-session-preference | 56.67% | 30 |
| single-session-user | 81.43% | 70 |
| temporal-reasoning | 63.16% | 133 |
Interpretation
This baseline is useful as a frozen comparison point. Stronger areas are single-session direct user facts, assistant detail recall, and knowledge-update. Weak areas remain multi-session aggregation, preference abstraction, and temporal reasoning.
The full model output directory is included to preserve all runtime weights and training checkpoints, not only deployment weights.