Identifiable Latent Rubrics

Completed local research evidence: 67 synthetic fits and 180 real fits, covering 36 real configurations with five seeds each. The engine completed all feasible required experiments; reporting includes the actual verifier substitution, negative results, raw-array checks and final confidence intervals.

Directional interventions identify axes in the favorable synthetic construction. On HelpSteer2, ILR improves sensitivity to generated edits but does not establish useful human semantic identification or better rubric composition. At the predeclared Qwen3-32B block 48 mean representation, composition falls from 0.6061 ± 0.0068 to 0.5667 ± 0.0437, paired difference −0.0395, 95% CI [−0.0808, −0.0028]. The 14B replication also declines. These intervals condition on the saved evaluation pipeline; no preference-equivalence claim is made.

• Final evidence report: theory, methods, results, actual failures, supported/unsupported claims and prioritized missing experiments.
• Paper tables and reproducibility appendix: mean ± SD, key 95% intervals, exact metrics, all layer/pooling/architecture/budget/quality results and model revisions.
• Main table, criterion results, example rubrics, failure analysis, all ablations.
• Synthetic report, dataset report, metric definitions, theory and related work.
• Engine completion, handoff, 180-run inventory, final validation, deep raw-array checks.
• Statistics handoff, historical audit, final audit reconciliation, figure inspection.
• API budget protocol, final reported usage and unknown reserves, engine decisions.

Four named criteria are correctness, coherence, complexity and verbosity, plus a residual in primary linear K5 heads. Helpfulness is descriptive, not a fifth atomic axis. Main N=50 edits per criterion was chosen solely from frozen clean availability before outcomes; feasible common budgets are 10/25/50. Dense Attribute+BT is an empirical reference with additional human semantic supervision, not a guaranteed ceiling.

GLM-5.3 generated 2,400 candidates; local pinned BF16 Qwen3-32B verified all 1,431 feasible edits after requested DeepSeek-V4.1-Flash failed to provide research labels. The verifier shares primary backbone weights. No Pro research labels were used. Strict heldout counts are 31/19/12/53; no strict coherence+ edit exists. The planning account is CNY 51.099842, comprising CNY 48.047082 known-usage estimate and CNY 3.052760 reserves for 36 unknown attempts, below the CNY 200 cap. Provider rates are unverified; this is not an invoice. ChatGPT quota, local verification tokens and pre-ledger connectivity smokes are separately disclosed.

Rebuild from saved fits, CPU only

Run from /home/ningyuxi/workspace/latent-rubric. Large source data, feature caches, fitted weights and trajectories are local ignored artifacts; retain them alongside this repository. The actual environment is documented in environment, full lock and direct requirements. The project .venv uses Python 3.10 and inherits working packages read-only from /opt/anaconda3/envs/drkernel.

Final bootstrap has already been run once on the completed arrays. Rebuild it when intentionally reproducing or after relevant artifact changes; do not repeatedly bootstrap merely while waiting for an engine. These commands use no API and no GPU:

cd /home/ningyuxi/workspace/latent-rubric
test -f outputs/ENGINE_DONE.json
export CUDA_VISIBLE_DEVICES=''
export OPENBLAS_NUM_THREADS=2 OMP_NUM_THREADS=2
export PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=.
export MPLCONFIGDIR="$PWD/reports/reporter/mpl"
MAIN_N=$(.venv/bin/python -c 'import json; print(json.load(open("outputs/main_design.json"))["main_n"])')
.venv/bin/python -m analysis.real_report --main-n "$MAIN_N" \
  --bootstrap --semantic-bootstrap-samples 1000
.venv/bin/python -m analysis.api_usage

This generates main_table, confidence_intervals, criterion_results, example_rubrics, failure_analysis and real figures from existing saved evaluations. The exact rubric-first composition statistic is an equal mean of each rubric's valid-pair accuracy, followed by a seed mean. Semantic/stability intervals condition on fitted seeds; composition also conditions on TEST scales, fixed rubric weights and DEV mapping.

Finalize publication layouts and preserve distinct reduced-criterion ablation groups. The real layout wrapper invokes existing analysis without repeating bootstrap. Its numerical inputs are unchanged. Run the group correction after it:

.venv/bin/python reports/reporter/real_publication_figures.py
.venv/bin/python reports/reporter/refresh_auxiliary_tables.py
.venv/bin/python reports/reporter/synthetic_publication_figures.py
.venv/bin/python reports/reporter/intervention_diagnostics.py
.venv/bin/python reports/reporter/selectivity_distributions.py
.venv/bin/python reports/reporter/reduced_reference.py
.venv/bin/python reports/reporter/collect_final_evidence.py
.venv/bin/python reports/reporter/build_paper.py --main-n "$MAIN_N" --final
.venv/bin/python reports/reporter/finalize_narrative.py
.venv/bin/python reports/reporter/deep_checks.py
.venv/bin/python reports/reporter/validate_artifacts.py --main-n "$MAIN_N" --final

The reduced comparator uses existing K4 predictions and frozen DEV matching to criteria 0/1/3, without fitting another model. It reports a separate exploratory projected-rubric endpoint. Synthetic figure regeneration uses existing analysis with a nonnegative budget axis. Human visual/failure review and content hashes accompany the final artifacts; regeneration alone is not a completion certificate.

Current non-audit/statistics suite: 82 passed. Reproduce on CPU:

.venv/bin/python -m pytest tests/test_*.py tests/thread_statistics \
  -q -p no:cacheprovider --basetemp=outputs/reporter/pytest \
  --junitxml=outputs/reporter/final_pytest.xml

Historical audit tests intentionally characterize preserved old defects; their passes are not current green regression evidence. Engine's 22 focused tests overlap the current suite and should not be summed as independent tests.

Durable continuation and experiment ownership

The user-authorized engine and reporter run in existing ChatGPT-login Codex threads under tmux; ordinary terminal disconnection does not end them. Inspect recorded status without creating duplicate workers:

cat outputs/ENGINE_DONE.json
cat reports/engine_handoff.md
cat reports/reporter/status.md
tail -n 10 outputs/threads/engine_events.jsonl
tmux attach -t latent-rubric-engine
# Detach with Ctrl-b d; reporter session:
tmux attach -t latent-rubric-reporter

The original engine task, configs and saved per-run config.json, fit_spec.json, source snapshots, learning curves and evaluated arrays document actual execution. Preserve the frozen intervention table and caches. A changed objective/data/version belongs in a new explicitly identified experiment; do not relabel or overwrite existing results. The engine owns API/GPU/model/data execution, the reporter uses CPU saved-output analysis, and root owns Git integration. Reporter made no API/GPU calls or Git commits.

For an authorized engine/root continuation, the existing restartable script checks completed fit specifications and reuses valid fits. It requires the project's dedicated GPU reservation/locking workflow. In an engine shell—not the CPU reporting shell—the existing GPU wrapper binds physical GPU1 UUID GPU-2a9c799b-ab9e-c677-db0e-379f480cb00f, pauses its own 110 GiB idle reservation and restores it after the job. Baseline launcher, synthetic launcher, real ablation chain and 14B configuration retain the executable chain; no rerun is needed for this completed report. Do not start duplicate extraction or mutate the frozen table under existing fits.

Completion and remaining work

outputs/ENGINE_DONE.json records all feasible core experiments complete. outputs/threads/REPORTER_DONE.json records local final reporting/validation only after its checks finish. Root can review and correct the package afterward. The existing external Wiki is an earlier published snapshot; this CPU-only reporter did not synchronize it.

Still unrun: independent human edit adjudication, a successful independent external-verifier research cohort, balanced coherence directions, larger controlled human subsets, other datasets/preference contexts, inductive scale calibration/full-pipeline inference, a prespecified equivalence test, clean common N100/250, a full 14B head sweep, SAE and LoRA. Completion of this evidence package does not turn those missing experiments into results or settle provider invoice cost.