File size: 18,703 Bytes
3dfc93b
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
"""run λ²ˆλ“€ ν•˜λ‚˜ β†’ `proofread_diff@2` 채점 λ²ˆλ“€ (κ³„νš 2026-09-26 L3b Β§2.4Β·D5).

μˆœμ„œ: subject verify β†’ μƒμ‚°μž νŒλ³„(`is_legacy_results_producer`) β†’ `results.jsonl` λ ˆμ½”λ“œ 쀄 μˆœμ„œλŒ€λ‘œ 읽기 β†’ ν–‰λ§ˆλ‹€ 채점(순수
λͺ¨λ“ˆ) β†’ νŒμ • μ΄ˆμ•ˆΒ·μƒμ‚°μž 집계 β†’ 같은 `evaluation_run_id` 의 κΈ°μ‘΄ μ΅œμ‹ κ³Ό 비ꡐ β†’ μƒˆ λ²ˆλ“€ μ“°κΈ°(`write_scoring_bundle`, subject
λ₯Ό λ„˜κ²¨ E1Β·E2 κΉŒμ§€ 컀밋 전에 λ³Έλ‹€) β†’ (`mirror_root` κ°€ 있으면) `mirror_bundle(area="evaluations")`.

- `evaluation_run_id` = `code_rescore_run_id(subject bundle_id, evaluator_config_hash)` β€” subject λ²ˆλ“€μ΄ λ°”λ€Œλ©΄ μƒˆ id λ‹€.
- **λ©±λ“±(D5).** κΈ°μ‘΄ μ΅œμ‹ (같은 id)이 subject 와 ν•¨κ»˜ `verify_scoring_bundle` 을 ν†΅κ³Όν•˜κ³ (E4 둜 `evaluator_config.json` 파일의
  config ν•΄μ‹œ = λ§€λ‹ˆνŽ˜μŠ€νŠΈ κ°’κΉŒμ§€ λ³Έλ‹€), 평가기 ν•΄μ‹œκ°€ κ°™κ³ , `producer_aggregates.json` λ°”μ΄νŠΈκ°€ κ°™κ³ , `judgments.jsonl` ν–‰μ—μ„œ
  `created_at`(μž¬μ±„μ  μ‹œκ°)을 λΊ€ λͺ©λ‘μ΄ μƒˆ λͺ©λ‘κ³Ό κ°™μœΌλ©΄ μƒˆ λ²ˆλ“€μ„ μ“°μ§€ μ•Šκ³  κΈ°μ‘΄ 경둜λ₯Ό `exists` 둜 λŒλ €μ€€λ‹€. ν•˜λ‚˜λΌλ„
  λ‹€λ₯΄λ©΄(κΈ°μ‘΄ μ΅œμ‹ μ΄ κΉ¨μ‘Œμ–΄λ„) `supersedes` = κΈ°μ‘΄ μ΅œμ‹ μœΌλ‘œ μƒˆ ULID λ²ˆλ“€μ„ μ“΄λ‹€ β€” μƒˆ ULID ≀ 기쑴이면 `clock_skew`(eval-wrapΒ·
  wrap κ³Ό 같은 κ·œμΉ™). 비ꡐ가 id μˆœμ„œ 검사보닀 λ¨Όμ €λ‹€. μƒμ‚°μž μ§‘κ³„λŠ” 폐포(`impl_hash`) λ°–μ˜ λ ˆκ±°μ‹œ 집계 μ½”λ“œκ°€ λ‚΄λ―€λ‘œ νŒμ •λ§Œ
  보면 κ·Έ μ½”λ“œκ°€ 바뀐 것을 λ†“μΉœλ‹€.
- **subject 경계.** `manifest.json` 이 μ—†λŠ” λ””λ ‰ν† λ¦¬λŠ” `not_a_run_bundle` 였λ₯˜(λ²ˆλ“€μ΄ μ•„λ‹ˆλ‹€ β€” 경둜 μ‹€μˆ˜), verify μ‹€νŒ¨λŠ”
  `latest_bundle_invalid` skip(`items.jsonl` 이 κΉ¨μ§„ 경우 포함 β€” case λͺ¨λ“œμ˜ 채택과 같은 λΆ„λ₯˜).
- 좜λ ₯·였λ₯˜ λ©”μ‹œμ§€μ—λŠ” idΒ·μ½”λ“œΒ·κ²½λ‘œλ§Œ μ‹£λŠ”λ‹€. μž…λ ₯·좜λ ₯Β·μ •λ‹΅ 원문은 어디에도 μ“°μ§€ μ•ŠλŠ”λ‹€(census D-04 와 같은 κ·œμΉ™).
"""

from __future__ import annotations

import re
import shutil
import tempfile
from collections.abc import Mapping, Sequence
from dataclasses import dataclass
from datetime import UTC, datetime
from pathlib import Path
from typing import Any, Literal

from poc_bundle.adopt import AdoptError, Committed, committed_scoring_bundles, latest
from poc_bundle.bundle import verify_bundle
from poc_bundle.canonical import load_json_strict
from poc_bundle.jsonl import iter_record_lines
from poc_bundle.legacy_run import is_legacy_results_producer
from poc_bundle.mirror import MirrorError, mirror_bundle
from poc_bundle.scoring.ids import code_rescore_run_id, evaluator_config_hash
from poc_bundle.scoring.verify import RunBundleRef, load_run_bundle_ref, verify_scoring_bundle
from poc_bundle.scoring.writer import JudgmentDraft, ScoringWriteError, write_scoring_bundle
from ulid import ULID

from solar_eval._version import __version__
from solar_eval.rescore.drafts import ScoredRow, judgment_drafts, producer_aggregates
from solar_eval.rescore.evaluator_doc import current_evaluator_document
from solar_eval.rescore.paths import EVALUATIONS_AREA, RescorePathError, mirror_area_dir
from solar_eval.rescore.proofread_diff_v2 import has_original, score_row

PRODUCER_NAME = "solar-eval score"
PURPOSE = "rescore_after_fix"
RESULTS_FILE = "results.jsonl"
JUDGMENTS_FILE = "judgments.jsonl"
AGGREGATES_FILE = "producer_aggregates.json"
#: λ©±λ“± λΉ„κ΅μš© μž„μ‹œ λ²ˆλ“€ id β€” κ°€μž₯ 큰 ULID 라 μ–΄λ–€ `supersedes` 보닀도 뒀에 μ˜¨λ‹€. 좜λ ₯ λ””λ ‰ν† λ¦¬μ—λŠ” μ“°μ§€ μ•ŠλŠ”λ‹€.
STAGING_BUNDLE_ID = "7ZZZZZZZZZZZZZZZZZZZZZZZZZ"

# run λ‹¨μœ„ skip μ½”λ“œ(μ’…λ£Œ 0 에 포함)와 였λ₯˜ μ½”λ“œ(μ’…λ£Œ 1) β€” κ³„νš Β§2.2.
SKIP_LATEST_INVALID = "latest_bundle_invalid"
SKIP_UNKNOWN_FORMAT = "unknown_results_format"
SKIP_NO_RESULTS = "no_results"
ERR_ADOPT = "adopt_unreadable_manifest"
ERR_CLOCK_SKEW = "clock_skew"
ERR_MIRROR = "mirror_failed"
ERR_RESULTS = "results_unreadable"
ERR_OUT_INSIDE = "out_inside_bundle"
ERR_NOT_A_BUNDLE = "not_a_run_bundle"
WRITE_FAILED_PREFIX = "write_failed:"
_RULE_ID = re.compile(r"^\[([A-Z]+[0-9]*)\]")

Status = Literal["written", "exists", "skip"]


class RescoreError(RuntimeError):
    """이 run 을 μž¬μ±„μ ν•  수 μ—†λ‹€(μ’…λ£Œ 1). `code` λŠ” Β§2.2 였λ₯˜ μ–΄νœ˜, `run_id` λŠ” μ•Œλ©΄ μ‹£λŠ”λ‹€."""

    def __init__(self, code: str, message: str, run_id: str | None = None) -> None:
        super().__init__(f"{code}: {message}")
        self.code = code
        self.run_id = run_id


@dataclass(frozen=True)
class RescoreOutcome:
    """run ν•˜λ‚˜μ˜ κ²°κ³Ό β€” `written`Β·`exists` λŠ” λ²ˆλ“€ 경둜, `skip` 은 μ½”λ“œ."""

    status: Status
    run_id: str | None
    path: Path | None = None
    code: str | None = None
    n_original_missing: int = 0


class _Skip(Exception):
    def __init__(self, code: str) -> None:
        super().__init__(code)
        self.code = code


@dataclass(frozen=True)
class _Subject:
    bundle_dir: Path
    run_id: str
    bundle_id: str
    case: dict[str, Any]
    ref: RunBundleRef


def rescore_run_bundle(
    run_bundle_dir: Path,
    out_dir: Path | None = None,
    *,
    mirror_root: Path | None = None,
    now: datetime | None = None,
    bundle_id: str | None = None,
    evaluator: Mapping[str, Any] | None = None,
    repo_root: Path | None = None,
) -> RescoreOutcome:
    """run λ²ˆλ“€ ν•˜λ‚˜λ₯Ό μž¬μ±„μ ν•œλ‹€. `out_dir`(채점 λ²ˆλ“€λ“€μ˜ λΆ€λͺ¨) 와 `mirror_root` 쀑 **μ •ν™•νžˆ ν•˜λ‚˜**λ₯Ό μ€€λ‹€.

    Args:
        mirror_root: 미러 λ²„ν‚·μ˜ λΆ€λͺ¨ β€” μ£Όλ©΄ 좜λ ₯ μ˜μ—­μ€ subject `case` 의 미러 `evaluations` 이고 `mirror_bundle` 둜 λ†“λŠ”λ‹€.
        now Β· bundle_id: μž¬ν˜„ κ°€λŠ₯ν•œ ν…ŒμŠ€νŠΈ μ „μš©. λΉ„μš°λ©΄ μ§€κΈˆ μ‹œκ°Β·μƒˆ ULID.
        evaluator: 평가기 λ¬Έμ„œ(case 전체에 ν•œ 번 뜬 것). λΉ„μš°λ©΄ μ§€κΈˆ λ””μŠ€ν¬μ˜ κ΅¬ν˜„μœΌλ‘œ λœ¬λ‹€.
        repo_root: `mirror_bundle` 의 레포 λ°– 검사 κΈ°μ€€(κΈ°λ³Έ: 레포λ₯Ό μ°Ύμ•„ λ³Έλ‹€).

    Raises:
        RescoreError: Β§2.2 였λ₯˜(채택 μ˜μ—­ λ§€λ‹ˆνŽ˜μŠ€νŠΈΒ·μ‹œκ³„ μ—­ν–‰Β·μ“°κΈ°Β·λ―ΈλŸ¬Β·results 읽기).
    """
    if (out_dir is None) == (mirror_root is None):
        raise ValueError("give exactly one of out_dir / mirror_root")
    try:
        subject = _load_subject(Path(run_bundle_dir))
        rows, n_missing = _scored_rows(subject)
    except _Skip as skip:
        run_id = _run_id_or_none(Path(run_bundle_dir))
        return RescoreOutcome("skip", run_id, code=skip.code)
    doc = dict(evaluator) if evaluator is not None else current_evaluator_document()
    at = _utc_seconds(now)
    drafts = judgment_drafts(subject.run_id, rows, at)
    target = out_dir if out_dir is not None else _mirror_evaluations(mirror_root, subject)
    _check_out_dir(Path(target), subject)
    plan = _Plan(subject, doc, drafts, producer_aggregates(rows, drafts), at)
    status, path = _commit(plan, Path(target), bundle_id, mirror_root, repo_root)
    return RescoreOutcome(status, subject.run_id, path=path, n_original_missing=n_missing)


# ── 읽기 ─────────────────────────────────────────────────────────────────────


def _load_subject(bundle_dir: Path) -> _Subject:
    ref = _subject_ref(bundle_dir)
    report = ref.verify_report
    if not report.is_bundle:
        raise RescoreError(ERR_NOT_A_BUNDLE, f"no manifest.json in {bundle_dir}")
    if not report.ok or report.manifest is None:
        raise _Skip(SKIP_LATEST_INVALID)
    manifest = report.manifest
    if not is_legacy_results_producer(manifest.producer.name):
        raise _Skip(SKIP_UNKNOWN_FORMAT)
    return _Subject(
        bundle_dir=bundle_dir,
        run_id=manifest.run_id,
        bundle_id=manifest.bundle_id,
        case=manifest.case.model_dump(mode="json"),
        ref=ref,
    )


def _subject_ref(bundle_dir: Path) -> RunBundleRef:
    """`RunBundleRef` β€” `items.jsonl` 을 λͺ» 읽으면 verify 결과둜 κ°€λ₯Έλ‹€: μ‹€νŒ¨λ©΄ skip, 톡과인데 λͺ» 읽으면 였λ₯˜.

    `load_run_bundle_ref` λŠ” verify 결과와 λ¬΄κ΄€ν•˜κ²Œ items νŒŒμ‹± 였λ₯˜λ₯Ό λ˜μ§€λ―€λ‘œ, κ·ΈλŒ€λ‘œ 두면 κΉ¨μ§„ λ²ˆλ“€μ΄ case λͺ¨λ“œ(채택이
    verify λ₯Ό λ¨Όμ € λ³Έλ‹€)와 달리 `results_unreadable` 둜 λΆ„λ₯˜λœλ‹€.
    """
    try:
        return load_run_bundle_ref(bundle_dir)
    except ValueError as exc:
        if not verify_bundle(bundle_dir).ok:
            raise _Skip(SKIP_LATEST_INVALID) from exc
        raise RescoreError(ERR_RESULTS, f"items.jsonl unreadable in {bundle_dir.name}") from exc


def _scored_rows(subject: _Subject) -> tuple[list[ScoredRow], int]:
    """results λ ˆμ½”λ“œ 쀄 μˆœμ„œλŒ€λ‘œ μ±„μ ν•œ ν–‰κ³Ό `original` 이 μ—†λŠ” ν–‰ 수. λ ˆμ½”λ“œκ°€ 0이면 `no_results`."""
    rows: list[ScoredRow] = []
    missing = 0
    path = subject.bundle_dir / RESULTS_FILE
    for line_no, raw in iter_record_lines(path):
        result = _parse_result(raw, line_no, subject)
        sample_idx = result["sample_idx"]
        input_key, golden_key = subject.ref.items[sample_idx]
        rows.append(ScoredRow(sample_idx, input_key, golden_key, score_row(result)))
        missing += 0 if has_original(result) else 1
    if not rows:
        raise _Skip(SKIP_NO_RESULTS)
    return rows, missing


def _parse_result(raw: bytes, line_no: int, subject: _Subject) -> Mapping[str, Any]:
    """λ ˆμ½”λ“œ ν•œ 쀄 β†’ 객체. λ©”μ‹œμ§€μ—λŠ” 쀄 번호만 μ‹£λŠ”λ‹€(ν–‰ λ‚΄μš© μ—†μŒ)."""
    where = f"{RESULTS_FILE}:{line_no} of run bundle {subject.bundle_id}"
    try:
        row = load_json_strict(raw, safe_integers=False)
    except (ValueError, RecursionError) as exc:
        raise RescoreError(ERR_RESULTS, f"{where} is not strict JSON", subject.run_id) from exc
    sample_idx = row.get("sample_idx") if isinstance(row, dict) else None
    if isinstance(sample_idx, bool) or not isinstance(sample_idx, int):
        raise RescoreError(ERR_RESULTS, f"{where} has no integer sample_idx", subject.run_id)
    if sample_idx not in subject.ref.items:
        raise RescoreError(ERR_RESULTS, f"{where} sample_idx is not in items.jsonl", subject.run_id)
    return row


def _run_id_or_none(bundle_dir: Path) -> str | None:
    """skip 쀄에 싀을 run_id β€” λ§€λ‹ˆνŽ˜μŠ€νŠΈλ₯Ό 읽을 수 μžˆμ„ λ•Œλ§Œ(검증 μ‹€νŒ¨μ—¬λ„)."""
    try:
        data = load_json_strict((bundle_dir / "manifest.json").read_bytes())
    except (OSError, ValueError, RecursionError):
        return None
    run_id = data.get("run_id") if isinstance(data, dict) else None
    return run_id if isinstance(run_id, str) else None


def _mirror_evaluations(mirror_root: Path | None, subject: _Subject) -> Path:
    try:
        return mirror_area_dir(
            mirror_root, subject.case["account"], subject.case["case"], EVALUATIONS_AREA
        )
    except RescorePathError as exc:
        raise RescoreError(ERR_MIRROR, str(exc), subject.run_id) from exc


def _check_out_dir(out: Path, subject: _Subject) -> None:
    target, run_abs = out.resolve(), subject.bundle_dir.resolve()
    if target == run_abs or run_abs in target.parents:
        raise RescoreError(
            ERR_OUT_INSIDE, f"output {target} is inside the run bundle", subject.run_id
        )


def _utc_seconds(now: datetime | None) -> str:
    moment = (now or datetime.now(UTC)).astimezone(UTC)
    return moment.strftime("%Y-%m-%dT%H:%M:%SZ")


# ── 멱등·컀밋 ─────────────────────────────────────────────────────────────────


@dataclass(frozen=True)
class _Plan:
    subject: _Subject
    evaluator: Mapping[str, Any]
    drafts: Sequence[JudgmentDraft]
    aggregates: Mapping[str, Any]
    created_at: str

    @property
    def config_hash(self) -> str:
        return evaluator_config_hash(self.evaluator)

    @property
    def evaluation_run_id(self) -> str:
        return code_rescore_run_id(self.subject.bundle_id, self.config_hash)


def _commit(
    plan: _Plan,
    out: Path,
    bundle_id: str | None,
    mirror_root: Path | None,
    repo_root: Path | None,
) -> tuple[Status, Path]:
    previous = _latest_committed(out, plan)
    if previous is not None and _same_as(previous, plan):
        return "exists", previous.path
    new_id = bundle_id or str(ULID())
    if previous is not None and previous.bundle_id >= new_id:
        raise RescoreError(
            ERR_CLOCK_SKEW,
            f"new bundle_id {new_id} is not newer than committed {previous.bundle_id}",
            plan.subject.run_id,
        )
    supersedes = None if previous is None else previous.bundle_id
    fields = _manifest_fields(plan, new_id, supersedes)
    if mirror_root is None:
        dest = out / new_id
        _write(dest, fields, plan)
        return "written", dest
    return "written", _write_and_mirror(fields, plan, new_id, Path(mirror_root), repo_root)


def _latest_committed(out: Path, plan: _Plan) -> Committed | None:
    try:
        committed = committed_scoring_bundles(out)
    except AdoptError as exc:
        raise RescoreError(ERR_ADOPT, str(exc), plan.subject.run_id) from exc
    cands = committed.get(plan.evaluation_run_id, [])
    return latest(cands) if cands else None


def _same_as(previous: Committed, plan: _Plan) -> bool:
    """D5 β€” κΈ°μ‘΄ μ΅œμ‹ μ΄ μ˜¨μ „ν•˜κ³  평가기 ν•΄μ‹œΒ·μƒμ‚°μž 집계·`created_at` 을 λΊ€ νŒμ • 행이 같은가.

    μžλ¦¬ν‘œμ‹œ id 둜 써 λ³Έ λ²ˆλ“€κ³Ό λΉ„κ΅ν•œλ‹€. κΈ°μ‘΄ μ΅œμ‹ μ΄ verify 에 μ‹€νŒ¨ν•˜λ©΄ κ°™μ§€ μ•Šλ‹€ β€” `exists` 둜 λ„˜κΈ°λ©΄ κΉ¨μ§„ λ²ˆλ“€μ΄ 영영
    μ΅œμ‹ μœΌλ‘œ λ‚¨λŠ”λ‹€(hub λŠ” μ΅œμ‹ μ΄ invalid λ©΄ μ΄μ „μœΌλ‘œ λ„˜μ–΄κ°€μ§€ μ•ŠλŠ”λ‹€).
    """
    evaluator_ref = previous.manifest.get("evaluator")
    if (
        not isinstance(evaluator_ref, Mapping)
        or evaluator_ref.get("config_hash") != plan.config_hash
        or not _previous_verifies(previous, plan)
    ):
        return False
    fields = _manifest_fields(plan, STAGING_BUNDLE_ID, previous.bundle_id)
    with tempfile.TemporaryDirectory(prefix="solar-eval-score-") as tmp:
        staged = Path(tmp) / STAGING_BUNDLE_ID
        _write(staged, fields, plan)
        new_rows = _rows_without_time(staged / JUDGMENTS_FILE)
        new_aggregates = _bytes_or_none(staged / AGGREGATES_FILE)
    old_rows = _rows_without_time(previous.path / JUDGMENTS_FILE)
    old_aggregates = _bytes_or_none(previous.path / AGGREGATES_FILE)
    return (
        old_rows is not None
        and old_rows == new_rows
        and old_aggregates is not None
        and old_aggregates == new_aggregates
    )


def _previous_verifies(previous: Committed, plan: _Plan) -> bool:
    """κΈ°μ‘΄ μ΅œμ‹ μ„ subject 와 ν•¨κ»˜ κ²€μ¦ν•œλ‹€ β€” SB1(파일 sha)Β·E4(`evaluator_config.json` config ν•΄μ‹œ)Β·E1Β·E2 κΉŒμ§€."""
    subject = plan.subject
    report = verify_scoring_bundle(previous.path, run_bundles={subject.bundle_id: subject.ref})
    return report.ok


def _bytes_or_none(path: Path) -> bytes | None:
    """파일 λ°”μ΄νŠΈ. λͺ» 읽으면 None(비ꡐ가 κ±°μ§“ β†’ μƒˆ λ²ˆλ“€)."""
    try:
        return path.read_bytes()
    except OSError:
        return None


def _rows_without_time(path: Path) -> list[dict[str, Any]] | None:
    """νŒμ • ν–‰μ—μ„œ `created_at` 을 λΊ€ λͺ©λ‘. λͺ» 읽으면 None(비ꡐ가 κ±°μ§“ β†’ μƒˆ λ²ˆλ“€)."""
    try:
        rows = [load_json_strict(raw) for _, raw in iter_record_lines(path)]
    except (OSError, ValueError, RecursionError):
        return None
    if not all(isinstance(row, dict) for row in rows):
        return None
    return [{k: v for k, v in row.items() if k != "created_at"} for row in rows]


def _manifest_fields(plan: _Plan, bundle_id: str, supersedes: str | None) -> dict[str, Any]:
    subject = plan.subject
    return {
        "bundle_id": bundle_id,
        "evaluation_run_id": plan.evaluation_run_id,
        "id_origin": "deterministic",
        "supersedes": supersedes,
        "created_at": plan.created_at,
        "producer": {"name": PRODUCER_NAME, "version": __version__},
        "case": subject.case,
        "subject": {
            "kind": "run",
            "runs": [{"run_id": subject.run_id, "bundle_id": subject.bundle_id, "role": "subject"}],
        },
        "purpose": PURPOSE,
        "retry_of": None,
        "golden_basis": "run",
        "status": "completed",
        "finished_at": plan.created_at,
        "reproducible": {"snapshot_complete": True, "missing": []},
        "known_defects": [],
        "observed": None,
        "usage": None,
    }


def _write(dest: Path, fields: Mapping[str, Any], plan: _Plan) -> None:
    try:
        write_scoring_bundle(
            dest,
            fields=fields,
            evaluator=plan.evaluator,
            drafts=plan.drafts,
            producer_aggregates=plan.aggregates,
            run_bundle=plan.subject.ref,
        )
    except ScoringWriteError as exc:
        _remove_uncommitted(dest)
        raise RescoreError(_write_code(exc.issues), str(exc), plan.subject.run_id) from exc
    except OSError as exc:
        _remove_uncommitted(dest)
        raise RescoreError(f"{WRITE_FAILED_PREFIX}io_error", str(exc), plan.subject.run_id) from exc


def _write_code(issues: Sequence[str]) -> str:
    """`write_failed:<κ·œμΉ™ id λͺ©λ‘>` β€” 이슈 λ¬Έμž₯의 `[<id>]` 머리만 λͺ¨μ€λ‹€(μ—†μœΌλ©΄ `input`)."""
    ids = sorted({m.group(1) for issue in issues if (m := _RULE_ID.match(issue))})
    return WRITE_FAILED_PREFIX + (",".join(ids) if ids else "input")


def _write_and_mirror(
    fields: Mapping[str, Any], plan: _Plan, new_id: str, mirror_root: Path, repo_root: Path | None
) -> Path:
    """μž„μ‹œ 디렉토리에 μ“΄ λ’€ λ―ΈλŸ¬μ— λ†“λŠ”λ‹€ β€” 미러의 컀밋 κ·œμΉ™(manifest λ§ˆμ§€λ§‰)·레포 λ°– 검사λ₯Ό `mirror_bundle` 이 λ§‘λŠ”λ‹€."""
    case = plan.subject.case
    with tempfile.TemporaryDirectory(prefix="solar-eval-score-") as tmp:
        staged = Path(tmp) / new_id
        _write(staged, fields, plan)
        try:
            return mirror_bundle(
                staged,
                account=case["account"],
                case=case["case"],
                area=EVALUATIONS_AREA,
                root=mirror_root,
                repo_root=repo_root,
            )
        except (MirrorError, OSError) as exc:
            raise RescoreError(ERR_MIRROR, str(exc), plan.subject.run_id) from exc


def _remove_uncommitted(staging: Path) -> None:
    """writer κ°€ manifest 없이 남긴 디렉토리λ₯Ό μ§€μš΄λ‹€ β€” μ»€λ°‹λœ λ²ˆλ“€(manifest 있음)은 κ±΄λ“œλ¦¬μ§€ μ•ŠλŠ”λ‹€."""
    if staging.is_dir() and not (staging / "manifest.json").exists():
        shutil.rmtree(staging)