File size: 16,163 Bytes
3dfc93b
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
"""๋ฒˆ๋“ค ๋‹ค๋ฆฌ (๊ณ„ํš 2026-09-27 L6 ยง2.5 ยท D-L6-2 ยท D-L6-8 ยท D-L6-9).

case ์ž ๊ธˆ **์•ˆ**์—์„œ hook ์ด ๋ถ€๋ฅธ๋‹ค. ์ˆœ์„œ:

1. **๋ฐฑํ•„** โ€” `BackfillOptions(case=a/c, root=target.projects_dir, mirror_root=blob_mirror_root(), index_root=hub_index_root(),
   repo_root=target.repo_root, run_ids=frozenset({run_id}))` โ†’ `backfill_case(opts, target.context)`(wrap โ†’ ๋ฏธ๋Ÿฌ โ†’ ๋ฒˆ๋“ค ์ƒ‰์ธ โ†’
   eval-wrap). `CaseReport.runs` ๋Š” ์ •ํ™•ํžˆ ํ•˜๋‚˜์—ฌ์•ผ ํ•œ๋‹ค(0 ์ด๋ฉด `run_not_in_archive`). ๋ถ€๋ฅ˜ โ†’ ์ฝ”๋“œ:
   wrap `writtenยทreusedยทrewrapped_input_fields` โ†’ ๋ฒˆ๋“ค ์žˆ์Œ ยท `frozen:<c>` โ†’ `bundle_frozen:<c>`(hub ๋Š” `no_bundle`) ยท
   `error:<c>` โ†’ `bundle_error:<c>` ยท ๋ฒˆ๋“ค ์ƒ‰์ธ `error:<c>` โ†’ `bundle_index:<c>`(๊ณ„์†) ยท eval-wrap `error:*` โ†’
   `eval_wrap:<error_code>`(๊ณ„์† โ€” ๋ฒˆ๋“ค์€ ์žˆ๋‹ค) ยท eval-wrap `skip:*`ยท`frozen:*` ๋Š” ์ •๋ณด ยท `BackfillError` โ†’ `mirror_unadoptable`.
2. **์žฌ์ฑ„์ **(๋ฒˆ๋“ค์ด ์žˆ์„ ๋•Œ๋งŒ) โ€” lcs_diff ๊ณ„์—ด(D-L6-8: run.json ์Šค๋ƒ…์ƒท `config.evaluator_config.config.type`, ์Šค๋ƒ…์ƒท์ด ์—†์œผ๋ฉด run
   ๋””๋ ‰ํ† ๋ฆฌ ํ‰๊ฐ€ ํŒŒ์ผ์˜ `poc_bundle.legacy_run.evaluator_family`)์ด๋ฉด `rescore_case(a, c, run_ids={run_id})`(case ๋ชจ๋“œ ์ฑ„ํƒ ๊ทœ์น™,
   D-L6-9) โ†’ `written|exists` ๊ฐ€ ok ยท `no_results` skip โ†’ `rescore_empty` ยท ๊ทธ ๋ฐ– skip โ†’ `rescore_skip:<code>` ยท ์˜ค๋ฅ˜ โ†’
   `rescore_error:<code>`. ๊ณ„์—ด์ด ์•„๋‹ˆ๋ฉด `not_proofread_family`(์ •๋ณด โ€” "์ด ํ‰๊ฐ€๊ธฐ๋Š” M2 ์ฑ„์  ๋ฒˆ๋“ค ์ƒ์‚ฐ์ž๊ฐ€ ์—†๋‹ค", Q-M2-1).
3. **์ƒ‰์ธ** โ€” `indexes.refresh`(lockยทํ…œํ”Œ๋ฆฟยทํ…์ŠคํŠธ ๋ชจ๋“œยท์ถ”์ถœ๊ธฐยท์ƒ‰์ธ ์ฝ”๋“œ ํ‚ค๊ฐ€ ๋ฐ”๋€Œ์—ˆ์„ ๋•Œ๋งŒ, D-L6-7).

๋‹ค๋ฆฌ ๊ฒฐ๊ณผ๋Š” ํ•˜๋‚˜์˜ `LegResult` ๋‹ค. ์‹คํŒจ๊ฐ€ ์—ฌ๋Ÿฟ์ด๋ฉด **์ˆœ์„œ์ƒ ์ฒซ ์‹คํŒจ**๊ฐ€ `code` ์ด๊ณ , ๋‹จ๊ณ„๋งˆ๋‹ค์˜ ๊ฒฐ๊ณผ๋Š” `detail` ์— ๋ชจ๋‘ ๋‚จ๋Š”๋‹ค
(์ƒํƒœ ํŒŒ์ผ ยง2.7): `{"backfill": run_entry, "rescore": {"status", "code"?, "bundle_id"?}, "indexes": <์ฝ”๋“œ>}`. ๋ฒˆ๋“ค์ด ์žˆ์œผ๋ฉด ์‹คํŒจ์—ฌ๋„
`detail.backfill.wrap` ์ด ๋‚จ์•„ hook ์ด `types.bundle_ready` ๋กœ hub ๋‹ค๋ฆฌ๋ฅผ ๋ˆ๋‹ค(D-L6-2). ์ƒ‰์ธ CLI ์˜ ์ •ํ•ฉ์„ฑ ๊ฒฝ๊ณ ๊ฐ€ ์žˆ์œผ๋ฉด
`index_warnings: <๊ฑด์ˆ˜>`, ์น˜์šด ์˜› ์ƒ‰์ธ ๋””๋ ‰ํ† ๋ฆฌ๊ฐ€ ์žˆ์œผ๋ฉด `index_removed: [<์ด๋ฆ„>โ€ฆ]` ๋ฅผ ๋”ํ•œ๋‹ค(๋ฆฌ๋ทฐ ์ง€์  2ยท4 โ€” ์ค„ ์ž์ฒด๋Š”
`indexes` ๊ฐ€ stderr ๋กœ ๋‚ธ๋‹ค).

**๋‹จ๊ณ„ ์˜ˆ์™ธ ๋ด‰์‡„.** ์žฌ์ฑ„์ ยท์ƒ‰์ธ ๋‹จ๊ณ„์˜ ์˜ˆ๊ธฐ์น˜ ์•Š์€ ์˜ˆ์™ธ๋Š” ๊ทธ ๋‹จ๊ณ„๋งŒ `rescore_exception:<Type>`ยท`indexes_exception:<Type>` ์‹คํŒจ๋กœ
๊ฐ•๋“ฑํ•œ๋‹ค(ํƒ€์ž… ์ด๋ฆ„๋งŒ) โ€” ๋ฐฑํ•„์ด ๋งŒ๋“  ๋ฒˆ๋“ค์„ ๋ฒ„๋ฆฌ์ง€ ์•Š๊ณ  hub ๋‹ค๋ฆฌ๊นŒ์ง€ ๊ฐ€๊ฒŒ. ๋ฐฑํ•„ ์ž์ฒด์˜ ์˜ˆ์™ธ๋Š” hook ์ด `bundle_exception:<Type>`
์œผ๋กœ ๊ฐ•๋“ฑํ•œ๋‹ค(๋ฒˆ๋“ค ์œ ๋ฌด๋ฅผ ๋ชจ๋ฅธ๋‹ค). ์›๋ฌธยทlocals ๋Š” ์‹ฃ์ง€ ์•Š์ง€๋งŒ **๋ฐœ์ƒ ์œ„์น˜**๋Š” ๋‚จ๊ธด๋‹ค(๋ฆฌ๋ทฐ ์ง€์  5): ๊ฐ€์žฅ ์•ˆ์ชฝ ํ”„๋ ˆ์ž„์˜
`<ํŒŒ์ผ basename>:<์ค„>:<ํ•จ์ˆ˜>` ๋ฅผ `detail.exception_at.<๋‹จ๊ณ„>` ์—(๊ณ ๊ฐ ๋ฐ์ดํ„ฐ๊ฐ€ ์—†๋Š” ์ฝ”๋“œ ์ขŒํ‘œ), ํŠธ๋ ˆ์ด์Šค๋ฐฑ ์ „์ฒด๋Š” ์ด ๋ชจ๋“ˆ ๋กœ๊ฑฐ์—
(`hook` ๊ณผ ๊ฐ™์€ ๊ทœ์น™ โ€” ๊ธฐ๋ณธ DEBUG, `SOLAR_EVAL_HOOK_DEBUG=1` ์ด๋ฉด WARNING).

`poc_bundle`ยท`solar_eval.rescore` ๋Š” ํ•จ์ˆ˜ ์•ˆ์—์„œ๋งŒ import ํ•œ๋‹ค(D-L6-14). ์ถœ๋ ฅยทdetail ์—๋Š” idยท๋ถ€๋ฅ˜ยท์ฝ”๋“œ๋งŒ(๊ณ ๊ฐ ํ…์ŠคํŠธ ์—†์Œ).
"""

from __future__ import annotations

import logging
import traceback
from collections.abc import Mapping
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any

from solar_eval.hub_hook import indexes, optin, target
from solar_eval.hub_hook.types import (
    BUNDLE_ERROR_PREFIX,
    BUNDLE_FROZEN_PREFIX,
    BUNDLE_INDEX_FAILED_PREFIX,
    EVAL_WRAP_FAILED_PREFIX,
    MIRROR_UNADOPTABLE,
    NO_BUNDLE,
    NOT_PROOFREAD_FAMILY,
    PROOFREAD_EVALUATOR_TYPE,
    RESCORE_EMPTY,
    RESCORE_ERROR_PREFIX,
    RESCORE_NO_RESULTS,
    RESCORE_OK,
    RESCORE_SKIP_PREFIX,
    RUN_NOT_IN_ARCHIVE,
    STATUS_FAILED,
    STATUS_NOT_RUN,
    STATUS_SKIPPED,
    STEP_BACKFILL,
    STEP_INDEXES,
    STEP_RESCORE,
    WRAP_BUNDLE_READY,
    WRAP_ERROR_PREFIX,
    WRAP_FROZEN_PREFIX,
    LegResult,
    Target,
    exception_code,
)

_RUN_JSON = "run.json"
_LOG = logging.getLogger(__name__)
# detail ํ‚ค `index_warnings`ยท`index_removed`ยท`exception_at`(๋ฆฌ๋ทฐ ์ง€์  2ยท4ยท5)๋Š” ์ฝ”๋“œ ๋ชจ์–‘ ๋ฌธ์ž์—ด์ด๋ผ ์ด ๋ชจ๋“ˆ ์ตœ์ƒ์œ„ ์ƒ์ˆ˜๋กœ ๋‘์ง€
# ์•Š๋Š”๋‹ค(`test_types_home` โ€” ์ฝ”๋“œ ๋ฌธ์ž์—ด์€ `types.py` ํ•œ ๊ณณ). ์“ฐ๋Š” ์ž๋ฆฌ์— ํ•œ ๋ฒˆ์”ฉ๋งŒ ์ ์—ˆ๋‹ค โ€” l6a-verify ๊ฐ€ `types.py` ๋กœ ์˜ฎ๊ธด๋‹ค
# (๊ณ„ํš ยง2.5 ๊ตฌํ˜„ ์‹œ ๋ณ€๊ฒฝ ๋ฉ”๋ชจ).


@dataclass(frozen=True)
class _Backfilled:
    """๋ฐฑํ•„ ๊ฒฐ๊ณผ โ€” `entry` ๋Š” ๋ณด๊ณ ์„œ `run_entry`(์—†์œผ๋ฉด run ์ด ์—†๊ฑฐ๋‚˜ case ์˜ค๋ฅ˜), `failures` ๋Š” ์ˆœ์„œ๋Œ€๋กœ์˜ ์‹คํŒจ ์ฝ”๋“œ."""

    entry: Mapping[str, Any] | None
    run_id: str | None = None
    bundle_id: str | None = None
    failures: tuple[str, ...] = ()

    @property
    def ready(self) -> bool:
        return self.entry is not None and self.entry.get("wrap") in WRAP_BUNDLE_READY


@dataclass
class _Steps:
    """๋‹ค๋ฆฌ ์•ˆ ๋‹จ๊ณ„๋“ค์˜ ๊ฒฐ๊ณผ โ€” ์ฒซ ์‹คํŒจ์™€ ๊ทธ hint ๋ฅผ ๋ชจ์€๋‹ค."""

    detail: dict[str, Any] = field(default_factory=dict)
    failures: list[str] = field(default_factory=list)
    hint: str | None = None

    def fail(self, code: str, hint: str | None = None) -> None:
        if not self.failures:
            self.hint = hint
        self.failures.append(code)

    def result(self) -> LegResult:
        if self.failures:
            return LegResult.failed(self.failures[0], self.hint, self.detail)
        return LegResult.ok(self.detail)

    def contain(self, step: str, exc: Exception) -> str:
        """๋‹จ๊ณ„ ์˜ˆ์™ธ โ†’ ์ฝ”๋“œ(ํƒ€์ž… ์ด๋ฆ„๋งŒ). ์œ„์น˜ ์ขŒํ‘œ๋Š” detail ์—, ํŠธ๋ ˆ์ด์Šค๋ฐฑ์€ ๋กœ๊ฑฐ๋กœ(๋ชจ๋“ˆ ์„ค๋ช… ยท ๋ฆฌ๋ทฐ ์ง€์  5)."""
        code = exception_code(step, exc)
        where = exception_location(exc)
        if where is not None:
            self.detail.setdefault("exception_at", {})[step] = where
        level = logging.WARNING if optin.hook_debug_requested() else logging.DEBUG
        _LOG.log(level, "Experiment Hub ํ›… ๋ฒˆ๋“ค ๋‹ค๋ฆฌ ๋‹จ๊ณ„ ์˜ˆ์™ธ ๋ด‰์‡„ โ€” %s", code, exc_info=exc)
        return code


def exception_location(exc: BaseException) -> str | None:
    """๊ฐ€์žฅ ์•ˆ์ชฝ ํ”„๋ ˆ์ž„์˜ `<ํŒŒ์ผ basename>:<์ค„>:<ํ•จ์ˆ˜>` โ€” ๋ฉ”์‹œ์ง€ยทlocals ๋Š” ์‹ฃ์ง€ ์•Š๋Š”๋‹ค(๊ณ ๊ฐ ๋ฐ์ดํ„ฐ๊ฐ€ ์—†๋Š” ์ฝ”๋“œ ์ขŒํ‘œ๋งŒ)."""
    frames = traceback.extract_tb(exc.__traceback__)
    if not frames:
        return None
    last = frames[-1]
    return f"{Path(last.filename).name}:{last.lineno}:{last.name}"


def run_bundle_leg(target: Target, run_dir: Path, run_id: str) -> LegResult:
    """run ํ•˜๋‚˜์˜ ๋ฐฑํ•„ โ†’ ์žฌ์ฑ„์  โ†’ ์ƒ‰์ธ(ยง2.5). ๋ฐฑํ•„ ์˜ˆ์™ธ๋Š” hook ์ด `bundle_exception:<Type>` ์œผ๋กœ ๊ฐ•๋“ฑํ•œ๋‹ค."""
    backfilled = _backfill(target, run_id)
    if backfilled.entry is None:
        return LegResult.failed(backfilled.failures[0])
    steps = _Steps({STEP_BACKFILL: dict(backfilled.entry)})
    for code in backfilled.failures:
        steps.fail(code)
    if backfilled.ready:
        _rescore_step(target, Path(run_dir), backfilled, steps)
    else:
        steps.detail[STEP_RESCORE] = {"status": STATUS_NOT_RUN, "code": NO_BUNDLE}
    _indexes_step(target, steps)
    return steps.result()


def ensure_run_bundle(target: Target, run_id: str) -> tuple[LegResult, Path | None]:
    """๋ฐฑํ•„ ๋ถ€๋ถ„๋งŒ(L6b `runs eval` ์ƒˆ ๊ฒฝ๋กœ โ‘ ) โ€” ์ฑ„ํƒ๋œ ๋ฏธ๋Ÿฌ run ๋ฒˆ๋“ค ๊ฒฝ๋กœ๋ฅผ ํ•จ๊ป˜ ๋Œ๋ ค์ค€๋‹ค. ๋ฒˆ๋“ค์ด ์—†์œผ๋ฉด None.

    **์ „์ œ: ํ˜ธ์ถœ์ž๊ฐ€ `lock.case_lock(target.account, target.case)` ๋ฅผ ์ฅ” ์ฑ„ ๋ถ€๋ฅธ๋‹ค**(๊ณ„ํš ยง2.12 B4 "new_path (case ์ž ๊ธˆ ์•ˆ)" ยท
    ๋ฆฌ๋ทฐ ์ง€์  6) โ€” ๋ฐฑํ•„์€ ๋ฏธ๋Ÿฌยท๋ฒˆ๋“ค ์ƒ‰์ธ ์‚ฌ์ด๋“œ์นดยทeval-wrap ์„ ์“ฐ๋ฏ€๋กœ ๊ฐ™์€ case ์˜ ํ›…๊ณผ ๊ฒน์น˜๋ฉด ์•ˆ ๋œ๋‹ค. ์—ฌ๊ธฐ์„œ ์ž ๊ทธ์ง€ ์•Š๋Š”
    ๊นŒ๋‹ญ: L6b ๋Š” โ‘ ~โ‘ค(๋ฒˆ๋“ค ํ™•์ธยท์ฑ„์ ยท๋ฐœํ–‰ยทhubยท์ƒํƒœ ํŒŒ์ผ)๋ฅผ ํ•œ ์ž ๊ธˆ ์•ˆ์—์„œ ๋ˆ๋‹ค โ€” `flock` ์€ ๊ฐ™์€ ํ”„๋กœ์„ธ์Šค๋ผ๋„ ๋”ฐ๋กœ ์—ฐ ๊ธฐ์ˆ ์ž๋ผ๋ฆฌ
    ๋ง‰์œผ๋ฏ€๋กœ ์•ˆ์—์„œ ๋‹ค์‹œ ์žก์œผ๋ฉด ์ž๊ธฐ ์ž์‹ ์„ ๊ธฐ๋‹ค๋ฆฌ๋‹ค ํ•œ๋„์— ๊ฑธ๋ฆฐ๋‹ค.

    ํŒ์ • ์ง‘ํ•ฉ์€ `run_bundle_leg` ์™€ ๊ฐ™๋‹ค(`WRAP_BUNDLE_READY`) โ€” ํ›…์—์„œ ํ†ต๊ณผํ•œ run ์ด ์—ฌ๊ธฐ์„œ ์‹คํŒจํ•˜์ง€ ์•Š๊ฒŒ(๊ฒ€ํ†  F12).
    eval-wrapยท๋ฒˆ๋“ค ์ƒ‰์ธ ์‹คํŒจ๋Š” `run_bundle_leg` ์ฒ˜๋Ÿผ ์‹คํŒจ๋กœ ์‹ฃ๋˜ ๊ฒฝ๋กœ๋Š” ๋Œ๋ ค์ค€๋‹ค(๋ฒˆ๋“ค์€ ์žˆ๋‹ค).
    """
    backfilled = _backfill(target, run_id)
    if backfilled.entry is None:
        return LegResult.failed(backfilled.failures[0]), None
    detail = {STEP_BACKFILL: dict(backfilled.entry)}
    path = _mirror_bundle_path(target, backfilled.bundle_id) if backfilled.ready else None
    if backfilled.failures:
        return LegResult.failed(backfilled.failures[0], detail=detail), path
    return LegResult.ok(detail), path


# โ”€โ”€ ๋ฐฑํ•„ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€


def _backfill(target: Target, run_id: str) -> _Backfilled:
    """`backfill_case` ๋ฅผ run ํ•˜๋‚˜๋กœ. ์ถฉ๋Œ ๊ฒ€์‚ฌ๋Š” PB ๊ฐ€ case ์˜ run ๋””๋ ‰ํ† ๋ฆฌ ์ „๋ถ€๋กœ ํ•œ๋‹ค."""
    from poc_bundle.backfill.report import run_entry
    from poc_bundle.backfill.runner import BackfillError, BackfillOptions, backfill_case
    from poc_bundle.home import blob_mirror_root, hub_index_root

    opts = BackfillOptions(
        case=target.case_ref,
        root=target.projects_dir,
        mirror_root=blob_mirror_root(),
        index_root=hub_index_root(),
        repo_root=target.repo_root,
        run_ids=frozenset({run_id}),
    )
    try:
        report = backfill_case(opts, target.context)
    except BackfillError:
        return _Backfilled(None, failures=(MIRROR_UNADOPTABLE,))
    if not report.runs:
        return _Backfilled(None, failures=(RUN_NOT_IN_ARCHIVE,))
    if len(report.runs) != 1:
        raise RuntimeError(f"backfill returned {len(report.runs)} runs for one run directory")
    run = report.runs[0]
    return _Backfilled(run_entry(run), run.run_id, run.bundle_id, _backfill_failures(run))


def _backfill_failures(run: Any) -> tuple[str, ...]:
    """`RunReport` ๋ถ€๋ฅ˜ โ†’ ๋ฒˆ๋“ค ๋‹ค๋ฆฌ ์‹คํŒจ ์ฝ”๋“œ(์ˆœ์„œ: ์–ด๋Œ‘ํ„ฐยทwrap โ†’ ๋ฒˆ๋“ค ์ƒ‰์ธ โ†’ eval-wrap)."""
    failures: list[str] = []
    wrap_failure = _wrap_failure(run.adapter, run.wrap)
    if wrap_failure is not None:
        failures.append(wrap_failure)
    if isinstance(run.index, str) and run.index.startswith(WRAP_ERROR_PREFIX):
        failures.append(f"{BUNDLE_INDEX_FAILED_PREFIX}{run.index.removeprefix(WRAP_ERROR_PREFIX)}")
    if isinstance(run.eval_wrap, str) and run.eval_wrap.startswith(WRAP_ERROR_PREFIX):
        detail = run.error_code or run.eval_wrap.removeprefix(WRAP_ERROR_PREFIX)
        failures.append(f"{EVAL_WRAP_FAILED_PREFIX}{detail}")
    return tuple(failures)


def _wrap_failure(adapter: str | None, wrap: str | None) -> str | None:
    """wrap ์ด ๋ฒˆ๋“ค์„ ๋‚จ๊ธฐ์ง€ ๋ชปํ–ˆ์œผ๋ฉด ๊ทธ ์ฝ”๋“œ. ์–ด๋Œ‘ํ„ฐ๊ฐ€ ๋ง‰์•„ wrap ์— ๋‹ฟ์ง€ ์•Š์•˜์œผ๋ฉด ์–ด๋Œ‘ํ„ฐ ๋ถ€๋ฅ˜๋กœ."""
    if wrap in WRAP_BUNDLE_READY:
        return None
    if isinstance(wrap, str) and wrap.startswith(WRAP_FROZEN_PREFIX):
        return f"{BUNDLE_FROZEN_PREFIX}{wrap.removeprefix(WRAP_FROZEN_PREFIX)}"
    stage = wrap if isinstance(wrap, str) else adapter
    if isinstance(stage, str) and stage.startswith(WRAP_ERROR_PREFIX):
        return f"{BUNDLE_ERROR_PREFIX}{stage.removeprefix(WRAP_ERROR_PREFIX)}"
    return f"{BUNDLE_ERROR_PREFIX}{stage or 'none'}"


def _mirror_bundle_path(t: Target, bundle_id: str | None) -> Path | None:
    """๋ฏธ๋Ÿฌ์˜ `accounts/<a>/<c>/eval/bundles/<bundle_id>`(์žˆ์„ ๋•Œ๋งŒ)."""
    from solar_eval.rescore.paths import BUNDLES_AREA, mirror_area_dir

    if not bundle_id:
        return None
    path = mirror_area_dir(None, t.account, t.case, BUNDLES_AREA) / bundle_id
    return path if path.is_dir() else None


# โ”€โ”€ ์žฌ์ฑ„์  โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€


def _rescore_step(t: Target, run_dir: Path, backfilled: _Backfilled, steps: _Steps) -> None:
    try:
        if not is_proofread_family(run_dir):
            steps.detail[STEP_RESCORE] = {"status": STATUS_SKIPPED, "code": NOT_PROOFREAD_FAMILY}
            return
        status, code, bundle_id = _rescore(t, backfilled.run_id or run_dir.name)
    except Exception as exc:  # ๋‹จ๊ณ„ ๊ฒฝ๊ณ„ โ€” ๋ฐฑํ•„์ด ๋งŒ๋“  ๋ฒˆ๋“ค๋กœ hub ๋‹ค๋ฆฌ๋Š” ๊ณ„์† ๊ฐ„๋‹ค(๋ชจ๋“ˆ ์„ค๋ช…)
        status, code, bundle_id = STATUS_FAILED, steps.contain(STEP_RESCORE, exc), None
    entry: dict[str, Any] = {"status": status}
    if code is not None:
        entry["code"] = code
    if bundle_id is not None:
        entry["bundle_id"] = bundle_id
    steps.detail[STEP_RESCORE] = entry
    if status == STATUS_FAILED and code is not None:
        steps.fail(code)


def is_proofread_family(run_dir: Path) -> bool:
    """์žฌ์ฑ„์  ๋Œ€์ƒ์ธ๊ฐ€(D-L6-8). ์Šค๋ƒ…์ƒท ํ‰๊ฐ€๊ธฐ type ์ด ์žˆ์œผ๋ฉด ๊ทธ๊ฒƒ, ์—†์œผ๋ฉด run ๋””๋ ‰ํ† ๋ฆฌ ํ‰๊ฐ€ ํŒŒ์ผ์˜ ๊ณ„์—ด(๊ฒ€ํ†  F18).

    run.jsonยทํ‰๊ฐ€ ํŒŒ์ผ์„ ๋ชป ์ฝ์œผ๋ฉด ๊ณ„์—ด์„ ๋ชจ๋ฅด๋ฏ€๋กœ ๋Œ€์ƒ์ด ์•„๋‹ˆ๋‹ค(`not_proofread_family` โ€” wrap ์ด ๊ทธ run ์„ ์ด๋ฏธ ๊ฐ€๋ฅธ๋‹ค).
    """
    from poc_bundle.canonical import load_json_strict
    from poc_bundle.legacy_run import (
        FAMILY_LCS,
        LegacyRunError,
        evaluator_family,
        read_eval_files,
        split_rounds,
    )

    try:
        run_json = load_json_strict((Path(run_dir) / _RUN_JSON).read_bytes(), safe_integers=False)
    except (OSError, ValueError, RecursionError):
        return False
    if not isinstance(run_json, dict):
        return False
    kind = target.snapshot_evaluator_type(run_json)
    if kind is not None:
        return kind == PROOFREAD_EVALUATOR_TYPE
    try:
        evaluation, lines = read_eval_files(Path(run_dir))
    except LegacyRunError:
        return False
    return evaluator_family(run_json, evaluation, split_rounds(evaluation, lines)) == FAMILY_LCS


def _rescore(t: Target, run_id: str) -> tuple[str, str | None, str | None]:
    """`rescore_case(run_ids={run_id})` โ†’ (์ƒํƒœ, ์ฝ”๋“œ, ์ฑ„์  ๋ฒˆ๋“ค id). ์ƒํƒœ๋Š” `written|exists`(ok) ๋˜๋Š” `failed`."""
    from poc_bundle.home import blob_mirror_root

    from solar_eval.rescore.case import rescore_case

    summary = rescore_case(
        t.account,
        t.case,
        mirror_root=blob_mirror_root(),
        repo_root=t.repo_root,
        run_ids=frozenset({run_id}),
    )
    if summary.failures:
        return STATUS_FAILED, f"{RESCORE_ERROR_PREFIX}{summary.failures[0].code}", None
    if len(summary.outcomes) != 1:
        return STATUS_FAILED, RESCORE_EMPTY, None
    outcome = summary.outcomes[0]
    if outcome.status in RESCORE_OK:
        bundle_id = None if outcome.path is None else Path(outcome.path).name
        return outcome.status, None, bundle_id
    if outcome.code == RESCORE_NO_RESULTS:
        return STATUS_FAILED, RESCORE_EMPTY, None
    return STATUS_FAILED, f"{RESCORE_SKIP_PREFIX}{outcome.code}", None


# โ”€โ”€ ์ƒ‰์ธ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€


def _indexes_step(t: Target, steps: _Steps) -> None:
    try:
        outcome = indexes.refresh(t)
    except Exception as exc:  # ๋‹จ๊ณ„ ๊ฒฝ๊ณ„ โ€” ์ƒ‰์ธ์€ case ํŒŒ์ƒ๋ฌผ์ด๋ผ run ๋ฒˆ๋“คยทhub ๋‹ค๋ฆฌ๋ฅผ ๋ง‰์ง€ ์•Š๋Š”๋‹ค
        code = steps.contain(STEP_INDEXES, exc)
        steps.detail[STEP_INDEXES] = f"{STATUS_FAILED}:{code}"
        steps.fail(code)
        return
    steps.detail[STEP_INDEXES] = outcome.code
    if outcome.warnings:
        steps.detail["index_warnings"] = len(outcome.warnings)
    if outcome.removed:
        steps.detail["index_removed"] = list(outcome.removed)
    if outcome.failed:
        steps.fail(outcome.code.removeprefix(f"{STATUS_FAILED}:"), outcome.reason)


__all__ = ["ensure_run_bundle", "is_proofread_family", "run_bundle_leg"]