Spaces:
Running
feat: add v24 pipeline (FT + self-consistency × 2 + tool-calling judge)
Browse filesNew tab "v24 (new · 260429)" alongside existing v16 legacy tab.
Pipeline architecture (5 LLM calls/doc):
vocab → FT → basic_correction × 2 + intersection → tool-calling judge
(round 1 lookups + round 2 decisions) → apply_judge_decisions →
vocab_reapply → paragraph_dedupe → post_process
paragraph F1 47.04 (vs solar-pro2 prod_251231 mean 44.86, +2.18).
Key changes:
- pipelines_v24.py: new module with self-consistency wrapper, 2-round
tool-calling judge emulation, apply_judge_decisions revert logic, and
PMI lookup loader (in-memory dict from gzipped TSV).
- pipelines.py: register 260429_v24, add _process_single_bulk_with_raw
helper (returns post-text + raw JSON for self-consistency intersection),
dispatch new step types (llm_self_consistency, tool_calling_judge,
apply_judge_decisions rule), reset v24 state per request.
- app.py: gr.Tabs(v16 legacy, v24 new). Keyboard shortcut JS targets the
visible tab via offsetParent.
- blindtest/ui.py: build_feedback_tab(elem_id_prefix=...) parameter to
avoid duplicate elem_id collisions across tabs.
- prompts/dev_260429_v24/: copies of prompt_dev_v23 (step1.txt, judge.txt).
- data/pmi_compounds_top.tsv.gz (LFS): top-K PMI subset (53K bigrams +
134K unigrams, 1.5MB compressed) extracted with pmi≥5 AND count≥30.
Lets the demo's tool-calling judge run without requiring the full
500MB+ chosun corpus.
Chunking strategy (article-level):
- 1 paragraph = 1 bulk (existing splitter, no cross-paragraph packing).
- Per-bulk self-consistency: each paragraph runs basic_correction × 2
independently, intersects within its own scope.
- Per-bulk judge: each paragraph gets its own 2-round judge call. State
keyed by (bulk_idx, step_name) so paragraphs don't cross-contaminate.
- Per-bulk apply: revert decisions applied within each paragraph, then
'\n'.join to reconstruct the article.
Verified end-to-end against the live Upstage API + Supabase:
- Single paragraph: 8.3s, 4 corrections (사이시옷, 의존명사, 단위명사, 오탈자).
- Multi-paragraph (3 paragraphs): 6.2s, parallel chunking works,
step_errors empty, post_process pronoun replacement applied.
- Judge revert: 사법리스크/헌재소장 preserved by PMI lookup.
- Supabase schema unchanged: pipeline_key='260429_v24' inserts cleanly,
fetch_rating_counts shows correct delta on insert/cleanup cycle.
- app.py +43 -9
- blindtest/ui.py +17 -12
- data/pmi_compounds_top.tsv.gz +3 -0
- pipelines.py +300 -2
- pipelines_v24.py +737 -0
- prompts/dev_260429_v24/judge.txt +95 -0
- prompts/dev_260429_v24/step1.txt +92 -0
|
@@ -1,6 +1,9 @@
|
|
| 1 |
"""Chosun proofreading demo — Gradio UI.
|
| 2 |
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
| 4 |
"""
|
| 5 |
|
| 6 |
import os
|
|
@@ -33,16 +36,38 @@ def _default_prompt(preferred_prefix: str, fallback: str) -> str:
|
|
| 33 |
|
| 34 |
|
| 35 |
_v16_prompt = _default_prompt("dev_260408_v16", prompt_choices[-1])
|
|
|
|
| 36 |
|
| 37 |
with gr.Blocks(title="Chosun 교정교열 데모") as demo:
|
| 38 |
gr.Markdown("# Chosun 교정교열 데모")
|
| 39 |
gr.Markdown("solar-pro3 기반 교정 결과를 확인하고 피드백을 남겨주세요.")
|
| 40 |
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
|
| 47 |
_SHORTCUT_JS = """
|
| 48 |
() => {
|
|
@@ -51,9 +76,18 @@ with gr.Blocks(title="Chosun 교정교열 데모") as demo:
|
|
| 51 |
document.addEventListener('keydown', (e) => {
|
| 52 |
if ((e.metaKey || e.ctrlKey) && e.key === 'Enter') {
|
| 53 |
e.preventDefault();
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
}
|
| 58 |
});
|
| 59 |
}
|
|
|
|
| 1 |
"""Chosun proofreading demo — Gradio UI.
|
| 2 |
|
| 3 |
+
Two tabs:
|
| 4 |
+
- v16 (legacy): solar-pro3 with the 260408_v16 single-pass pipeline.
|
| 5 |
+
- v24 (new): solar-pro3 with FT + self-consistency × 2 + tool-calling
|
| 6 |
+
judge. paragraph F1 47.04 (vs solar-pro2 prod_251231 mean 44.86).
|
| 7 |
"""
|
| 8 |
|
| 9 |
import os
|
|
|
|
| 36 |
|
| 37 |
|
| 38 |
_v16_prompt = _default_prompt("dev_260408_v16", prompt_choices[-1])
|
| 39 |
+
_v24_prompt = _default_prompt("dev_260429_v24", "dev_260429_v24")
|
| 40 |
|
| 41 |
with gr.Blocks(title="Chosun 교정교열 데모") as demo:
|
| 42 |
gr.Markdown("# Chosun 교정교열 데모")
|
| 43 |
gr.Markdown("solar-pro3 기반 교정 결과를 확인하고 피드백을 남겨주세요.")
|
| 44 |
|
| 45 |
+
with gr.Tabs():
|
| 46 |
+
with gr.Tab("v16 (legacy · 260408)"):
|
| 47 |
+
gr.Markdown(
|
| 48 |
+
"**solar-pro3 + step1 single-pass.** 빠르고 가벼움. "
|
| 49 |
+
"paragraph F1 ~45.6 / ci-v1 ~19.3."
|
| 50 |
+
)
|
| 51 |
+
build_feedback_tab(
|
| 52 |
+
client=client,
|
| 53 |
+
vocabulary=vocabulary,
|
| 54 |
+
pipeline_config=("260408_v16", "solar-pro3", _v16_prompt),
|
| 55 |
+
elem_id_prefix="v16",
|
| 56 |
+
)
|
| 57 |
+
|
| 58 |
+
with gr.Tab("v24 (new · 260429)"):
|
| 59 |
+
gr.Markdown(
|
| 60 |
+
"**solar-pro3 + FT + self-consistency × 2 + tool-calling judge.** "
|
| 61 |
+
"5 LLM calls/doc — 응답 다소 느림 (~5-10s/문단). "
|
| 62 |
+
"paragraph F1 47.04 (solar-pro2 대비 +2.18). "
|
| 63 |
+
"긴 기사는 자동으로 문단 단위 청킹 → 병렬 처리됨."
|
| 64 |
+
)
|
| 65 |
+
build_feedback_tab(
|
| 66 |
+
client=client,
|
| 67 |
+
vocabulary=vocabulary,
|
| 68 |
+
pipeline_config=("260429_v24", "solar-pro3", _v24_prompt),
|
| 69 |
+
elem_id_prefix="v24",
|
| 70 |
+
)
|
| 71 |
|
| 72 |
_SHORTCUT_JS = """
|
| 73 |
() => {
|
|
|
|
| 76 |
document.addEventListener('keydown', (e) => {
|
| 77 |
if ((e.metaKey || e.ctrlKey) && e.key === 'Enter') {
|
| 78 |
e.preventDefault();
|
| 79 |
+
// Find the run button inside the currently visible tab. Tab
|
| 80 |
+
// panels gradio doesn't render get hidden via CSS, so checking
|
| 81 |
+
// offsetParent / getBoundingClientRect filters them out.
|
| 82 |
+
const candidates = document.querySelectorAll(
|
| 83 |
+
'#v16-run-btn, #v24-run-btn, #v16-run-btn button, #v24-run-btn button'
|
| 84 |
+
);
|
| 85 |
+
for (const el of candidates) {
|
| 86 |
+
if (el.offsetParent !== null) {
|
| 87 |
+
el.click();
|
| 88 |
+
return;
|
| 89 |
+
}
|
| 90 |
+
}
|
| 91 |
}
|
| 92 |
});
|
| 93 |
}
|
|
@@ -30,20 +30,22 @@ def _format_summary(counts: dict[str, int]) -> str:
|
|
| 30 |
g = counts.get("good", 0)
|
| 31 |
n = counts.get("not_bad", 0)
|
| 32 |
c = counts.get("critical", 0)
|
| 33 |
-
return
|
| 34 |
-
f"**총 피드백**: {total}\n\n"
|
| 35 |
-
f"- Good: **{g}**\n"
|
| 36 |
-
f"- Not Bad: **{n}**\n"
|
| 37 |
-
f"- Critical: **{c}**"
|
| 38 |
-
)
|
| 39 |
|
| 40 |
|
| 41 |
def build_feedback_tab(
|
| 42 |
client: Any,
|
| 43 |
vocabulary: list[dict],
|
| 44 |
pipeline_config: PipelineConfig,
|
|
|
|
| 45 |
) -> None:
|
| 46 |
-
"""Build the single-pipeline feedback UI. Call inside a gr.Blocks/Tab.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
|
| 48 |
pipeline_key, model, prompt_key = pipeline_config
|
| 49 |
|
|
@@ -58,7 +60,7 @@ def build_feedback_tab(
|
|
| 58 |
run_btn = gr.Button(
|
| 59 |
"교정 실행 (⌘+Enter / Ctrl+Enter)",
|
| 60 |
variant="primary",
|
| 61 |
-
elem_id="
|
| 62 |
)
|
| 63 |
|
| 64 |
status = gr.Markdown("")
|
|
@@ -96,9 +98,7 @@ def build_feedback_tab(
|
|
| 96 |
|
| 97 |
start = time.time()
|
| 98 |
try:
|
| 99 |
-
result = run_pipeline(
|
| 100 |
-
text, pipeline_key, model, prompt_key, client, vocabulary
|
| 101 |
-
)
|
| 102 |
except Exception as exc:
|
| 103 |
return (
|
| 104 |
gr.update(value=f"에러: {exc}"),
|
|
@@ -138,9 +138,14 @@ def build_feedback_tab(
|
|
| 138 |
def _make_rating_handler(rating: str):
|
| 139 |
def handler(run_id, comment_text):
|
| 140 |
saved = db.save_rating(run_id, rating, comment_text)
|
| 141 |
-
note =
|
|
|
|
|
|
|
|
|
|
|
|
|
| 142 |
summary = _format_summary(db.fetch_rating_counts())
|
| 143 |
return gr.update(value=note), gr.update(value=summary)
|
|
|
|
| 144 |
return handler
|
| 145 |
|
| 146 |
rate_good.click(
|
|
|
|
| 30 |
g = counts.get("good", 0)
|
| 31 |
n = counts.get("not_bad", 0)
|
| 32 |
c = counts.get("critical", 0)
|
| 33 |
+
return f"**총 피드백**: {total}\n\n- Good: **{g}**\n- Not Bad: **{n}**\n- Critical: **{c}**"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
|
| 35 |
|
| 36 |
def build_feedback_tab(
|
| 37 |
client: Any,
|
| 38 |
vocabulary: list[dict],
|
| 39 |
pipeline_config: PipelineConfig,
|
| 40 |
+
elem_id_prefix: str = "compare",
|
| 41 |
) -> None:
|
| 42 |
+
"""Build the single-pipeline feedback UI. Call inside a gr.Blocks/Tab.
|
| 43 |
+
|
| 44 |
+
Args:
|
| 45 |
+
elem_id_prefix: Unique prefix for component elem_ids. Required
|
| 46 |
+
when building this UI multiple times in the same Blocks (e.g.
|
| 47 |
+
for two tabs) — Gradio errors on duplicate elem_ids.
|
| 48 |
+
"""
|
| 49 |
|
| 50 |
pipeline_key, model, prompt_key = pipeline_config
|
| 51 |
|
|
|
|
| 60 |
run_btn = gr.Button(
|
| 61 |
"교정 실행 (⌘+Enter / Ctrl+Enter)",
|
| 62 |
variant="primary",
|
| 63 |
+
elem_id=f"{elem_id_prefix}-run-btn",
|
| 64 |
)
|
| 65 |
|
| 66 |
status = gr.Markdown("")
|
|
|
|
| 98 |
|
| 99 |
start = time.time()
|
| 100 |
try:
|
| 101 |
+
result = run_pipeline(text, pipeline_key, model, prompt_key, client, vocabulary)
|
|
|
|
|
|
|
| 102 |
except Exception as exc:
|
| 103 |
return (
|
| 104 |
gr.update(value=f"에러: {exc}"),
|
|
|
|
| 138 |
def _make_rating_handler(rating: str):
|
| 139 |
def handler(run_id, comment_text):
|
| 140 |
saved = db.save_rating(run_id, rating, comment_text)
|
| 141 |
+
note = (
|
| 142 |
+
"✅ 피드백 저장됨"
|
| 143 |
+
if saved
|
| 144 |
+
else "⚠️ 저장되지 않았습니다 (먼저 교정 실행 후 피드백을 남겨주세요)"
|
| 145 |
+
)
|
| 146 |
summary = _format_summary(db.fetch_rating_counts())
|
| 147 |
return gr.update(value=note), gr.update(value=summary)
|
| 148 |
+
|
| 149 |
return handler
|
| 150 |
|
| 151 |
rate_good.click(
|
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:476b927dd3ac9f1590eb2658c56dd8f009db4a41d94b324798e8eb184eb91621
|
| 3 |
+
size 1585788
|
|
@@ -24,6 +24,11 @@ from postprocess import (
|
|
| 24 |
)
|
| 25 |
from text_splitter import split_into_bulks
|
| 26 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
# --- Pipeline definitions ---
|
| 28 |
|
| 29 |
PIPELINES: dict[str, list[dict]] = {
|
|
@@ -178,6 +183,82 @@ PIPELINES: dict[str, list[dict]] = {
|
|
| 178 |
{"name": "paragraph_dedupe", "type": "rule", "rule": "paragraph_dedupe"},
|
| 179 |
{"name": "post_process", "type": "rule", "rule": "post_process"},
|
| 180 |
],
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 181 |
# E1 taxonomy pipeline — sequential specialists, grammar last @ RE=medium
|
| 182 |
# Use with prompt_name='dev_260513_taxonomy_v6'.
|
| 183 |
"260513_taxonomy_e1": [
|
|
@@ -816,6 +897,14 @@ def apply_rule(
|
|
| 816 |
if "prefix_len" in config:
|
| 817 |
kwargs["prefix_len"] = config["prefix_len"]
|
| 818 |
return apply_paragraph_dedupe(text, original_text, **kwargs)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 819 |
|
| 820 |
return text
|
| 821 |
|
|
@@ -953,6 +1042,94 @@ def _process_single_bulk(
|
|
| 953 |
return extracted
|
| 954 |
|
| 955 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 956 |
def process_bulks_parallel(
|
| 957 |
bulks: list[str],
|
| 958 |
original_bulks: list[str],
|
|
@@ -1029,6 +1206,23 @@ def run_pipeline(
|
|
| 1029 |
prompts = load_prompts(prompt_name)
|
| 1030 |
original_text = text
|
| 1031 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1032 |
start_time = time.time()
|
| 1033 |
|
| 1034 |
# step1(basic_correction)의 <원문> 소스. 기존 파이프라인(run.py)에서 step1은
|
|
@@ -1046,7 +1240,35 @@ def run_pipeline(
|
|
| 1046 |
prev_text = text
|
| 1047 |
step_name = step.get("name", step["type"])
|
| 1048 |
try:
|
| 1049 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1050 |
text = apply_rule(step, text, original_text, vocabulary)
|
| 1051 |
elif step["type"] == "llm":
|
| 1052 |
step_model_name = step.get("model", model)
|
|
@@ -1084,11 +1306,87 @@ def run_pipeline(
|
|
| 1084 |
)
|
| 1085 |
elif step["type"] == "specialist":
|
| 1086 |
text = run_specialist(step, text, original_text, prompts, model, client)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1087 |
# If a step produced an unusably short output (< 10% of input), the
|
| 1088 |
# model likely truncated or refused. Treat as no-op so downstream
|
| 1089 |
# steps see a sensible baseline.
|
| 1090 |
if (
|
| 1091 |
-
step["type"] in ("llm", "specialist")
|
| 1092 |
and len(prev_text) > 50
|
| 1093 |
and len(text) < len(prev_text) * 0.1
|
| 1094 |
):
|
|
|
|
| 24 |
)
|
| 25 |
from text_splitter import split_into_bulks
|
| 26 |
|
| 27 |
+
# v24 helpers (self-consistency, tool-calling judge, apply_judge_decisions).
|
| 28 |
+
# Imported lazily inside step handlers to avoid initialising the PMI lookup
|
| 29 |
+
# at module import — the demo only loads the corpus when v24 is selected.
|
| 30 |
+
import pipelines_v24 as _v24
|
| 31 |
+
|
| 32 |
# --- Pipeline definitions ---
|
| 33 |
|
| 34 |
PIPELINES: dict[str, list[dict]] = {
|
|
|
|
| 183 |
{"name": "paragraph_dedupe", "type": "rule", "rule": "paragraph_dedupe"},
|
| 184 |
{"name": "post_process", "type": "rule", "rule": "post_process"},
|
| 185 |
],
|
| 186 |
+
# v24 — solar-pro3 + FT + self-consistency × 2 + tool-calling judge.
|
| 187 |
+
# 5 LLM calls/doc (FT + basic×2 + judge round1 + judge round2). Tool-calling
|
| 188 |
+
# judge consults a top-K PMI table loaded from data/pmi_compounds_top.tsv.gz
|
| 189 |
+
# (no external corpus required). Use with prompt_name='dev_260429_v24'.
|
| 190 |
+
# paragraph F1 47.04 (vs solar-pro2 prod_251231 mean 44.86, +2.18).
|
| 191 |
+
"260429_v24": [
|
| 192 |
+
{"name": "vocab_sub_pre", "type": "rule", "rule": "vocabulary"},
|
| 193 |
+
{
|
| 194 |
+
"name": "ft_correction",
|
| 195 |
+
"type": "llm",
|
| 196 |
+
"model": "ft:solar-news-correction",
|
| 197 |
+
"max_tokens": 4000,
|
| 198 |
+
},
|
| 199 |
+
{
|
| 200 |
+
"name": "basic_correction",
|
| 201 |
+
"type": "llm_self_consistency",
|
| 202 |
+
"prompt_key": "step1",
|
| 203 |
+
"reasoning_effort": "low",
|
| 204 |
+
"max_tokens": 4000,
|
| 205 |
+
"repeat": 2,
|
| 206 |
+
"repeat_temperatures": [0.0, 0.2],
|
| 207 |
+
"intersection_threshold": 2,
|
| 208 |
+
"response_format": {
|
| 209 |
+
"type": "json_schema",
|
| 210 |
+
"json_schema": {
|
| 211 |
+
"name": "proofreading_result",
|
| 212 |
+
"strict": True,
|
| 213 |
+
"schema": {
|
| 214 |
+
"type": "object",
|
| 215 |
+
"properties": {
|
| 216 |
+
"corrections": {
|
| 217 |
+
"type": "array",
|
| 218 |
+
"items": {
|
| 219 |
+
"type": "object",
|
| 220 |
+
"properties": {
|
| 221 |
+
"original": {"type": "string"},
|
| 222 |
+
"corrected": {"type": "string"},
|
| 223 |
+
"type": {"type": "string"},
|
| 224 |
+
"reason": {"type": "string"},
|
| 225 |
+
},
|
| 226 |
+
"required": ["original", "corrected", "type", "reason"],
|
| 227 |
+
"additionalProperties": False,
|
| 228 |
+
},
|
| 229 |
+
},
|
| 230 |
+
"output": {"type": "string"},
|
| 231 |
+
"explanation": {"type": "string"},
|
| 232 |
+
},
|
| 233 |
+
"required": ["corrections", "output", "explanation"],
|
| 234 |
+
"additionalProperties": False,
|
| 235 |
+
},
|
| 236 |
+
},
|
| 237 |
+
},
|
| 238 |
+
},
|
| 239 |
+
{
|
| 240 |
+
"name": "tool_calling_judge",
|
| 241 |
+
"type": "tool_calling_judge",
|
| 242 |
+
"prompt_key": "judge",
|
| 243 |
+
"reasoning_effort": "low",
|
| 244 |
+
"max_tokens": 4000,
|
| 245 |
+
"max_tool_iterations": 4,
|
| 246 |
+
"corrections_step": "basic_correction",
|
| 247 |
+
"pmi_lookup": "data/pmi_compounds_top.tsv.gz",
|
| 248 |
+
},
|
| 249 |
+
{
|
| 250 |
+
"name": "apply_judge_decisions",
|
| 251 |
+
"type": "rule",
|
| 252 |
+
"rule": "apply_judge_decisions",
|
| 253 |
+
"config": {
|
| 254 |
+
"judge_step": "tool_calling_judge",
|
| 255 |
+
"corrections_step": "basic_correction",
|
| 256 |
+
},
|
| 257 |
+
},
|
| 258 |
+
{"name": "vocabulary_reapply", "type": "rule", "rule": "vocabulary"},
|
| 259 |
+
{"name": "paragraph_dedupe", "type": "rule", "rule": "paragraph_dedupe"},
|
| 260 |
+
{"name": "post_process", "type": "rule", "rule": "post_process"},
|
| 261 |
+
],
|
| 262 |
# E1 taxonomy pipeline — sequential specialists, grammar last @ RE=medium
|
| 263 |
# Use with prompt_name='dev_260513_taxonomy_v6'.
|
| 264 |
"260513_taxonomy_e1": [
|
|
|
|
| 897 |
if "prefix_len" in config:
|
| 898 |
kwargs["prefix_len"] = config["prefix_len"]
|
| 899 |
return apply_paragraph_dedupe(text, original_text, **kwargs)
|
| 900 |
+
elif rule == "apply_judge_decisions":
|
| 901 |
+
# v24 rule. text here is the post-judge bulk text — but the judge step
|
| 902 |
+
# is a no-op on text (decisions live in pipelines_v24._JUDGE_RAW_BY_BULK
|
| 903 |
+
# keyed by bulk index). The dispatcher in run_pipeline applies this
|
| 904 |
+
# rule per-bulk by re-splitting and looking up each bulk's decisions.
|
| 905 |
+
# When called at the doc level (this path), we just return text — the
|
| 906 |
+
# actual revert happens in the per-bulk apply path inside run_pipeline.
|
| 907 |
+
return text
|
| 908 |
|
| 909 |
return text
|
| 910 |
|
|
|
|
| 1042 |
return extracted
|
| 1043 |
|
| 1044 |
|
| 1045 |
+
def _process_single_bulk_with_raw(
|
| 1046 |
+
*,
|
| 1047 |
+
bulk: str,
|
| 1048 |
+
step: dict,
|
| 1049 |
+
model: str,
|
| 1050 |
+
prompts: dict[str, str],
|
| 1051 |
+
client: OpenAI,
|
| 1052 |
+
original_bulk: str,
|
| 1053 |
+
) -> tuple[str, str]:
|
| 1054 |
+
"""Run a non-FT LLM step on one bulk and return (post_text, raw_response).
|
| 1055 |
+
|
| 1056 |
+
Used by v24's self-consistency wrapper, which needs both the extracted
|
| 1057 |
+
text AND the raw JSON (so it can intersect ``corrections`` arrays across
|
| 1058 |
+
runs). Mirrors the LLM branch of ``_process_single_bulk`` but skips the
|
| 1059 |
+
FT-specific duplication guard. The hallucination guard at ``len >= 1.4×``
|
| 1060 |
+
is preserved.
|
| 1061 |
+
|
| 1062 |
+
Returns ``("", "")`` on call failure rather than raising — the caller
|
| 1063 |
+
falls back to per-step error handling.
|
| 1064 |
+
"""
|
| 1065 |
+
step_model = step.get("model", model)
|
| 1066 |
+
prompt_key = step.get("prompt_key")
|
| 1067 |
+
reasoning_effort = step.get("reasoning_effort")
|
| 1068 |
+
max_tokens = step.get("max_tokens")
|
| 1069 |
+
response_format = step.get("response_format")
|
| 1070 |
+
temperature = step.get("temperature")
|
| 1071 |
+
|
| 1072 |
+
if prompt_key:
|
| 1073 |
+
system_prompt = prompts.get(prompt_key, "")
|
| 1074 |
+
if prompt_key == "step1":
|
| 1075 |
+
original_sep = original_bulk.replace("\n", PARAGRAPH_SEP)
|
| 1076 |
+
bulk_sep = bulk.replace("\n", PARAGRAPH_SEP)
|
| 1077 |
+
user_content = (
|
| 1078 |
+
f"<원문>\n{original_sep}\n</원문>\n\n"
|
| 1079 |
+
f"<교열_모델_수정결과>\n{bulk_sep}\n</교열_모델_수정결과>"
|
| 1080 |
+
)
|
| 1081 |
+
else:
|
| 1082 |
+
user_content = bulk.replace("\n", PARAGRAPH_SEP)
|
| 1083 |
+
else:
|
| 1084 |
+
system_prompt = FT_SYSTEM_PROMPT
|
| 1085 |
+
user_content = bulk
|
| 1086 |
+
|
| 1087 |
+
call_kwargs: dict = {
|
| 1088 |
+
"client": client,
|
| 1089 |
+
"system_prompt": system_prompt,
|
| 1090 |
+
"user_content": user_content,
|
| 1091 |
+
"model": step_model,
|
| 1092 |
+
"reasoning_effort": reasoning_effort,
|
| 1093 |
+
"max_tokens": max_tokens,
|
| 1094 |
+
"response_format": response_format,
|
| 1095 |
+
}
|
| 1096 |
+
if temperature is not None:
|
| 1097 |
+
call_kwargs["temperature"] = temperature
|
| 1098 |
+
|
| 1099 |
+
try:
|
| 1100 |
+
raw = call_llm(**{k: v for k, v in call_kwargs.items() if v is not None})
|
| 1101 |
+
except Exception as exc: # noqa: BLE001
|
| 1102 |
+
logger.warning(
|
| 1103 |
+
"[_process_single_bulk_with_raw] call_llm failed: %s", exc
|
| 1104 |
+
)
|
| 1105 |
+
return ("", "")
|
| 1106 |
+
|
| 1107 |
+
extracted = extract_json_output(raw, fallback=bulk)
|
| 1108 |
+
if prompt_key:
|
| 1109 |
+
extracted = extracted.replace(PARAGRAPH_SEP, "\n")
|
| 1110 |
+
|
| 1111 |
+
# Hallucination guard — same shape as _process_single_bulk's LLM branch.
|
| 1112 |
+
if extracted and len(extracted) >= len(bulk) * 1.4:
|
| 1113 |
+
stripped = extracted.strip()
|
| 1114 |
+
|
| 1115 |
+
def _leading_lookup(needle_src: str) -> str | None:
|
| 1116 |
+
head = needle_src.strip()[:80]
|
| 1117 |
+
if len(head) < 30:
|
| 1118 |
+
return None
|
| 1119 |
+
second = stripped.find(head, len(head))
|
| 1120 |
+
if second > len(head):
|
| 1121 |
+
return stripped[:second].rstrip()
|
| 1122 |
+
return None
|
| 1123 |
+
|
| 1124 |
+
recovered = _leading_lookup(bulk) or _leading_lookup(original_bulk)
|
| 1125 |
+
if recovered:
|
| 1126 |
+
extracted = recovered
|
| 1127 |
+
else:
|
| 1128 |
+
extracted = bulk
|
| 1129 |
+
|
| 1130 |
+
return (extracted, raw)
|
| 1131 |
+
|
| 1132 |
+
|
| 1133 |
def process_bulks_parallel(
|
| 1134 |
bulks: list[str],
|
| 1135 |
original_bulks: list[str],
|
|
|
|
| 1206 |
prompts = load_prompts(prompt_name)
|
| 1207 |
original_text = text
|
| 1208 |
|
| 1209 |
+
# v24 uses bulk-indexed scratch state (per-bulk corrections + judge
|
| 1210 |
+
# decisions). Reset at the start of each request so concurrent users
|
| 1211 |
+
# don't see each other's data.
|
| 1212 |
+
_v24.reset_v24_state()
|
| 1213 |
+
|
| 1214 |
+
# Locate v24's PMI table (the demo bundles a top-K subset). Activate
|
| 1215 |
+
# only if the pipeline declares one — otherwise the lookup is unused.
|
| 1216 |
+
for s in steps:
|
| 1217 |
+
pmi_path = s.get("pmi_lookup")
|
| 1218 |
+
if pmi_path:
|
| 1219 |
+
full_path = Path(__file__).parent / pmi_path
|
| 1220 |
+
try:
|
| 1221 |
+
_v24.set_active_pmi_path(full_path)
|
| 1222 |
+
except Exception as exc: # noqa: BLE001
|
| 1223 |
+
logger.warning("[v24] PMI load failed at %s: %s", full_path, exc)
|
| 1224 |
+
break
|
| 1225 |
+
|
| 1226 |
start_time = time.time()
|
| 1227 |
|
| 1228 |
# step1(basic_correction)의 <원문> 소스. 기존 파이프라인(run.py)에서 step1은
|
|
|
|
| 1240 |
prev_text = text
|
| 1241 |
step_name = step.get("name", step["type"])
|
| 1242 |
try:
|
| 1243 |
+
# v24 apply_judge_decisions handled BEFORE the generic rule
|
| 1244 |
+
# branch so the per-bulk revert path (which needs index-aligned
|
| 1245 |
+
# bulks) wins over apply_rule's no-op.
|
| 1246 |
+
if step["type"] == "rule" and step.get("rule") == "apply_judge_decisions":
|
| 1247 |
+
cfg = step.get("config", {})
|
| 1248 |
+
bulks = split_into_bulks(text)
|
| 1249 |
+
if bulks:
|
| 1250 |
+
judge_step_name = cfg.get("judge_step", "tool_calling_judge")
|
| 1251 |
+
corr_step_name = cfg.get("corrections_step", "basic_correction")
|
| 1252 |
+
reverted: list[str] = []
|
| 1253 |
+
for i, b in enumerate(bulks):
|
| 1254 |
+
try:
|
| 1255 |
+
reverted.append(
|
| 1256 |
+
_v24.apply_judge_decisions_for_bulk(
|
| 1257 |
+
bulk_idx=i,
|
| 1258 |
+
bulk=b,
|
| 1259 |
+
judge_step_name=judge_step_name,
|
| 1260 |
+
corrections_step_name=corr_step_name,
|
| 1261 |
+
)
|
| 1262 |
+
)
|
| 1263 |
+
except Exception as exc: # noqa: BLE001
|
| 1264 |
+
logger.warning(
|
| 1265 |
+
"[apply_judge_decisions] bulk %d failed (%s); keeping",
|
| 1266 |
+
i,
|
| 1267 |
+
exc,
|
| 1268 |
+
)
|
| 1269 |
+
reverted.append(b)
|
| 1270 |
+
text = "\n".join(reverted)
|
| 1271 |
+
elif step["type"] == "rule":
|
| 1272 |
text = apply_rule(step, text, original_text, vocabulary)
|
| 1273 |
elif step["type"] == "llm":
|
| 1274 |
step_model_name = step.get("model", model)
|
|
|
|
| 1306 |
)
|
| 1307 |
elif step["type"] == "specialist":
|
| 1308 |
text = run_specialist(step, text, original_text, prompts, model, client)
|
| 1309 |
+
elif step["type"] == "llm_self_consistency":
|
| 1310 |
+
# v24 basic_correction: per-bulk × N runs, intersect corrections.
|
| 1311 |
+
# Splits like a normal LLM step (per-paragraph bulks), but each
|
| 1312 |
+
# bulk runs through pipelines_v24.run_self_consistency_for_bulk
|
| 1313 |
+
# which calls _process_single_bulk_with_raw N times and emits
|
| 1314 |
+
# the synthesized post-text + raw JSON.
|
| 1315 |
+
bulks = split_into_bulks(text)
|
| 1316 |
+
if bulks:
|
| 1317 |
+
if step.get("prompt_key") == "step1":
|
| 1318 |
+
orig_candidate = split_into_bulks(pre_ft_snapshot)
|
| 1319 |
+
if len(orig_candidate) == len(bulks):
|
| 1320 |
+
original_bulks = orig_candidate
|
| 1321 |
+
else:
|
| 1322 |
+
original_bulks = [pre_ft_snapshot] * len(bulks)
|
| 1323 |
+
else:
|
| 1324 |
+
original_bulks = list(bulks)
|
| 1325 |
+
|
| 1326 |
+
def _sc_one(idx_bulk):
|
| 1327 |
+
i, b = idx_bulk
|
| 1328 |
+
return _v24.run_self_consistency_for_bulk(
|
| 1329 |
+
bulk_idx=i,
|
| 1330 |
+
bulk=b,
|
| 1331 |
+
original_bulk=original_bulks[i],
|
| 1332 |
+
step=step,
|
| 1333 |
+
model=model,
|
| 1334 |
+
prompts=prompts,
|
| 1335 |
+
client=client,
|
| 1336 |
+
process_single_bulk_fn=_process_single_bulk_with_raw,
|
| 1337 |
+
)
|
| 1338 |
+
|
| 1339 |
+
if len(bulks) == 1:
|
| 1340 |
+
processed = [_sc_one((0, bulks[0]))]
|
| 1341 |
+
else:
|
| 1342 |
+
with ThreadPoolExecutor(max_workers=10) as ex:
|
| 1343 |
+
processed = list(
|
| 1344 |
+
ex.map(_sc_one, list(enumerate(bulks)))
|
| 1345 |
+
)
|
| 1346 |
+
text = "\n".join(
|
| 1347 |
+
str(p) if not isinstance(p, str) else p for p in processed
|
| 1348 |
+
)
|
| 1349 |
+
elif step["type"] == "tool_calling_judge":
|
| 1350 |
+
# v24 judge: per-bulk 2-round emulation. Decisions are stashed
|
| 1351 |
+
# in _JUDGE_RAW_BY_BULK keyed by bulk index. text passes
|
| 1352 |
+
# through unchanged — the apply_judge_decisions rule reverts.
|
| 1353 |
+
bulks = split_into_bulks(text)
|
| 1354 |
+
if bulks:
|
| 1355 |
+
# Always anchor on pre-FT snapshot for <원문> since the
|
| 1356 |
+
# judge is reasoning about whether each correction was
|
| 1357 |
+
# over-correction relative to the source article.
|
| 1358 |
+
orig_candidate = split_into_bulks(pre_ft_snapshot)
|
| 1359 |
+
if len(orig_candidate) == len(bulks):
|
| 1360 |
+
original_bulks = orig_candidate
|
| 1361 |
+
else:
|
| 1362 |
+
original_bulks = [pre_ft_snapshot] * len(bulks)
|
| 1363 |
+
corrections_step = step.get("corrections_step", "basic_correction")
|
| 1364 |
+
|
| 1365 |
+
def _judge_one(idx_bulk):
|
| 1366 |
+
i, b = idx_bulk
|
| 1367 |
+
return _v24.run_tool_calling_judge_for_bulk(
|
| 1368 |
+
bulk_idx=i,
|
| 1369 |
+
bulk=b,
|
| 1370 |
+
original_bulk=original_bulks[i],
|
| 1371 |
+
corrections_step_name=corrections_step,
|
| 1372 |
+
step=step,
|
| 1373 |
+
model=model,
|
| 1374 |
+
prompts=prompts,
|
| 1375 |
+
client=client,
|
| 1376 |
+
call_llm_fn=call_llm,
|
| 1377 |
+
)
|
| 1378 |
+
|
| 1379 |
+
if len(bulks) == 1:
|
| 1380 |
+
_judge_one((0, bulks[0]))
|
| 1381 |
+
else:
|
| 1382 |
+
with ThreadPoolExecutor(max_workers=10) as ex:
|
| 1383 |
+
list(ex.map(_judge_one, list(enumerate(bulks))))
|
| 1384 |
+
# text unchanged
|
| 1385 |
# If a step produced an unusably short output (< 10% of input), the
|
| 1386 |
# model likely truncated or refused. Treat as no-op so downstream
|
| 1387 |
# steps see a sensible baseline.
|
| 1388 |
if (
|
| 1389 |
+
step["type"] in ("llm", "specialist", "llm_self_consistency")
|
| 1390 |
and len(prev_text) > 50
|
| 1391 |
and len(text) < len(prev_text) * 0.1
|
| 1392 |
):
|
|
@@ -0,0 +1,737 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""v24-specific pipeline helpers for the HF demo.
|
| 2 |
+
|
| 3 |
+
Adds three step types on top of the existing engine in `pipelines.py`:
|
| 4 |
+
|
| 5 |
+
1. ``llm_self_consistency`` — runs `basic_correction` N times at varying
|
| 6 |
+
temperatures, then keeps only corrections that appear in
|
| 7 |
+
≥ ``intersection_threshold`` runs. The synthesized corrections are
|
| 8 |
+
applied to the FT-pass text to produce both a final text AND a
|
| 9 |
+
synthetic raw-JSON blob downstream judges can read.
|
| 10 |
+
|
| 11 |
+
2. ``tool_calling_judge`` — two-round emulation of OpenAI tool-calling
|
| 12 |
+
since the Upstage chat-completions endpoint doesn't natively support
|
| 13 |
+
``tools=[...]``. Round 1 emits a ``{"lookups": [...]}`` JSON; the
|
| 14 |
+
engine resolves each lookup against the local PMI/unigram tables and
|
| 15 |
+
feeds results back. Round 2 emits the final
|
| 16 |
+
``{"decisions": [{correction_index, verdict, reason}]}`` blob.
|
| 17 |
+
|
| 18 |
+
3. ``apply_judge_decisions`` — rule that reads the corrections list
|
| 19 |
+
(from `basic_correction`'s synthetic raw JSON) and the judge's
|
| 20 |
+
verdicts, and reverts any ``verdict="revert"`` correction by
|
| 21 |
+
replacing the corrected span with the original.
|
| 22 |
+
|
| 23 |
+
The PMI lookup is served by an in-memory dict loaded once from the
|
| 24 |
+
``data/pmi_compounds_top.tsv.gz`` file produced by
|
| 25 |
+
``build_pmi_top_for_demo.py``. No external corpus dependency at runtime.
|
| 26 |
+
"""
|
| 27 |
+
|
| 28 |
+
from __future__ import annotations
|
| 29 |
+
|
| 30 |
+
import gzip
|
| 31 |
+
import json
|
| 32 |
+
import logging
|
| 33 |
+
import math
|
| 34 |
+
from concurrent.futures import ThreadPoolExecutor
|
| 35 |
+
from pathlib import Path
|
| 36 |
+
from typing import Any
|
| 37 |
+
|
| 38 |
+
logger = logging.getLogger(__name__)
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
# ---------------------------------------------------------------------------
|
| 42 |
+
# PMI lookup (loaded once at module import / first call)
|
| 43 |
+
# ---------------------------------------------------------------------------
|
| 44 |
+
|
| 45 |
+
# Per-corpus caches (keyed by absolute path so multiple PMI sources can
|
| 46 |
+
# coexist if we ever ship more than one).
|
| 47 |
+
_BIGRAM_CACHE: dict[Path, dict[str, dict[str, tuple[float, int]]]] = {}
|
| 48 |
+
_UNIGRAM_CACHE: dict[Path, dict[str, int]] = {}
|
| 49 |
+
_TOTAL_TOKENS_CACHE: dict[Path, int] = {}
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+
def load_pmi_lookup(path: str | Path) -> None:
|
| 53 |
+
"""Load PMI table from the gzipped TSV produced by build_pmi_top_for_demo.
|
| 54 |
+
|
| 55 |
+
File format::
|
| 56 |
+
# SECTION: bigrams
|
| 57 |
+
# left<TAB>right<TAB>count<TAB>pmi
|
| 58 |
+
...rows...
|
| 59 |
+
# SECTION: unigrams
|
| 60 |
+
# token<TAB>count
|
| 61 |
+
...rows...
|
| 62 |
+
|
| 63 |
+
Idempotent — second call with same path is a no-op.
|
| 64 |
+
"""
|
| 65 |
+
target = Path(path).resolve()
|
| 66 |
+
if target in _BIGRAM_CACHE:
|
| 67 |
+
return
|
| 68 |
+
if not target.exists():
|
| 69 |
+
logger.warning("[pmi_v24] file not found: %s", target)
|
| 70 |
+
_BIGRAM_CACHE[target] = {}
|
| 71 |
+
_UNIGRAM_CACHE[target] = {}
|
| 72 |
+
_TOTAL_TOKENS_CACHE[target] = 1
|
| 73 |
+
return
|
| 74 |
+
|
| 75 |
+
bigram: dict[str, dict[str, tuple[float, int]]] = {}
|
| 76 |
+
unigram: dict[str, int] = {}
|
| 77 |
+
section: str | None = None
|
| 78 |
+
with gzip.open(target, "rt", encoding="utf-8") as f:
|
| 79 |
+
for line in f:
|
| 80 |
+
line = line.rstrip("\n")
|
| 81 |
+
if not line:
|
| 82 |
+
continue
|
| 83 |
+
if line.startswith("# SECTION: bigrams"):
|
| 84 |
+
section = "bigrams"
|
| 85 |
+
continue
|
| 86 |
+
if line.startswith("# SECTION: unigrams"):
|
| 87 |
+
section = "unigrams"
|
| 88 |
+
continue
|
| 89 |
+
if line.startswith("#"):
|
| 90 |
+
continue
|
| 91 |
+
parts = line.split("\t")
|
| 92 |
+
if section == "bigrams" and len(parts) >= 4:
|
| 93 |
+
try:
|
| 94 |
+
left, right = parts[0], parts[1]
|
| 95 |
+
c = int(parts[2])
|
| 96 |
+
pmi = float(parts[3])
|
| 97 |
+
except ValueError:
|
| 98 |
+
continue
|
| 99 |
+
bigram.setdefault(left, {})[right] = (pmi, c)
|
| 100 |
+
elif section == "unigrams" and len(parts) >= 2:
|
| 101 |
+
try:
|
| 102 |
+
unigram[parts[0]] = int(parts[1])
|
| 103 |
+
except ValueError:
|
| 104 |
+
continue
|
| 105 |
+
|
| 106 |
+
total = sum(unigram.values()) or 1
|
| 107 |
+
_BIGRAM_CACHE[target] = bigram
|
| 108 |
+
_UNIGRAM_CACHE[target] = unigram
|
| 109 |
+
_TOTAL_TOKENS_CACHE[target] = total
|
| 110 |
+
logger.info(
|
| 111 |
+
"[pmi_v24] loaded %s: %d unigrams, %d bigrams (over %d left-tokens)",
|
| 112 |
+
target.name,
|
| 113 |
+
len(unigram),
|
| 114 |
+
sum(len(v) for v in bigram.values()),
|
| 115 |
+
len(bigram),
|
| 116 |
+
)
|
| 117 |
+
|
| 118 |
+
|
| 119 |
+
# Active PMI path for the current process. Set by the engine via
|
| 120 |
+
# ``set_active_pmi_path`` before dispatching tool calls.
|
| 121 |
+
_ACTIVE_PMI: Path | None = None
|
| 122 |
+
|
| 123 |
+
|
| 124 |
+
def set_active_pmi_path(path: str | Path) -> None:
|
| 125 |
+
global _ACTIVE_PMI
|
| 126 |
+
target = Path(path).resolve()
|
| 127 |
+
load_pmi_lookup(target)
|
| 128 |
+
_ACTIVE_PMI = target
|
| 129 |
+
|
| 130 |
+
|
| 131 |
+
def _bigram() -> dict[str, dict[str, tuple[float, int]]]:
|
| 132 |
+
if _ACTIVE_PMI is None:
|
| 133 |
+
return {}
|
| 134 |
+
return _BIGRAM_CACHE.get(_ACTIVE_PMI, {})
|
| 135 |
+
|
| 136 |
+
|
| 137 |
+
def _unigram() -> dict[str, int]:
|
| 138 |
+
if _ACTIVE_PMI is None:
|
| 139 |
+
return {}
|
| 140 |
+
return _UNIGRAM_CACHE.get(_ACTIVE_PMI, {})
|
| 141 |
+
|
| 142 |
+
|
| 143 |
+
def lookup_pmi(left: str, right: str) -> dict[str, Any]:
|
| 144 |
+
if not left or not right:
|
| 145 |
+
return {"pmi": None, "count": 0}
|
| 146 |
+
inner = _bigram().get(left, {})
|
| 147 |
+
if right not in inner:
|
| 148 |
+
return {"pmi": None, "count": 0}
|
| 149 |
+
pmi, count = inner[right]
|
| 150 |
+
return {"pmi": round(pmi, 4), "count": count}
|
| 151 |
+
|
| 152 |
+
|
| 153 |
+
def lookup_pmi_joined(joined: str) -> dict[str, Any]:
|
| 154 |
+
if not joined or len(joined) < 4:
|
| 155 |
+
return {"pmi": None, "count": 0, "left": None, "right": None}
|
| 156 |
+
bigram = _bigram()
|
| 157 |
+
best: tuple[float, int, str, str] | None = None
|
| 158 |
+
for i in range(2, len(joined) - 1):
|
| 159 |
+
left = joined[:i]
|
| 160 |
+
right = joined[i:]
|
| 161 |
+
inner = bigram.get(left, {})
|
| 162 |
+
if right in inner:
|
| 163 |
+
pmi, c = inner[right]
|
| 164 |
+
if best is None or pmi > best[0]:
|
| 165 |
+
best = (pmi, c, left, right)
|
| 166 |
+
if best is None:
|
| 167 |
+
return {"pmi": None, "count": 0, "left": None, "right": None}
|
| 168 |
+
pmi, c, left, right = best
|
| 169 |
+
return {"pmi": round(pmi, 4), "count": c, "left": left, "right": right}
|
| 170 |
+
|
| 171 |
+
|
| 172 |
+
def lookup_unigram_freq(token: str) -> dict[str, Any]:
|
| 173 |
+
if not token:
|
| 174 |
+
return {"count": 0}
|
| 175 |
+
return {"count": _unigram().get(token, 0)}
|
| 176 |
+
|
| 177 |
+
|
| 178 |
+
def dispatch_tool(tool: str, args: dict[str, Any]) -> dict[str, Any]:
|
| 179 |
+
"""Run a single tool. Wraps in try/except — never raises."""
|
| 180 |
+
if not isinstance(args, dict):
|
| 181 |
+
return {"ok": False, "error": "args must be an object"}
|
| 182 |
+
try:
|
| 183 |
+
if tool == "lookup_pmi":
|
| 184 |
+
return {
|
| 185 |
+
"ok": True,
|
| 186 |
+
"result": lookup_pmi(
|
| 187 |
+
str(args.get("left", "")),
|
| 188 |
+
str(args.get("right", "")),
|
| 189 |
+
),
|
| 190 |
+
}
|
| 191 |
+
if tool == "lookup_pmi_joined":
|
| 192 |
+
return {
|
| 193 |
+
"ok": True,
|
| 194 |
+
"result": lookup_pmi_joined(str(args.get("joined", ""))),
|
| 195 |
+
}
|
| 196 |
+
if tool == "lookup_unigram_freq":
|
| 197 |
+
return {
|
| 198 |
+
"ok": True,
|
| 199 |
+
"result": lookup_unigram_freq(str(args.get("token", ""))),
|
| 200 |
+
}
|
| 201 |
+
# Note: lookup_chosun_dict is intentionally omitted from the demo
|
| 202 |
+
# build (the dictionaries live on internal storage).
|
| 203 |
+
return {"ok": False, "error": f"unknown tool: {tool}"}
|
| 204 |
+
except Exception as exc: # noqa: BLE001
|
| 205 |
+
logger.warning("[pmi_v24.dispatch] %s raised: %s", tool, exc)
|
| 206 |
+
return {"ok": False, "error": f"{type(exc).__name__}: {exc}"}
|
| 207 |
+
|
| 208 |
+
|
| 209 |
+
# ---------------------------------------------------------------------------
|
| 210 |
+
# JSON parsers — robust against truncated / fenced model output
|
| 211 |
+
# ---------------------------------------------------------------------------
|
| 212 |
+
|
| 213 |
+
|
| 214 |
+
def _parse_json_robust(content: str) -> dict | None:
|
| 215 |
+
"""Parse JSON object from arbitrary LLM content.
|
| 216 |
+
|
| 217 |
+
Tries: raw, ```json``` block, last brace-balanced object.
|
| 218 |
+
"""
|
| 219 |
+
if not content:
|
| 220 |
+
return None
|
| 221 |
+
# Raw
|
| 222 |
+
try:
|
| 223 |
+
data = json.loads(content)
|
| 224 |
+
if isinstance(data, dict):
|
| 225 |
+
return data
|
| 226 |
+
except (json.JSONDecodeError, AttributeError):
|
| 227 |
+
pass
|
| 228 |
+
# Fenced
|
| 229 |
+
import re
|
| 230 |
+
|
| 231 |
+
for m in reversed(list(re.finditer(r"```(?:json)?\s*(.*?)```", content, re.DOTALL))):
|
| 232 |
+
try:
|
| 233 |
+
data = json.loads(m.group(1).strip())
|
| 234 |
+
if isinstance(data, dict):
|
| 235 |
+
return data
|
| 236 |
+
except (json.JSONDecodeError, AttributeError):
|
| 237 |
+
continue
|
| 238 |
+
# Brace-balanced (last object)
|
| 239 |
+
last_open = content.rfind("{")
|
| 240 |
+
while last_open != -1:
|
| 241 |
+
depth = 0
|
| 242 |
+
for i in range(last_open, len(content)):
|
| 243 |
+
if content[i] == "{":
|
| 244 |
+
depth += 1
|
| 245 |
+
elif content[i] == "}":
|
| 246 |
+
depth -= 1
|
| 247 |
+
if depth == 0:
|
| 248 |
+
try:
|
| 249 |
+
data = json.loads(content[last_open : i + 1])
|
| 250 |
+
if isinstance(data, dict):
|
| 251 |
+
return data
|
| 252 |
+
except (json.JSONDecodeError, AttributeError):
|
| 253 |
+
pass
|
| 254 |
+
break
|
| 255 |
+
# Try the previous opening brace
|
| 256 |
+
last_open = content.rfind("{", 0, last_open)
|
| 257 |
+
return None
|
| 258 |
+
|
| 259 |
+
|
| 260 |
+
def _extract_corrections(raw_json: str) -> list[dict]:
|
| 261 |
+
"""Extract corrections array from a basic_correction JSON blob."""
|
| 262 |
+
data = _parse_json_robust(raw_json) or {}
|
| 263 |
+
arr = data.get("corrections", [])
|
| 264 |
+
if not isinstance(arr, list):
|
| 265 |
+
return []
|
| 266 |
+
return [
|
| 267 |
+
c
|
| 268 |
+
for c in arr
|
| 269 |
+
if isinstance(c, dict)
|
| 270 |
+
and isinstance(c.get("original"), str)
|
| 271 |
+
and isinstance(c.get("corrected"), str)
|
| 272 |
+
]
|
| 273 |
+
|
| 274 |
+
|
| 275 |
+
# ---------------------------------------------------------------------------
|
| 276 |
+
# Self-consistency — run basic_correction N times and intersect
|
| 277 |
+
# ---------------------------------------------------------------------------
|
| 278 |
+
|
| 279 |
+
# Stash for the pipeline runner: maps `(bulk_idx, step_name)` to the
|
| 280 |
+
# synthesized raw JSON of basic_correction so a downstream judge step can
|
| 281 |
+
# read corrections by index. Keyed at the **bulk** level — each paragraph
|
| 282 |
+
# has its own corrections list because chunking is per-paragraph.
|
| 283 |
+
_SC_RAW_BY_BULK: dict[tuple[int, str], str] = {}
|
| 284 |
+
_JUDGE_RAW_BY_BULK: dict[tuple[int, str], str] = {}
|
| 285 |
+
|
| 286 |
+
|
| 287 |
+
def reset_v24_state() -> None:
|
| 288 |
+
"""Clear the per-document caches between pipeline runs.
|
| 289 |
+
|
| 290 |
+
Call at the start of each ``run_pipeline`` invocation so state from a
|
| 291 |
+
previous request never leaks into the next one. Without this, two
|
| 292 |
+
consecutive calls would see stale bulk-indexed corrections from an
|
| 293 |
+
earlier doc.
|
| 294 |
+
"""
|
| 295 |
+
_SC_RAW_BY_BULK.clear()
|
| 296 |
+
_JUDGE_RAW_BY_BULK.clear()
|
| 297 |
+
|
| 298 |
+
|
| 299 |
+
def _apply_corrections_to_text(text: str, corrections: list[dict]) -> str:
|
| 300 |
+
"""Apply (original → corrected) replacements left-to-right, longest-first.
|
| 301 |
+
|
| 302 |
+
This mirrors how ``apply_specialist_corrections`` plans non-overlapping
|
| 303 |
+
edits, so the resulting text is consistent with what the basic_correction
|
| 304 |
+
LLM intended.
|
| 305 |
+
"""
|
| 306 |
+
if not corrections:
|
| 307 |
+
return text
|
| 308 |
+
seen: set[tuple[str, str]] = set()
|
| 309 |
+
clean: list[dict[str, str]] = []
|
| 310 |
+
for c in corrections:
|
| 311 |
+
orig = c.get("original")
|
| 312 |
+
new = c.get("corrected")
|
| 313 |
+
if not isinstance(orig, str) or not isinstance(new, str):
|
| 314 |
+
continue
|
| 315 |
+
if not orig or orig == new:
|
| 316 |
+
continue
|
| 317 |
+
key = (orig, new)
|
| 318 |
+
if key in seen:
|
| 319 |
+
continue
|
| 320 |
+
seen.add(key)
|
| 321 |
+
clean.append({"original": orig, "corrected": new})
|
| 322 |
+
clean.sort(key=lambda c: -len(c["original"]))
|
| 323 |
+
|
| 324 |
+
edits: list[tuple[int, int, str]] = []
|
| 325 |
+
for c in clean:
|
| 326 |
+
idx = text.find(c["original"])
|
| 327 |
+
while idx != -1:
|
| 328 |
+
end = idx + len(c["original"])
|
| 329 |
+
if not any(not (end <= s or idx >= e) for s, e, _ in edits):
|
| 330 |
+
edits.append((idx, end, c["corrected"]))
|
| 331 |
+
break
|
| 332 |
+
idx = text.find(c["original"], idx + 1)
|
| 333 |
+
edits.sort(key=lambda e: e[0], reverse=True)
|
| 334 |
+
for start, end, corrected in edits:
|
| 335 |
+
text = text[:start] + corrected + text[end:]
|
| 336 |
+
return text
|
| 337 |
+
|
| 338 |
+
|
| 339 |
+
def run_self_consistency_for_bulk(
|
| 340 |
+
bulk_idx: int,
|
| 341 |
+
bulk: str,
|
| 342 |
+
original_bulk: str,
|
| 343 |
+
step: dict,
|
| 344 |
+
model: str,
|
| 345 |
+
prompts: dict[str, str],
|
| 346 |
+
client: Any,
|
| 347 |
+
*,
|
| 348 |
+
process_single_bulk_fn: Any,
|
| 349 |
+
) -> str:
|
| 350 |
+
"""Run basic_correction N times for one bulk and intersect corrections.
|
| 351 |
+
|
| 352 |
+
Calls the existing ``_process_single_bulk`` helper from ``pipelines.py``
|
| 353 |
+
once per temperature, parses each response's ``corrections`` array, and
|
| 354 |
+
keeps only entries seen in ≥ ``intersection_threshold`` runs.
|
| 355 |
+
|
| 356 |
+
Stashes the synthesized raw JSON in ``_SC_RAW_BY_BULK[(bulk_idx, step_name)]``
|
| 357 |
+
so the downstream judge step can read corrections by index.
|
| 358 |
+
|
| 359 |
+
Returns the post-correction text for this bulk.
|
| 360 |
+
"""
|
| 361 |
+
repeat = int(step.get("repeat", 2) or 2)
|
| 362 |
+
temps = list(step.get("repeat_temperatures") or [])
|
| 363 |
+
while len(temps) < repeat:
|
| 364 |
+
temps.append(0.0 if not temps else temps[-1])
|
| 365 |
+
threshold = int(step.get("intersection_threshold", 2) or 2)
|
| 366 |
+
step_name = step.get("name", "basic_correction")
|
| 367 |
+
|
| 368 |
+
raw_per_run: list[str] = []
|
| 369 |
+
corrs_per_run: list[list[dict]] = []
|
| 370 |
+
|
| 371 |
+
def _run_once(i: int) -> tuple[str, list[dict]]:
|
| 372 |
+
sub_step = dict(step)
|
| 373 |
+
sub_step["temperature"] = temps[i]
|
| 374 |
+
# Drop self-consistency fields so the inner call is a normal LLM step.
|
| 375 |
+
sub_step.pop("repeat", None)
|
| 376 |
+
sub_step.pop("repeat_temperatures", None)
|
| 377 |
+
sub_step.pop("intersection_threshold", None)
|
| 378 |
+
# ``process_single_bulk_fn`` is the existing _process_single_bulk
|
| 379 |
+
# function from pipelines.py — invoking it returns the *post*-text
|
| 380 |
+
# of the bulk (i.e. JSON-extracted "output" field). We need the raw
|
| 381 |
+
# response too, so we re-parse from the function's cache. Easier:
|
| 382 |
+
# call the helper that we provide which exposes both. The injected
|
| 383 |
+
# helper signature: (bulk, step, model, prompts, client, original_bulk)
|
| 384 |
+
# → (text, raw_response).
|
| 385 |
+
text_out, raw_out = process_single_bulk_fn(
|
| 386 |
+
bulk=bulk,
|
| 387 |
+
step=sub_step,
|
| 388 |
+
model=model,
|
| 389 |
+
prompts=prompts,
|
| 390 |
+
client=client,
|
| 391 |
+
original_bulk=original_bulk,
|
| 392 |
+
)
|
| 393 |
+
return raw_out or "", _extract_corrections(raw_out or "")
|
| 394 |
+
|
| 395 |
+
# Run sequentially to keep parallelism predictable per bulk; the outer
|
| 396 |
+
# process_bulks_parallel already parallelises across bulks.
|
| 397 |
+
for i in range(repeat):
|
| 398 |
+
try:
|
| 399 |
+
raw, corrs = _run_once(i)
|
| 400 |
+
except Exception as exc: # noqa: BLE001
|
| 401 |
+
logger.warning(
|
| 402 |
+
"[v24.self_consistency] bulk %d run %d failed: %s",
|
| 403 |
+
bulk_idx,
|
| 404 |
+
i,
|
| 405 |
+
exc,
|
| 406 |
+
)
|
| 407 |
+
raw, corrs = "", []
|
| 408 |
+
raw_per_run.append(raw)
|
| 409 |
+
corrs_per_run.append(corrs)
|
| 410 |
+
|
| 411 |
+
# Intersect: a correction survives if it appears in ≥ threshold runs.
|
| 412 |
+
counts: dict[tuple[str, str], int] = {}
|
| 413 |
+
metas: dict[tuple[str, str], dict] = {}
|
| 414 |
+
for run_corrs in corrs_per_run:
|
| 415 |
+
seen_in_run: set[tuple[str, str]] = set()
|
| 416 |
+
for c in run_corrs:
|
| 417 |
+
key = (c["original"], c["corrected"])
|
| 418 |
+
if key in seen_in_run:
|
| 419 |
+
continue
|
| 420 |
+
seen_in_run.add(key)
|
| 421 |
+
counts[key] = counts.get(key, 0) + 1
|
| 422 |
+
metas.setdefault(
|
| 423 |
+
key,
|
| 424 |
+
{
|
| 425 |
+
"original": c["original"],
|
| 426 |
+
"corrected": c["corrected"],
|
| 427 |
+
"type": c.get("type", ""),
|
| 428 |
+
"reason": c.get("reason", ""),
|
| 429 |
+
},
|
| 430 |
+
)
|
| 431 |
+
survivors = [metas[k] for k, n in counts.items() if n >= threshold]
|
| 432 |
+
|
| 433 |
+
final_text = _apply_corrections_to_text(bulk, survivors)
|
| 434 |
+
synth_raw = json.dumps(
|
| 435 |
+
{
|
| 436 |
+
"corrections": survivors,
|
| 437 |
+
"output": final_text,
|
| 438 |
+
"explanation": (
|
| 439 |
+
f"self-consistency intersection (n={repeat}, threshold={threshold}, "
|
| 440 |
+
f"survivors={len(survivors)})"
|
| 441 |
+
),
|
| 442 |
+
},
|
| 443 |
+
ensure_ascii=False,
|
| 444 |
+
)
|
| 445 |
+
_SC_RAW_BY_BULK[(bulk_idx, step_name)] = synth_raw
|
| 446 |
+
return final_text
|
| 447 |
+
|
| 448 |
+
|
| 449 |
+
# ---------------------------------------------------------------------------
|
| 450 |
+
# Tool-calling judge — two-round emulation
|
| 451 |
+
# ---------------------------------------------------------------------------
|
| 452 |
+
|
| 453 |
+
_TOOL_ENUM = ["lookup_pmi", "lookup_pmi_joined", "lookup_unigram_freq"]
|
| 454 |
+
|
| 455 |
+
_ROUND1_SCHEMA: dict = {
|
| 456 |
+
"type": "json_schema",
|
| 457 |
+
"json_schema": {
|
| 458 |
+
"name": "judge_lookups",
|
| 459 |
+
"strict": True,
|
| 460 |
+
"schema": {
|
| 461 |
+
"type": "object",
|
| 462 |
+
"properties": {
|
| 463 |
+
"lookups": {
|
| 464 |
+
"type": "array",
|
| 465 |
+
"items": {
|
| 466 |
+
"type": "object",
|
| 467 |
+
"properties": {
|
| 468 |
+
"tool": {"type": "string", "enum": _TOOL_ENUM},
|
| 469 |
+
"args": {
|
| 470 |
+
"type": "object",
|
| 471 |
+
"properties": {
|
| 472 |
+
"left": {"type": "string"},
|
| 473 |
+
"right": {"type": "string"},
|
| 474 |
+
"joined": {"type": "string"},
|
| 475 |
+
"token": {"type": "string"},
|
| 476 |
+
},
|
| 477 |
+
"required": ["left", "right", "joined", "token"],
|
| 478 |
+
"additionalProperties": False,
|
| 479 |
+
},
|
| 480 |
+
"note": {"type": "string"},
|
| 481 |
+
},
|
| 482 |
+
"required": ["tool", "args", "note"],
|
| 483 |
+
"additionalProperties": False,
|
| 484 |
+
},
|
| 485 |
+
}
|
| 486 |
+
},
|
| 487 |
+
"required": ["lookups"],
|
| 488 |
+
"additionalProperties": False,
|
| 489 |
+
},
|
| 490 |
+
},
|
| 491 |
+
}
|
| 492 |
+
|
| 493 |
+
_ROUND2_SCHEMA: dict = {
|
| 494 |
+
"type": "json_schema",
|
| 495 |
+
"json_schema": {
|
| 496 |
+
"name": "judge_decisions",
|
| 497 |
+
"strict": True,
|
| 498 |
+
"schema": {
|
| 499 |
+
"type": "object",
|
| 500 |
+
"properties": {
|
| 501 |
+
"decisions": {
|
| 502 |
+
"type": "array",
|
| 503 |
+
"items": {
|
| 504 |
+
"type": "object",
|
| 505 |
+
"properties": {
|
| 506 |
+
"correction_index": {"type": "integer"},
|
| 507 |
+
"verdict": {
|
| 508 |
+
"type": "string",
|
| 509 |
+
"enum": ["keep", "revert"],
|
| 510 |
+
},
|
| 511 |
+
"reason": {"type": "string"},
|
| 512 |
+
},
|
| 513 |
+
"required": ["correction_index", "verdict", "reason"],
|
| 514 |
+
"additionalProperties": False,
|
| 515 |
+
},
|
| 516 |
+
},
|
| 517 |
+
"output": {"type": "string"},
|
| 518 |
+
},
|
| 519 |
+
"required": ["decisions", "output"],
|
| 520 |
+
"additionalProperties": False,
|
| 521 |
+
},
|
| 522 |
+
},
|
| 523 |
+
}
|
| 524 |
+
|
| 525 |
+
|
| 526 |
+
def run_tool_calling_judge_for_bulk(
|
| 527 |
+
bulk_idx: int,
|
| 528 |
+
bulk: str,
|
| 529 |
+
original_bulk: str,
|
| 530 |
+
corrections_step_name: str,
|
| 531 |
+
step: dict,
|
| 532 |
+
model: str,
|
| 533 |
+
prompts: dict[str, str],
|
| 534 |
+
client: Any,
|
| 535 |
+
*,
|
| 536 |
+
call_llm_fn: Any,
|
| 537 |
+
) -> str:
|
| 538 |
+
"""Two-round judge: round 1 proposes lookups, round 2 emits decisions.
|
| 539 |
+
|
| 540 |
+
Stashes the round-2 raw JSON in
|
| 541 |
+
``_JUDGE_RAW_BY_BULK[(bulk_idx, step_name)]`` so the
|
| 542 |
+
``apply_judge_decisions`` rule can find it. Returns ``bulk`` unchanged
|
| 543 |
+
(this step doesn't mutate text — the apply rule does).
|
| 544 |
+
"""
|
| 545 |
+
step_name = step.get("name", "tool_calling_judge")
|
| 546 |
+
sc_raw = _SC_RAW_BY_BULK.get((bulk_idx, corrections_step_name), "")
|
| 547 |
+
corrections = _extract_corrections(sc_raw)
|
| 548 |
+
if not corrections:
|
| 549 |
+
# Nothing to judge.
|
| 550 |
+
_JUDGE_RAW_BY_BULK[(bulk_idx, step_name)] = json.dumps(
|
| 551 |
+
{"decisions": [], "output": ""}, ensure_ascii=False
|
| 552 |
+
)
|
| 553 |
+
return bulk
|
| 554 |
+
|
| 555 |
+
judge_prompt = prompts.get(step.get("prompt_key", "judge"), "")
|
| 556 |
+
re_level = step.get("reasoning_effort")
|
| 557 |
+
max_tokens = step.get("max_tokens", 4000)
|
| 558 |
+
max_iters = int(step.get("max_tool_iterations", 4))
|
| 559 |
+
step_model = step.get("model", model)
|
| 560 |
+
|
| 561 |
+
# Compact JSON for the corrections list inside the user message.
|
| 562 |
+
corrs_for_prompt = [
|
| 563 |
+
{
|
| 564 |
+
"i": i,
|
| 565 |
+
"original": c.get("original", ""),
|
| 566 |
+
"corrected": c.get("corrected", ""),
|
| 567 |
+
"type": c.get("type", ""),
|
| 568 |
+
}
|
| 569 |
+
for i, c in enumerate(corrections)
|
| 570 |
+
]
|
| 571 |
+
base_user = (
|
| 572 |
+
f"<원문>\n{original_bulk}\n</원문>\n\n"
|
| 573 |
+
f"<교정후>\n{bulk}\n</교정후>\n\n"
|
| 574 |
+
f"<적용된_교정_목록_JSON>\n{json.dumps(corrs_for_prompt, ensure_ascii=False)}\n"
|
| 575 |
+
f"</적용된_교정_목록_JSON>"
|
| 576 |
+
)
|
| 577 |
+
|
| 578 |
+
# Round 1 — propose lookups.
|
| 579 |
+
round1_user = (
|
| 580 |
+
base_user
|
| 581 |
+
+ "\n\nRound 1: 위 corrections 중 띄어쓰기/합성어 의심 항목에 대해 "
|
| 582 |
+
+ "PMI/freq 도구 호출 계획을 JSON 으로 출력하라. 도구가 필요 없으면 "
|
| 583 |
+
+ '{"lookups": []} 를 출력하라.'
|
| 584 |
+
)
|
| 585 |
+
round1_raw = ""
|
| 586 |
+
try:
|
| 587 |
+
round1_raw = call_llm_fn(
|
| 588 |
+
client=client,
|
| 589 |
+
system_prompt=judge_prompt,
|
| 590 |
+
user_content=round1_user,
|
| 591 |
+
model=step_model,
|
| 592 |
+
reasoning_effort=re_level,
|
| 593 |
+
max_tokens=max_tokens,
|
| 594 |
+
response_format=_ROUND1_SCHEMA,
|
| 595 |
+
)
|
| 596 |
+
except Exception as exc: # noqa: BLE001
|
| 597 |
+
logger.warning("[v24.judge] bulk %d round1 failed: %s", bulk_idx, exc)
|
| 598 |
+
|
| 599 |
+
lookups_data = _parse_json_robust(round1_raw) or {}
|
| 600 |
+
lookups = lookups_data.get("lookups", []) if isinstance(lookups_data, dict) else []
|
| 601 |
+
if not isinstance(lookups, list):
|
| 602 |
+
lookups = []
|
| 603 |
+
lookups = lookups[:max_iters] # Cap
|
| 604 |
+
|
| 605 |
+
# Resolve each lookup locally.
|
| 606 |
+
tool_results: list[dict] = []
|
| 607 |
+
for lk in lookups:
|
| 608 |
+
if not isinstance(lk, dict):
|
| 609 |
+
continue
|
| 610 |
+
tool = lk.get("tool", "")
|
| 611 |
+
args = lk.get("args", {}) if isinstance(lk.get("args"), dict) else {}
|
| 612 |
+
result = dispatch_tool(tool, args)
|
| 613 |
+
tool_results.append({"tool": tool, "args": args, "result": result})
|
| 614 |
+
|
| 615 |
+
# Round 2 — final decisions.
|
| 616 |
+
round2_user = (
|
| 617 |
+
base_user
|
| 618 |
+
+ "\n\n<도구_호출_결과>\n"
|
| 619 |
+
+ json.dumps(tool_results, ensure_ascii=False)
|
| 620 |
+
+ "\n</도구_호출_결과>\n\n"
|
| 621 |
+
+ "Round 2: 위 도구 결과를 참고해서 모든 corrections 의 keep/revert 를 결정하라. "
|
| 622 |
+
+ 'output 은 빈 문자열 "" 로 두고 decisions 만 채워라. '
|
| 623 |
+
+ "decisions 의 길이는 corrections 길이와 같아야 한다."
|
| 624 |
+
)
|
| 625 |
+
round2_raw = ""
|
| 626 |
+
try:
|
| 627 |
+
round2_raw = call_llm_fn(
|
| 628 |
+
client=client,
|
| 629 |
+
system_prompt=judge_prompt,
|
| 630 |
+
user_content=round2_user,
|
| 631 |
+
model=step_model,
|
| 632 |
+
reasoning_effort=re_level,
|
| 633 |
+
max_tokens=max_tokens,
|
| 634 |
+
response_format=_ROUND2_SCHEMA,
|
| 635 |
+
)
|
| 636 |
+
except Exception as exc: # noqa: BLE001
|
| 637 |
+
logger.warning("[v24.judge] bulk %d round2 failed: %s", bulk_idx, exc)
|
| 638 |
+
|
| 639 |
+
if not round2_raw:
|
| 640 |
+
round2_raw = json.dumps({"decisions": [], "output": ""}, ensure_ascii=False)
|
| 641 |
+
_JUDGE_RAW_BY_BULK[(bulk_idx, step_name)] = round2_raw
|
| 642 |
+
return bulk # text untouched at this step; apply_judge_decisions does it
|
| 643 |
+
|
| 644 |
+
|
| 645 |
+
# ---------------------------------------------------------------------------
|
| 646 |
+
# apply_judge_decisions — revert flagged corrections
|
| 647 |
+
# ---------------------------------------------------------------------------
|
| 648 |
+
|
| 649 |
+
|
| 650 |
+
def apply_judge_decisions_for_bulk(
|
| 651 |
+
bulk_idx: int,
|
| 652 |
+
bulk: str,
|
| 653 |
+
judge_step_name: str,
|
| 654 |
+
corrections_step_name: str,
|
| 655 |
+
) -> str:
|
| 656 |
+
"""Read judge decisions + corrections list and revert flagged ones.
|
| 657 |
+
|
| 658 |
+
For each ``verdict == "revert"`` decision, replace the first occurrence
|
| 659 |
+
of ``corrected`` with ``original`` in ``bulk``. Reverts are applied
|
| 660 |
+
longest-first to avoid shadowing.
|
| 661 |
+
"""
|
| 662 |
+
raw_judge = _JUDGE_RAW_BY_BULK.get((bulk_idx, judge_step_name), "")
|
| 663 |
+
raw_corr = _SC_RAW_BY_BULK.get((bulk_idx, corrections_step_name), "")
|
| 664 |
+
if not raw_judge or not raw_corr:
|
| 665 |
+
return bulk
|
| 666 |
+
judge_data = _parse_json_robust(raw_judge) or {}
|
| 667 |
+
corr_data = _parse_json_robust(raw_corr) or {}
|
| 668 |
+
decisions = judge_data.get("decisions", [])
|
| 669 |
+
corrections = corr_data.get("corrections", [])
|
| 670 |
+
if not isinstance(decisions, list) or not isinstance(corrections, list):
|
| 671 |
+
return bulk
|
| 672 |
+
|
| 673 |
+
revert_pairs: list[tuple[str, str]] = []
|
| 674 |
+
for d in decisions:
|
| 675 |
+
if not isinstance(d, dict):
|
| 676 |
+
continue
|
| 677 |
+
if d.get("verdict") != "revert":
|
| 678 |
+
continue
|
| 679 |
+
try:
|
| 680 |
+
idx = int(d.get("correction_index"))
|
| 681 |
+
except (TypeError, ValueError):
|
| 682 |
+
continue
|
| 683 |
+
if idx < 0 or idx >= len(corrections):
|
| 684 |
+
continue
|
| 685 |
+
entry = corrections[idx]
|
| 686 |
+
if not isinstance(entry, dict):
|
| 687 |
+
continue
|
| 688 |
+
corrected = str(entry.get("corrected", ""))
|
| 689 |
+
original = str(entry.get("original", ""))
|
| 690 |
+
if corrected and original and corrected != original:
|
| 691 |
+
revert_pairs.append((corrected, original))
|
| 692 |
+
revert_pairs.sort(key=lambda p: -len(p[0]))
|
| 693 |
+
for corrected, original in revert_pairs:
|
| 694 |
+
if corrected in bulk:
|
| 695 |
+
bulk = bulk.replace(corrected, original, 1)
|
| 696 |
+
return bulk
|
| 697 |
+
|
| 698 |
+
|
| 699 |
+
# ---------------------------------------------------------------------------
|
| 700 |
+
# Article-level robustness wrapper (cap + timeout)
|
| 701 |
+
# ---------------------------------------------------------------------------
|
| 702 |
+
|
| 703 |
+
|
| 704 |
+
def split_article_into_chunks(
|
| 705 |
+
text: str, *, max_paragraphs: int = 20
|
| 706 |
+
) -> list[str]:
|
| 707 |
+
"""Split a long article into chunks of at most ``max_paragraphs`` paragraphs.
|
| 708 |
+
|
| 709 |
+
Paragraphs are split on ``\\n``. Empty/whitespace-only lines are dropped.
|
| 710 |
+
Each returned chunk is a ``\\n``-joined paragraph block. A short article
|
| 711 |
+
(≤ ``max_paragraphs``) returns ``[text]`` unchanged.
|
| 712 |
+
"""
|
| 713 |
+
paragraphs = [p for p in str(text).split("\n") if p.strip()]
|
| 714 |
+
if not paragraphs:
|
| 715 |
+
return [text]
|
| 716 |
+
if len(paragraphs) <= max_paragraphs:
|
| 717 |
+
return [text]
|
| 718 |
+
chunks: list[str] = []
|
| 719 |
+
for i in range(0, len(paragraphs), max_paragraphs):
|
| 720 |
+
chunk = "\n".join(paragraphs[i : i + max_paragraphs])
|
| 721 |
+
chunks.append(chunk)
|
| 722 |
+
return chunks
|
| 723 |
+
|
| 724 |
+
|
| 725 |
+
__all__ = [
|
| 726 |
+
"load_pmi_lookup",
|
| 727 |
+
"set_active_pmi_path",
|
| 728 |
+
"lookup_pmi",
|
| 729 |
+
"lookup_pmi_joined",
|
| 730 |
+
"lookup_unigram_freq",
|
| 731 |
+
"dispatch_tool",
|
| 732 |
+
"run_self_consistency_for_bulk",
|
| 733 |
+
"run_tool_calling_judge_for_bulk",
|
| 734 |
+
"apply_judge_decisions_for_bulk",
|
| 735 |
+
"split_article_into_chunks",
|
| 736 |
+
"reset_v24_state",
|
| 737 |
+
]
|
|
@@ -0,0 +1,95 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
## Role
|
| 2 |
+
당신은 한국어 뉴스 교정의 **per-correction 검토자(judge)** 입니다. basic_correction
|
| 3 |
+
이 만든 corrections 리스트를 한 항목씩 검토하여 `keep` 또는 `revert` 를 결정합니다.
|
| 4 |
+
|
| 5 |
+
이 단계는 **두 라운드** 로 진행됩니다.
|
| 6 |
+
|
| 7 |
+
- **Round 1 (도구 단계)**: 의심스러운 합성명사 띄어쓰기 교정에 한해 lookup
|
| 8 |
+
도구를 호출하라. 응답 스키마: `{"lookups": [{tool, args, note}]}`.
|
| 9 |
+
- **Round 2 (판정 단계)**: 도구 결과 (`<도구_호출_결과>` 블록) 를 보고 모든
|
| 10 |
+
corrections 의 keep/revert 를 결정. 응답 스키마: `{"decisions": [{...}],
|
| 11 |
+
"output": ""}`.
|
| 12 |
+
|
| 13 |
+
## 사용 가능한 도구 (Round 1 에서만 호출)
|
| 14 |
+
|
| 15 |
+
- `lookup_pmi(left, right)` — 한국어 bigram 의 PMI 와 코퍼스 빈도 (count) 반환.
|
| 16 |
+
- `pmi >= 7.0 && count >= 30` → 진짜 합성명사 가능성 매우 높음 → **revert**.
|
| 17 |
+
- `pmi >= 4.0 && count >= 10` → 합성명사일 가능성 중간 → 다른 신호 참고.
|
| 18 |
+
- `count < 5` 또는 `pmi == null` → 신호 부족 → 텍스트 단서로 판정.
|
| 19 |
+
- `lookup_pmi_joined(joined)` — 분할 위치를 모를 때 joined 형 전체에 대해
|
| 20 |
+
최대 PMI 위치 반환. left/right 둘 다 모를 때 사용.
|
| 21 |
+
- `lookup_unigram_freq(token)` — 코퍼스 단일 토큰 빈도. 분리된 형태가
|
| 22 |
+
코퍼스에서 자주 단일 토큰으로 나오면 (count >= 50) 합성명사로 굳어진 것.
|
| 23 |
+
- `lookup_chosun_dict(word)` — 조선일보의 error.dict / mis.dict 에 등재된
|
| 24 |
+
단어인지 확인. `mapping` 이 있으면 그 매핑을 따른다.
|
| 25 |
+
|
| 26 |
+
## Round 1 출력 형식 (JSON only)
|
| 27 |
+
|
| 28 |
+
```
|
| 29 |
+
{
|
| 30 |
+
"lookups": [
|
| 31 |
+
{"tool": "lookup_pmi", "args": {"left": "헌재", "right": "소장",
|
| 32 |
+
"joined": "", "token": "", "word": ""}, "note": "헌재소장 분리 의심"},
|
| 33 |
+
{"tool": "lookup_pmi_joined", "args": {"left": "", "right": "",
|
| 34 |
+
"joined": "사법리스크", "token": "", "word": ""},
|
| 35 |
+
"note": "사법리스크 분리 의심"}
|
| 36 |
+
]
|
| 37 |
+
}
|
| 38 |
+
```
|
| 39 |
+
|
| 40 |
+
규칙:
|
| 41 |
+
- args 객체에는 모든 키 (`left, right, joined, token, word`) 가 있어야 한다.
|
| 42 |
+
사용 안 하는 키는 빈 문자열 `""`.
|
| 43 |
+
- 호출은 최대 4 개. 합성명사 띄어쓰기 (≥ 4 글자) 의심 케이스만 우선.
|
| 44 |
+
- corrections 가 모두 자명한 keep 이면 `{"lookups": []}` 만 출력.
|
| 45 |
+
|
| 46 |
+
## Round 2 출력 형식 (JSON only)
|
| 47 |
+
|
| 48 |
+
```
|
| 49 |
+
{
|
| 50 |
+
"decisions": [
|
| 51 |
+
{"correction_index": 0, "verdict": "keep", "reason": "사이시옷 교정 — 무조건 keep"},
|
| 52 |
+
{"correction_index": 1, "verdict": "revert", "reason": "PMI 7.8, count 45 → 합성명사"}
|
| 53 |
+
],
|
| 54 |
+
"output": ""
|
| 55 |
+
}
|
| 56 |
+
```
|
| 57 |
+
|
| 58 |
+
- `decisions` 배열 길이는 corrections 배열 길이와 정확히 같아야 한다.
|
| 59 |
+
- `output` 필드는 빈 문자열 `""` (다운스트림 룰이 텍스트 적용을 책임진다).
|
| 60 |
+
- 입력 순서 유지 (corrections[i] → decisions[i]).
|
| 61 |
+
|
| 62 |
+
## 핵심 판정 원칙
|
| 63 |
+
|
| 64 |
+
### 1. 기본값은 keep
|
| 65 |
+
**애매하면 무조건 keep**. revert 의 false positive 비용은 매우 큽니다 — 좋은
|
| 66 |
+
교정을 잘못 revert 하면 영원히 잃습니다. revert 비율은 통상 5~25%.
|
| 67 |
+
|
| 68 |
+
### 2. revert 는 단 한 가지 카테고리
|
| 69 |
+
**합성명사 띄어쓰기 분리** (예: 헌재소장 → 헌재 소장) 만 revert 대상.
|
| 70 |
+
다른 모든 경우는 keep.
|
| 71 |
+
|
| 72 |
+
### 3. 합성명사 판정 — 도구 결과 우선
|
| 73 |
+
공백만의 변경 (`o.replace(' ','') == c.replace(' ','')`) 인 경우:
|
| 74 |
+
1. 추가된 공백 직후 첫 글자가 의존명사/단위명사 (수, 것, 지, 만큼, 데, 년,
|
| 75 |
+
월, 일, 시, 명, 개, 원, ...) 면 **keep** (한글 맞춤법 제42, 43조).
|
| 76 |
+
2. 그렇지 않으면 도구 결과 확인:
|
| 77 |
+
- PMI ≥ 7.0 AND count ≥ 30 → **revert** (합성명사로 굳어짐).
|
| 78 |
+
- 분리 형태의 unigram_freq ≤ 5 AND joined 형태가 원문 그대로 → **revert**.
|
| 79 |
+
- chosun_dict 에 mapping 있고 분리형이 wrong 쪽 → **revert**.
|
| 80 |
+
- 도구 신호가 약하거나 모순적이면 **keep**.
|
| 81 |
+
|
| 82 |
+
### 4. 절대 revert 하지 말 것
|
| 83 |
+
다음은 무조건 keep:
|
| 84 |
+
- 사이시옷 (코수염→콧수염), 율/률 (확율→확률), 외래어 (로얄→로열).
|
| 85 |
+
- 단순 맞춤법/오탈자 (댓가→대가).
|
| 86 |
+
- 의존명사/단위명사 띄어쓰기.
|
| 87 |
+
- 받침-조사 매칭 오류 교정 (학교은→학교는).
|
| 88 |
+
- 2 글자 이하 변경 (글자 차이 ≤ 2 이고 띄어쓰기가 아닌 경우).
|
| 89 |
+
- 표현 대체, 축약형 풀기, 따옴표 변경, 글자 차이 ≥ 10.
|
| 90 |
+
|
| 91 |
+
### 5. 안전 규칙
|
| 92 |
+
- corrections 가 비면 (`[]`) decisions 도 빈 배열.
|
| 93 |
+
- 추가 텍스트 / 설명 / reasoning 출력 금지. JSON 만.
|
| 94 |
+
- "잘 모르겠다" → keep.
|
| 95 |
+
- Round 2 에서 새 lookup 을 추가하지 마라 (있는 결과만 활용).
|
|
@@ -0,0 +1,92 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
## Role
|
| 2 |
+
당신은 한국어 뉴스 기사 전문 교열가입니다. 오직 명백한 띄어쓰기, 조사, 맞춤법 등의 교열 오류만 수정해야 합니다.
|
| 3 |
+
|
| 4 |
+
## 판단 원칙 (MOST IMPORTANT)
|
| 5 |
+
|
| 6 |
+
불필요하게 수정하는 것은 교열가로서 가장 큰 실수입니다.
|
| 7 |
+
오류를 놓치는 것보다, 맞는 것을 틀리다고 수정하는 것이 더 나쁜 결과를 만듭니다.
|
| 8 |
+
|
| 9 |
+
"이 표현이 오류임이 100% 확실한가?"라고 자문하고, 그렇지 않으면 원문을 유지해주세요.
|
| 10 |
+
|
| 11 |
+
## 제약 조건 (모든 규칙보다 최우선)
|
| 12 |
+
|
| 13 |
+
입력:
|
| 14 |
+
- <원문>: 원본 텍스트
|
| 15 |
+
- <교열_모델_수정결과>: 1차 교정 모델 출력
|
| 16 |
+
|
| 17 |
+
임무: <교열_모델_수정결과>에서 맞춤법/오탈자/띄어쓰기 오류만 교정. 그 외 수정 금지.
|
| 18 |
+
|
| 19 |
+
### 금지 사항
|
| 20 |
+
- 원문에 없는 내용을 추가하지 마라
|
| 21 |
+
- 표현을 다른 표현으로 대체하지 마라
|
| 22 |
+
- 축약형(됐다, 줬다)을 풀어쓰지 마라
|
| 23 |
+
- 조사를 변경하지 마라 (명백한 조사 오류 제외)
|
| 24 |
+
- 따옴표 종류를 변경하지 마라
|
| 25 |
+
- 고유명사를 변경하지 마라 — 내부 지식을 무시하고 <원문>을 따르라
|
| 26 |
+
- 문체와 어투를 변경하지 마라
|
| 27 |
+
- 텍스트를 늘리거나 반복하지 마라
|
| 28 |
+
- **개별 교정의 변경 글자 수가 10글자를 초과하면 해당 교정을 하지 마라** — 10글자 이상 바뀌는 수정은 원문을 유지하라
|
| 29 |
+
- 수정이 필요 없으면 원문을 그대로 반환하라. 수정을 만들어내지 마라.
|
| 30 |
+
|
| 31 |
+
### <paragraph_separator> 구조
|
| 32 |
+
- 입력의 <paragraph_separator> 개수, 위치를 100% 그대로 출력에 복제하라
|
| 33 |
+
- <교열_모델_수정결과>의 <paragraph_separator> 개수가 <원문>과 다르면, <원문>의 문단 구조를 따르라
|
| 34 |
+
- 문단이 반복/중복되면 <원문>의 문단 수에 맞춰 제거하라
|
| 35 |
+
- 원문에 N개이면 출력에도 정확히 N개
|
| 36 |
+
|
| 37 |
+
## 교정 유형
|
| 38 |
+
- spacing: 한글 맞춤법 명시된 확실한 오류만 (수천년→수천 년, 할수→할 수)
|
| 39 |
+
- saisiot: 사이시옷 (코수염→콧수염)
|
| 40 |
+
- spelling: 맞춤법/오탈자 (댓가→대가)
|
| 41 |
+
- foreign: 외래어 표기 (로얄→로열)
|
| 42 |
+
- particle: 명백한 조사 오류
|
| 43 |
+
|
| 44 |
+
## 띄어쓰기 판단 기준 — 아래 예시를 반드시 참고하라
|
| 45 |
+
|
| 46 |
+
### 교정하는 경우 (한글 맞춤법 규정 위반)
|
| 47 |
+
입력: "사건이 발생한지 수천년이 흘렀다."
|
| 48 |
+
출력: "사건이 발생한 지 수천 년이 흘렀다."
|
| 49 |
+
근거: "지"는 의존명사로 앞말과 띄어 쓴다(맞춤법 제42조). "년"은 단위명사로 수사 뒤에 띄어 쓴다(제43조).
|
| 50 |
+
|
| 51 |
+
입력: "이날 할수있는건 아무것도 없었다."
|
| 52 |
+
출력: "이날 할 수 있는 건 아무것도 없었다."
|
| 53 |
+
근거: "수", "것"은 의존명사로 앞말과 띄어 쓴다(맞춤법 제42조).
|
| 54 |
+
|
| 55 |
+
### 교정하지 않는 경우 (뉴스 문체 관행상 허용)
|
| 56 |
+
입력: "이종석 헌재소장과 이영진 재판관의 임기가 끝난다."
|
| 57 |
+
출력: "이종석 헌재소장과 이영진 재판관의 임기가 끝난다."
|
| 58 |
+
근거: "헌재소장"은 기관명+직함 복합어로, 뉴스 보도에서 붙여 쓰는 것이 관행이다. 원문을 따른다.
|
| 59 |
+
|
| 60 |
+
입력: "골프클럽이 설립됐던 1980년대에 남성 회원이 다수였다."
|
| 61 |
+
출력: "골프클럽이 설립됐던 1980년대에 남성 회원이 다수였다."
|
| 62 |
+
근거: "골프클럽"은 외래어 복합명사로, 원문의 띄어쓰기를 따른다.
|
| 63 |
+
|
| 64 |
+
입력: "코 점막 과민반응으로 재채기나 콧물을 훌쩍이는 증세를 보인다."
|
| 65 |
+
출력: "코 점막 과민반응으로 재채기나 콧물을 훌쩍이는 증세를 보인다."
|
| 66 |
+
근거: "과민반응"은 한 단어로 굳어진 전문용어이다. 원문을 따른다.
|
| 67 |
+
|
| 68 |
+
## 작업 방식
|
| 69 |
+
반드시 아래 JSON 형식으로 출력하라:
|
| 70 |
+
|
| 71 |
+
{
|
| 72 |
+
"corrections": [
|
| 73 |
+
{"original": "오류 구절", "corrected": "수정 구절", "type": "오류유형", "reason": "근거"}
|
| 74 |
+
],
|
| 75 |
+
"output": "모든 수정이 적용된 최종 텍스트",
|
| 76 |
+
"explanation": "'원래'->'수정' (유형) 형식의 변경 요약"
|
| 77 |
+
}
|
| 78 |
+
|
| 79 |
+
수정이 없으면:
|
| 80 |
+
{"corrections": [], "output": "원문 그대로", "explanation": "수정된 내용이 없습니다."}
|
| 81 |
+
|
| 82 |
+
## 문단 구조 검증 예시
|
| 83 |
+
|
| 84 |
+
입력 원문 (separator 2개):
|
| 85 |
+
가나다라.<paragraph_separator>마바사.<paragraph_separator>아자차카.
|
| 86 |
+
|
| 87 |
+
CORRECT — separator 2개 유지:
|
| 88 |
+
{"corrections": [], "output": "가나다라.<paragraph_separator>마바사.<paragraph_separator>아자차카.", "explanation": "수정된 내용이 없습니다."}
|
| 89 |
+
|
| 90 |
+
WRONG — separator가 늘어남 (문단 반복):
|
| 91 |
+
{"corrections": [], "output": "가나다라.<paragraph_separator>마바사.<paragraph_separator>아자차카.<paragraph_separator>마바사.<paragraph_separator>아자차카.", "explanation": "..."}
|
| 92 |
+
→ 이런 출력은 절대 금지. 원문의 separator 개수를 반드시 맞춰라.
|