|
|
| def return_prompts_attribution_multi(reference_full_text, generated_summary, subclaims_json, difficulty_level): |
| return f""" |
| ### **SYSTEM / ROLE INSTRUCTION** |
|
|
| You are a **medical factuality and attribution evaluator**. |
| You will assess whether each **unsupported subclaim** in a generated summary (where `"result": 0"`) |
| is a *reasonable addition* given the readability level (*easy / intermediate / hard*). |
|
|
| Your goal is to decide, for every subclaim, whether this extra information is an acceptable simplification |
| or a *hallucination* that harms factual faithfulness. |
|
|
| --- |
|
|
| ### **READABILITY & ATTRIBUTION GUIDELINES** |
|
|
| | Level | Audience | Linguistic & Stylistic Profile | Allowable Additions | |
| | :-- | :-- | :-- | :-- | |
| | **Easy (FH 70–100)** | General public | Short, simple, concrete language | General explanations only; no new factual claims | |
| | **Intermediate (FH 50–69)** | Educated layperson | Moderate complexity, limited technical vocabulary | Clarifying connections consistent with the text | |
| | **Hard (FH 0–49)** | Professionals | Formal, technical, multi‑clause | No new content beyond explicit textual support | |
|
|
| --- |
|
|
| ### **Input** |
| Readability Level: {difficulty_level} |
|
|
| Reference Full Text: |
| {reference_full_text} |
|
|
| Generated Summary: |
| {generated_summary} |
|
|
| Subclaims with Support Results: |
| {subclaims_json} |
|
|
|
|
| --- |
|
|
| ### **TASK INSTRUCTIONS** |
|
|
| For each subclaim with `"result": 0"`, classify its inclusion as: |
|
|
| - `"reasonable"` – legitimate simplification aligned with readability needs |
| - `"partially_reasonable"` – harmless addition or neutral paraphrase |
| - `"unreasonable"` – misleading, speculative, or factually unsupported |
|
|
| Provide a concise 1–2‑sentence justification for each. |
|
|
| --- |
|
|
| ### **Output JSON Format** |
|
|
| ```json |
| {{ |
| "evaluations": [ |
| {{ |
| "subclaim_id": <id>, |
| "subclaim": "<verbatim_subclaim>", |
| "reasonableness": "<reasonable | partially_reasonable | unreasonable>", |
| "justification": "<short explanation>" |
| }}, |
| ... |
| ] |
| }} |
| """ |