whatswrong/33-script-fallback-empty-articles.md

33 — Script Fallback Empty Articles

Summary

Both filesystem and DuckDB script stage runners catch missing or invalid extract artifacts and silently set articles = [], then proceed to generate_script with success reporting (file path) or partial failure (DB path only if zero lines). Missing extract stage is not surfaced as a stage error; the script falls back to story summaries only, degrading episode quality without operator visibility.

Should vs Actual

Should	Actual
Script stage fails fast if extract prerequisite missing	except → articles = [], continue
Clear warning in StageResult.error or items_in mismatch	File mode always success=True; DB mode succeeds if any lines generated
Operator knows extract was skipped	Only log inside generate_script content_map fallback to item.summary
Consistent exception handling	File: FileNotFoundError, json.JSONDecodeError; DB: bare Exception

Evidence

Filesystem script stage:

    async def _stage_script_file(self, reg: PipelineRegister, date: str) -> StageResult:
        ...
        try:
            articles = [Article(**a) for a in reg.read_jsonl(3, "articles.jsonl")]
        except (FileNotFoundError, json.JSONDecodeError):
            articles = []

        ...
        script = await generate_script(
            ranked, articles, self.config, date, jsonl_path=jsonl_path
        )

        return StageResult(
            stage=4,
            name="script",
            success=True,
            ...
        )

DuckDB script stage — broader catch:

    async def _stage_script_db(self, run_id: str, date: str) -> StageResult:
        ...
        try:
            articles = [Article(**a) for a in storage.read_stage(run_id, 3)]
        except (FileNotFoundError, Exception):
            articles = []

        script = await generate_script(ranked, articles, self.config, date)
        if not script.lines:
            return StageResult(..., success=False, error="script generation produced no lines")

generate_script silently uses summaries when article body missing:

    article_map = {a.url: a for a in (articles or [])}
    ...
    content_map = {
        item.title: (
            article_map[item.url].body
            if item.url in article_map and article_map[item.url].body
            else item.summary
        )
        for item in items
    }

Empty articles means all content comes from item.summary — no log at warning level for that fallback at pipeline level.

Impact

• Silent quality degradation: Skipping Extract in Individual Stages tab still produces a "successful" script with shallow content.
• Debugging pain: Users blame LLM/script stage when root cause is missing stage 3 artifacts.
• Inconsistent strictness: DB path fails on zero lines; file path reports success even for empty dialogue edge cases handled inside generate_script.
• Chaining bugs amplified: Combined with run_id heuristic (item 30), script may read wrong/missing extract data without error.

Suggested fix

1. Replace silent fallback with explicit policy flag: require_extract: bool (default True in production, False in dev).
2. When extract data missing: StageResult(success=False, error="extract stage output missing for run ...") unless flag set.
3. If soft fallback desired: success=True with error="warning: using summaries only (N articles missing)" and log items_in from extract vs rank counts.
4. Narrow DB except to FileNotFoundError only (what read_stage raises).
5. Add pipeline validation helper: assert_prerequisites(run_id, stage) before each stage in advanced UI.
6. Surface extract coverage metric in stage log: items_out articles / items_in ranked.

Related issues

• 30-single-stage-run-id-heuristic — wrong run → missing extract appears as empty articles
• 32-publish-split — downstream stages may succeed on thin scripts
• 26-app-py-god-module — Individual Stages tab enables running script without extract