pebaryan Claude Sonnet 4.6 commited on
Commit
d71fbfa
·
1 Parent(s): 06c8e4f

Add deterministic citation checker

Browse files

New checker cross-references .cite keys against parsed .bib entries:
- MISSING: cited key not in any .bib file (error)
- UNUSED: bib entry never cited (info)
- DUPLICATE: same key defined by multiple entries (warning)

Architectural changes to support it:
- Checker protocol now takes a Document (path + source + paragraphs)
instead of just paragraphs, so citation-style checkers can scan the
raw source and .bib discovery can use the document's directory.
- Finding gains an optional file field so bib-level findings can point
at a .bib entry rather than the .tex.
- Prose filtering moved from the CLI into GrammarChecker; citation
scanning needs headings too.
- CLI exit code is 1 only for error-severity findings, so unused-bib
info/warning noise in drafts doesn't fail CI.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

CLAUDE.md CHANGED
@@ -22,17 +22,17 @@ The `qalmsw check` command reads `QALMSW_BASE_URL` (default `http://localhost:80
22
 
23
  ## Architecture
24
 
25
- The pipeline is **parse → checkers → report**, and every checker produces a uniform `Finding` so the report/CI layer never branches on checker type.
26
 
27
  ```
28
- .tex file
29
 
30
 
31
- parse.parse_paragraphs # blank-line-split, comments stripped,
32
- list[Paragraph] # source line numbers preserved
33
 
34
  checkers.* # each implements the Checker protocol:
35
- │ list[Finding] # check(paragraphs) -> list[Finding]
36
 
37
  report.render_findings # rich-formatted terminal output
38
  ```
@@ -41,24 +41,35 @@ report.render_findings # rich-formatted terminal output
41
 
42
  - **`Paragraph.start_line` / `end_line` are 1-indexed against the original source file** (not the stripped body). Findings point at these so editors can jump to the right line. Do not change this without updating every checker that constructs Findings.
43
  - **Comment stripping preserves newlines** so line numbers stay stable after `%`-stripping. `parse.tex._COMMENT_RE` uses a negative look-behind to skip escaped `\%`.
 
 
 
44
  - **`LLMClient` is a `typing.Protocol`**, not an ABC. Tests pass a `FakeLLM` that only implements `complete_json`. Don't tighten the interface into a base class; the Protocol is intentional so any object with the right shape works.
45
  - **Checkers never ask the LLM for line numbers.** Small local models count lines unreliably. Instead they ask for an `excerpt` string and we locate it in the paragraph text (`grammar._locate_line`). New checkers should follow this pattern.
46
  - **Structured output uses `response_format={"type": "json_object"}`** — supported by llama.cpp server. The system prompt must also spell out the JSON shape, because small models otherwise drift.
 
 
47
 
48
  ### Checker status
49
 
50
  | Checker | State | Shape |
51
  |------------|------------|-------------------------------------------------------------|
52
  | `grammar` | working | Per-paragraph LLM call, parallelizable, cheap |
53
- | `citations`| **planned**| Mostly deterministic `.bib` vs `\cite` cross-check, LLM only to verify a cited work actually supports the claim |
54
  | `claims` | **planned**| Needs retrieval (arXiv / local PDF cache / S2) + LLM judge — most expensive |
55
  | `reviewer` | **planned**| Whole-document pass, must chunk to fit local-model context |
56
 
57
  When adding a checker: drop a file into `src/qalmsw/checkers/`, register it in `checkers/__init__.py`, wire it into `cli.py`'s `checkers` list, and add tests with a `FakeLLM` — don't hit the real server from tests.
58
 
 
 
 
 
 
 
59
  ### What's intentionally *not* here
60
 
61
  - No multi-file `\input{}` / `\include{}` resolution yet — single-file only.
62
- - No `.bib` parser yet will live in `src/qalmsw/bib/` when the citation checker lands.
63
  - No retrieval layer yet — will live in `src/qalmsw/retrieval/` when the claims checker lands.
64
  - No SARIF/JSON report formats yet — only `report/text.py`. The `Finding` pydantic model is the serialization seam when those arrive.
 
22
 
23
  ## Architecture
24
 
25
+ The pipeline is **load → checkers → report**, and every checker produces a uniform `Finding` so the report/CI layer never branches on checker type.
26
 
27
  ```
28
+ .tex file (+ .bib)
29
 
30
 
31
+ Document.load # path + source + parse_paragraphs()
32
+ Document #
33
 
34
  checkers.* # each implements the Checker protocol:
35
+ │ list[Finding] # check(doc: Document) -> list[Finding]
36
 
37
  report.render_findings # rich-formatted terminal output
38
  ```
 
41
 
42
  - **`Paragraph.start_line` / `end_line` are 1-indexed against the original source file** (not the stripped body). Findings point at these so editors can jump to the right line. Do not change this without updating every checker that constructs Findings.
43
  - **Comment stripping preserves newlines** so line numbers stay stable after `%`-stripping. `parse.tex._COMMENT_RE` uses a negative look-behind to skip escaped `\%`.
44
+ - **Checkers receive a `Document`**, not just paragraphs. Citation-style checkers need the raw source (cites can appear outside prose paragraphs) and the path (to resolve `.bib` files). The `Checker` protocol is `check(doc: Document) -> list[Finding]`.
45
+ - **Non-prose paragraphs are filtered by the consumer**, not at load time. `GrammarChecker` calls `has_prose()` per paragraph so citation-scanning checkers still see headings. Don't pre-filter in the CLI.
46
+ - **CLI exit code is `1` only when an `error`-severity finding exists.** `info` (unused bib entry) and `warning` (duplicate bib key) findings are non-fatal by design — drafts routinely have them.
47
  - **`LLMClient` is a `typing.Protocol`**, not an ABC. Tests pass a `FakeLLM` that only implements `complete_json`. Don't tighten the interface into a base class; the Protocol is intentional so any object with the right shape works.
48
  - **Checkers never ask the LLM for line numbers.** Small local models count lines unreliably. Instead they ask for an `excerpt` string and we locate it in the paragraph text (`grammar._locate_line`). New checkers should follow this pattern.
49
  - **Structured output uses `response_format={"type": "json_object"}`** — supported by llama.cpp server. The system prompt must also spell out the JSON shape, because small models otherwise drift.
50
+ - **`LLMClient` has `max_retries=0`.** A slow local model that times out once rarely succeeds silently; failing fast surfaces the real state instead of a 30-minute silent retry loop.
51
+ - **`.bib` parsing is regex-based**, not a full BibTeX parser. We extract `@type{key,` headers + line numbers because that's all MISSING / UNUSED / DUPLICATE needs. If we ever need field values, swap in `bibtexparser`; don't grow the regex.
52
 
53
  ### Checker status
54
 
55
  | Checker | State | Shape |
56
  |------------|------------|-------------------------------------------------------------|
57
  | `grammar` | working | Per-paragraph LLM call, parallelizable, cheap |
58
+ | `citations`| working | Deterministic `.bib` vs `\cite` cross-check (MISSING / UNUSED / DUPLICATE). No LLM. |
59
  | `claims` | **planned**| Needs retrieval (arXiv / local PDF cache / S2) + LLM judge — most expensive |
60
  | `reviewer` | **planned**| Whole-document pass, must chunk to fit local-model context |
61
 
62
  When adding a checker: drop a file into `src/qalmsw/checkers/`, register it in `checkers/__init__.py`, wire it into `cli.py`'s `checkers` list, and add tests with a `FakeLLM` — don't hit the real server from tests.
63
 
64
+ ### Citations module layout
65
+
66
+ - `src/qalmsw/bib/parser.py` — regex extractor over `@type{key,` headers + source lines.
67
+ - `src/qalmsw/parse/citations.py` — scanners for `\cite*` keys and `\bibliography{}` / `\addbibresource{}` paths.
68
+ - `src/qalmsw/checkers/citations.py` — the `CitationChecker`; receives parsed `BibEntry`s via its constructor so the CLI owns `.bib` discovery and resolution.
69
+
70
  ### What's intentionally *not* here
71
 
72
  - No multi-file `\input{}` / `\include{}` resolution yet — single-file only.
73
+ - No LLM-assisted citation verification (does this citation actually support this claim?). That's the `claims` checker's territory.
74
  - No retrieval layer yet — will live in `src/qalmsw/retrieval/` when the claims checker lands.
75
  - No SARIF/JSON report formats yet — only `report/text.py`. The `Finding` pydantic model is the serialization seam when those arrive.
README.md CHANGED
@@ -4,7 +4,7 @@ Automated QA for scientific LaTeX writing, powered by a local LLM (llama.cpp ser
4
 
5
  ## Status
6
 
7
- Early scaffold. Grammar checker works end-to-end; citation, claim-to-reference, and reviewer-style checkers are planned.
8
 
9
  ## Quick start
10
 
@@ -14,7 +14,9 @@ pip install -e '.[dev]'
14
  # Start llama.cpp server separately, e.g.
15
  # ./llama-server -m model.gguf -c 8192 --port 8080
16
 
17
- qalmsw check path/to/paper.tex
 
 
18
  ```
19
 
20
  Environment variables:
@@ -24,7 +26,9 @@ Environment variables:
24
 
25
  ## Checkers
26
 
27
- - `grammar` — per-paragraph grammar/style pass
28
- - `citations` *(planned)* — `.bib` cross-check for missing/unused keys
29
  - `claims` *(planned)* — claim-to-reference consistency via retrieval
30
  - `reviewer` *(planned)* — whole-document reviewer-style critique
 
 
 
4
 
5
  ## Status
6
 
7
+ Grammar and citation checkers work end-to-end. Claim-to-reference and reviewer-style checkers are planned.
8
 
9
  ## Quick start
10
 
 
14
  # Start llama.cpp server separately, e.g.
15
  # ./llama-server -m model.gguf -c 8192 --port 8080
16
 
17
+ qalmsw check path/to/paper.tex # run all checkers
18
+ qalmsw check --skip-grammar path/to/paper.tex # deterministic citation checks only
19
+ qalmsw check --bib refs.bib path/to/paper.tex # override .bib auto-discovery
20
  ```
21
 
22
  Environment variables:
 
26
 
27
  ## Checkers
28
 
29
+ - `grammar` — per-paragraph grammar/style pass (LLM-backed)
30
+ - `citations` — `.bib` vs `\cite` cross-check: MISSING keys, UNUSED entries, DUPLICATE keys
31
  - `claims` *(planned)* — claim-to-reference consistency via retrieval
32
  - `reviewer` *(planned)* — whole-document reviewer-style critique
33
+
34
+ Exit code is `1` only when an `error`-severity finding is present (missing citation, grammar error), so drafts with unused-bib-entry `info`s or duplicate-key `warning`s don't fail CI.
examples/refs.bib ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ @inproceedings{vaswani2017,
2
+ title={Attention is All You Need},
3
+ author={Vaswani, Ashish and others},
4
+ booktitle={NeurIPS},
5
+ year={2017},
6
+ }
7
+
8
+ @article{orphan2019,
9
+ title={A paper nobody cites},
10
+ author={Ghost, A.},
11
+ journal={Journal of Nothing},
12
+ year={2019},
13
+ }
14
+
15
+ @article{duplicate_key,
16
+ title={First definition},
17
+ author={First, A.},
18
+ year={2020},
19
+ }
20
+
21
+ @article{duplicate_key,
22
+ title={Second definition of the same key},
23
+ author={Second, B.},
24
+ year={2021},
25
+ }
examples/sample.tex CHANGED
@@ -9,10 +9,14 @@
9
  \section{Introduction}
10
  This paper investigate the role of attention mechanisms in transformer
11
  architectures. Recent work~\cite{vaswani2017} have shown that self-attention
12
- outperform recurrence on many task.
 
13
 
14
  We propose an novel method, which we call FastAttn, that reduce the quadratic
15
  cost of attention to $O(n \log n)$. Their approach are evaluated on three
16
  benchmarks and shows consistent improvment over the baselines.
17
 
 
 
 
18
  \end{document}
 
9
  \section{Introduction}
10
  This paper investigate the role of attention mechanisms in transformer
11
  architectures. Recent work~\cite{vaswani2017} have shown that self-attention
12
+ outperform recurrence on many task. See also~\cite{nonexistent2024} for
13
+ context.
14
 
15
  We propose an novel method, which we call FastAttn, that reduce the quadratic
16
  cost of attention to $O(n \log n)$. Their approach are evaluated on three
17
  benchmarks and shows consistent improvment over the baselines.
18
 
19
+ \bibliographystyle{plain}
20
+ \bibliography{refs}
21
+
22
  \end{document}
src/qalmsw/bib/__init__.py ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ from qalmsw.bib.parser import BibEntry, parse_bib_file, parse_bib_text
2
+
3
+ __all__ = ["BibEntry", "parse_bib_file", "parse_bib_text"]
src/qalmsw/bib/parser.py ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Minimal BibTeX entry extractor.
2
+
3
+ Pulls out ``@type{key,`` headers and their source line. We deliberately don't parse
4
+ fields — nested braces in values make full BibTeX parsing a rabbit hole, and the
5
+ citation checker only needs keys + line numbers to produce MISSING / UNUSED /
6
+ DUPLICATE findings. Entries declared inside comment blocks are excluded by pre-stripping
7
+ BibTeX ``@comment{...}`` blocks and ``%``-line comments.
8
+ """
9
+ from __future__ import annotations
10
+
11
+ import re
12
+ from dataclasses import dataclass
13
+ from pathlib import Path
14
+
15
+ _ENTRY_START_RE = re.compile(
16
+ r"^[ \t]*@(\w+)\s*\{\s*([^,\s}]+)\s*,",
17
+ re.MULTILINE,
18
+ )
19
+ _LINE_COMMENT_RE = re.compile(r"(?<!\\)%[^\n]*")
20
+
21
+ _SKIPPABLE_TYPES = frozenset({"string", "preamble", "comment"})
22
+
23
+
24
+ @dataclass(frozen=True)
25
+ class BibEntry:
26
+ key: str
27
+ entry_type: str
28
+ file: Path
29
+ line: int
30
+
31
+
32
+ def _strip_line_comments_preserve_lines(text: str) -> str:
33
+ return _LINE_COMMENT_RE.sub("", text)
34
+
35
+
36
+ def parse_bib_text(text: str, source: Path) -> list[BibEntry]:
37
+ cleaned = _strip_line_comments_preserve_lines(text)
38
+ entries: list[BibEntry] = []
39
+ for match in _ENTRY_START_RE.finditer(cleaned):
40
+ entry_type = match.group(1).lower()
41
+ if entry_type in _SKIPPABLE_TYPES:
42
+ continue
43
+ key = match.group(2).strip()
44
+ line = cleaned.count("\n", 0, match.start()) + 1
45
+ entries.append(BibEntry(key=key, entry_type=entry_type, file=source, line=line))
46
+ return entries
47
+
48
+
49
+ def parse_bib_file(path: Path) -> list[BibEntry]:
50
+ return parse_bib_text(path.read_text(encoding="utf-8"), source=path)
src/qalmsw/checkers/__init__.py CHANGED
@@ -1,4 +1,5 @@
1
  from qalmsw.checkers.base import Checker, Finding, Severity
 
2
  from qalmsw.checkers.grammar import GrammarChecker
3
 
4
- __all__ = ["Checker", "Finding", "GrammarChecker", "Severity"]
 
1
  from qalmsw.checkers.base import Checker, Finding, Severity
2
+ from qalmsw.checkers.citations import CitationChecker
3
  from qalmsw.checkers.grammar import GrammarChecker
4
 
5
+ __all__ = ["Checker", "CitationChecker", "Finding", "GrammarChecker", "Severity"]
src/qalmsw/checkers/base.py CHANGED
@@ -10,7 +10,7 @@ from typing import Protocol
10
 
11
  from pydantic import BaseModel, Field
12
 
13
- from qalmsw.parse import Paragraph
14
 
15
 
16
  class Severity(str, Enum):
@@ -26,9 +26,13 @@ class Finding(BaseModel):
26
  message: str
27
  suggestion: str | None = None
28
  excerpt: str | None = Field(default=None, description="Short source snippet this refers to")
 
 
 
 
29
 
30
 
31
  class Checker(Protocol):
32
  name: str
33
 
34
- def check(self, paragraphs: list[Paragraph]) -> list[Finding]: ...
 
10
 
11
  from pydantic import BaseModel, Field
12
 
13
+ from qalmsw.document import Document
14
 
15
 
16
  class Severity(str, Enum):
 
26
  message: str
27
  suggestion: str | None = None
28
  excerpt: str | None = Field(default=None, description="Short source snippet this refers to")
29
+ file: str | None = Field(
30
+ default=None,
31
+ description="Source file if different from the document being checked (e.g. a .bib)",
32
+ )
33
 
34
 
35
  class Checker(Protocol):
36
  name: str
37
 
38
+ def check(self, doc: Document) -> list[Finding]: ...
src/qalmsw/checkers/citations.py ADDED
@@ -0,0 +1,81 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Citation consistency checker.
2
+
3
+ Cross-references the ``\\cite{}`` keys found in the document against the parsed
4
+ ``.bib`` entries and emits three kinds of findings:
5
+
6
+ * **MISSING** — a cited key has no matching bib entry. Error; blocks a clean build.
7
+ * **UNUSED** — a bib entry that no ``\\cite`` references. Info; drafts routinely carry
8
+ these and the user can choose to prune.
9
+ * **DUPLICATE** — the same key is defined by multiple bib entries. Warning; BibTeX will
10
+ silently use one and drop the rest.
11
+
12
+ Bib entries are passed in at construction time because discovering and parsing ``.bib``
13
+ files is the CLI's responsibility (it knows the document's directory and can resolve
14
+ relative paths correctly).
15
+ """
16
+ from __future__ import annotations
17
+
18
+ from qalmsw.bib import BibEntry
19
+ from qalmsw.checkers.base import Finding, Severity
20
+ from qalmsw.document import Document
21
+ from qalmsw.parse import scan_citations
22
+
23
+
24
+ class CitationChecker:
25
+ name = "citations"
26
+
27
+ def __init__(self, bib_entries: list[BibEntry]) -> None:
28
+ self._bib_entries = bib_entries
29
+
30
+ def check(self, doc: Document) -> list[Finding]:
31
+ citations = scan_citations(doc.source)
32
+ cited_keys = {c.key for c in citations}
33
+
34
+ entries_by_key: dict[str, list[BibEntry]] = {}
35
+ for entry in self._bib_entries:
36
+ entries_by_key.setdefault(entry.key, []).append(entry)
37
+ known_keys = set(entries_by_key)
38
+
39
+ findings: list[Finding] = []
40
+
41
+ for cite in citations:
42
+ if cite.key not in known_keys:
43
+ findings.append(
44
+ Finding(
45
+ checker=self.name,
46
+ severity=Severity.error,
47
+ line=cite.line,
48
+ message=f"Cited key '{cite.key}' not found in bibliography.",
49
+ excerpt=f"\\cite{{{cite.key}}}",
50
+ )
51
+ )
52
+
53
+ for key, entries in entries_by_key.items():
54
+ if key not in cited_keys:
55
+ first = entries[0]
56
+ findings.append(
57
+ Finding(
58
+ checker=self.name,
59
+ severity=Severity.info,
60
+ file=str(first.file),
61
+ line=first.line,
62
+ message=f"Unused bibliography entry '{key}'.",
63
+ )
64
+ )
65
+ if len(entries) > 1:
66
+ primary = entries[0]
67
+ for dup in entries[1:]:
68
+ findings.append(
69
+ Finding(
70
+ checker=self.name,
71
+ severity=Severity.warning,
72
+ file=str(dup.file),
73
+ line=dup.line,
74
+ message=(
75
+ f"Duplicate bibliography key '{key}' "
76
+ f"(first defined at {primary.file}:{primary.line})."
77
+ ),
78
+ )
79
+ )
80
+
81
+ return findings
src/qalmsw/checkers/grammar.py CHANGED
@@ -7,8 +7,9 @@ LLM to count lines itself is unreliable for small local models, so we don't.
7
  from __future__ import annotations
8
 
9
  from qalmsw.checkers.base import Finding, Severity
 
10
  from qalmsw.llm import LLMClient
11
- from qalmsw.parse import Paragraph
12
 
13
  _SYSTEM_PROMPT = """You are a grammar and style checker for scientific LaTeX writing.
14
 
@@ -39,9 +40,11 @@ class GrammarChecker:
39
  def __init__(self, llm: LLMClient) -> None:
40
  self._llm = llm
41
 
42
- def check(self, paragraphs: list[Paragraph]) -> list[Finding]:
43
  findings: list[Finding] = []
44
- for para in paragraphs:
 
 
45
  result = self._llm.complete_json(_SYSTEM_PROMPT, para.text)
46
  for raw in result.get("issues", []):
47
  findings.append(_to_finding(raw, para, self.name))
 
7
  from __future__ import annotations
8
 
9
  from qalmsw.checkers.base import Finding, Severity
10
+ from qalmsw.document import Document
11
  from qalmsw.llm import LLMClient
12
+ from qalmsw.parse import Paragraph, has_prose
13
 
14
  _SYSTEM_PROMPT = """You are a grammar and style checker for scientific LaTeX writing.
15
 
 
40
  def __init__(self, llm: LLMClient) -> None:
41
  self._llm = llm
42
 
43
+ def check(self, doc: Document) -> list[Finding]:
44
  findings: list[Finding] = []
45
+ for para in doc.paragraphs:
46
+ if not has_prose(para.text):
47
+ continue
48
  result = self._llm.complete_json(_SYSTEM_PROMPT, para.text)
49
  for raw in result.get("issues", []):
50
  findings.append(_to_finding(raw, para, self.name))
src/qalmsw/cli.py CHANGED
@@ -7,9 +7,11 @@ import typer
7
  from rich.console import Console
8
 
9
  from qalmsw import __version__
10
- from qalmsw.checkers import Finding, GrammarChecker
 
 
11
  from qalmsw.llm import LlamaCppClient
12
- from qalmsw.parse import has_prose, parse_paragraphs
13
  from qalmsw.report import render_findings
14
 
15
  app = typer.Typer(no_args_is_help=True, add_completion=False)
@@ -25,20 +27,52 @@ def version() -> None:
25
  @app.command()
26
  def check(
27
  file: Path = typer.Argument(..., exists=True, readable=True, help="Path to a .tex file"),
 
 
 
 
 
 
 
28
  base_url: str | None = typer.Option(None, "--base-url", envvar="QALMSW_BASE_URL"),
29
  model: str | None = typer.Option(None, "--model", envvar="QALMSW_MODEL"),
30
  ) -> None:
31
  """Run QA checks on a LaTeX document."""
32
- source = file.read_text(encoding="utf-8")
33
- paragraphs = [p for p in parse_paragraphs(source) if has_prose(p.text)]
34
- console.print(f"[dim]{len(paragraphs)} prose paragraph(s) to check[/]")
35
 
36
- llm = LlamaCppClient(base_url=base_url, model=model)
37
- checkers = [GrammarChecker(llm)]
 
 
 
 
 
 
38
 
39
  findings: list[Finding] = []
40
  for c in checkers:
41
- findings.extend(c.check(paragraphs))
42
 
43
  render_findings(console, file, findings)
44
- raise typer.Exit(code=1 if findings else 0)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  from rich.console import Console
8
 
9
  from qalmsw import __version__
10
+ from qalmsw.bib import BibEntry, parse_bib_file
11
+ from qalmsw.checkers import Checker, CitationChecker, Finding, GrammarChecker
12
+ from qalmsw.document import Document
13
  from qalmsw.llm import LlamaCppClient
14
+ from qalmsw.parse import scan_bib_resources
15
  from qalmsw.report import render_findings
16
 
17
  app = typer.Typer(no_args_is_help=True, add_completion=False)
 
27
  @app.command()
28
  def check(
29
  file: Path = typer.Argument(..., exists=True, readable=True, help="Path to a .tex file"),
30
+ bib: list[Path] = typer.Option(
31
+ [],
32
+ "--bib",
33
+ help="Explicit .bib file(s). If omitted, .bib files are auto-discovered from "
34
+ "\\bibliography{} / \\addbibresource{} declarations.",
35
+ ),
36
+ skip_grammar: bool = typer.Option(False, "--skip-grammar", help="Skip LLM grammar checker"),
37
  base_url: str | None = typer.Option(None, "--base-url", envvar="QALMSW_BASE_URL"),
38
  model: str | None = typer.Option(None, "--model", envvar="QALMSW_MODEL"),
39
  ) -> None:
40
  """Run QA checks on a LaTeX document."""
41
+ doc = Document.load(file)
42
+ console.print(f"[dim]{len(doc.paragraphs)} paragraph(s) parsed[/]")
 
43
 
44
+ bib_paths = list(bib) if bib else _discover_bib_files(doc)
45
+ bib_entries = _load_bib_entries(bib_paths)
46
+ if bib_paths:
47
+ console.print(f"[dim]{len(bib_entries)} bib entries from {len(bib_paths)} file(s)[/]")
48
+
49
+ checkers: list[Checker] = [CitationChecker(bib_entries)]
50
+ if not skip_grammar:
51
+ checkers.append(GrammarChecker(LlamaCppClient(base_url=base_url, model=model)))
52
 
53
  findings: list[Finding] = []
54
  for c in checkers:
55
+ findings.extend(c.check(doc))
56
 
57
  render_findings(console, file, findings)
58
+ raise typer.Exit(code=1 if any(f.severity.value == "error" for f in findings) else 0)
59
+
60
+
61
+ def _discover_bib_files(doc: Document) -> list[Path]:
62
+ names = scan_bib_resources(doc.source)
63
+ base_dir = doc.path.parent
64
+ resolved: list[Path] = []
65
+ for name in names:
66
+ candidate = base_dir / (name if name.endswith(".bib") else f"{name}.bib")
67
+ if candidate.exists():
68
+ resolved.append(candidate)
69
+ else:
70
+ console.print(f"[yellow]warning[/]: referenced bib file not found: {candidate}")
71
+ return resolved
72
+
73
+
74
+ def _load_bib_entries(paths: list[Path]) -> list[BibEntry]:
75
+ entries: list[BibEntry] = []
76
+ for p in paths:
77
+ entries.extend(parse_bib_file(p))
78
+ return entries
src/qalmsw/document.py ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """The unit of input every checker operates on.
2
+
3
+ `Document` bundles the .tex path, full source, and parsed paragraphs so individual
4
+ checkers don't each re-parse the file and can choose which level of detail to consume
5
+ (raw source for citation scanning, paragraphs for grammar, etc.).
6
+ """
7
+ from __future__ import annotations
8
+
9
+ from dataclasses import dataclass
10
+ from pathlib import Path
11
+
12
+ from qalmsw.parse import Paragraph, parse_paragraphs
13
+
14
+
15
+ @dataclass(frozen=True)
16
+ class Document:
17
+ path: Path
18
+ source: str
19
+ paragraphs: list[Paragraph]
20
+
21
+ @classmethod
22
+ def load(cls, path: Path) -> Document:
23
+ source = path.read_text(encoding="utf-8")
24
+ return cls(path=path, source=source, paragraphs=parse_paragraphs(source))
src/qalmsw/parse/__init__.py CHANGED
@@ -1,3 +1,12 @@
 
1
  from qalmsw.parse.tex import Paragraph, extract_body, has_prose, parse_paragraphs
2
 
3
- __all__ = ["Paragraph", "extract_body", "has_prose", "parse_paragraphs"]
 
 
 
 
 
 
 
 
 
1
+ from qalmsw.parse.citations import CitationRef, scan_bib_resources, scan_citations
2
  from qalmsw.parse.tex import Paragraph, extract_body, has_prose, parse_paragraphs
3
 
4
+ __all__ = [
5
+ "CitationRef",
6
+ "Paragraph",
7
+ "extract_body",
8
+ "has_prose",
9
+ "parse_paragraphs",
10
+ "scan_bib_resources",
11
+ "scan_citations",
12
+ ]
src/qalmsw/parse/citations.py ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Scan a .tex source for citation keys and bibliography resource declarations.
2
+
3
+ Works directly on the raw source (after comment stripping) because citations can appear
4
+ anywhere — not only inside prose paragraphs — and we need file-accurate line numbers.
5
+ """
6
+ from __future__ import annotations
7
+
8
+ import re
9
+ from dataclasses import dataclass
10
+
11
+ from qalmsw.parse.tex import _COMMENT_RE
12
+
13
+ # Matches natbib + biblatex cite commands: \cite, \citep, \citet, \citeauthor,
14
+ # \citeyear, \nocite, \parencite, \textcite, \autocite, \footcite (+ star variants,
15
+ # optional pre/post-note brackets).
16
+ _CITE_RE = re.compile(
17
+ r"\\(?:no|paren|text|auto|foot)?cite[a-zA-Z]*\*?"
18
+ r"(?:\[[^\]]*\])*"
19
+ r"\{([^}]+)\}"
20
+ )
21
+
22
+ # \bibliography{a,b} (bibtex) and \addbibresource{a.bib} (biblatex)
23
+ _BIBLIOGRAPHY_RE = re.compile(r"\\bibliography\{([^}]+)\}")
24
+ _ADDBIBRESOURCE_RE = re.compile(r"\\addbibresource\{([^}]+)\}")
25
+
26
+
27
+ @dataclass(frozen=True)
28
+ class CitationRef:
29
+ key: str
30
+ line: int
31
+
32
+
33
+ def scan_citations(source: str) -> list[CitationRef]:
34
+ """Return every cited key with its 1-indexed source line."""
35
+ text = _COMMENT_RE.sub("", source)
36
+ refs: list[CitationRef] = []
37
+ for match in _CITE_RE.finditer(text):
38
+ line = text.count("\n", 0, match.start()) + 1
39
+ for raw_key in match.group(1).split(","):
40
+ key = raw_key.strip()
41
+ if key:
42
+ refs.append(CitationRef(key=key, line=line))
43
+ return refs
44
+
45
+
46
+ def scan_bib_resources(source: str) -> list[str]:
47
+ """Return the list of .bib resource names declared in the source.
48
+
49
+ Names from ``\\bibliography{}`` are returned without extension (BibTeX convention);
50
+ names from ``\\addbibresource{}`` are returned verbatim (biblatex always includes
51
+ the extension). Callers resolve these against the document's directory.
52
+ """
53
+ text = _COMMENT_RE.sub("", source)
54
+ names: list[str] = []
55
+ for match in _BIBLIOGRAPHY_RE.finditer(text):
56
+ names.extend(n.strip() for n in match.group(1).split(",") if n.strip())
57
+ for match in _ADDBIBRESOURCE_RE.finditer(text):
58
+ n = match.group(1).strip()
59
+ if n:
60
+ names.append(n)
61
+ return names
src/qalmsw/report/text.py CHANGED
@@ -19,10 +19,11 @@ def render_findings(console: Console, file: Path, findings: list[Finding]) -> No
19
  console.print(f"[green]{file}[/]: no issues found")
20
  return
21
 
22
- for f in sorted(findings, key=lambda f: (f.line, f.checker)):
23
  style = _SEVERITY_STYLE[f.severity]
 
24
  console.print(
25
- f"[{style}]{f.severity.value}[/] [dim]{file}:{f.line}[/] "
26
  f"[cyan]{f.checker}[/] {f.message}"
27
  )
28
  if f.excerpt:
 
19
  console.print(f"[green]{file}[/]: no issues found")
20
  return
21
 
22
+ for f in sorted(findings, key=lambda f: (f.file or "", f.line, f.checker)):
23
  style = _SEVERITY_STYLE[f.severity]
24
+ loc = f"{f.file or file}:{f.line}"
25
  console.print(
26
+ f"[{style}]{f.severity.value}[/] [dim]{loc}[/] "
27
  f"[cyan]{f.checker}[/] {f.message}"
28
  )
29
  if f.excerpt:
tests/test_bib_parser.py ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from pathlib import Path
2
+
3
+ from qalmsw.bib import parse_bib_text
4
+
5
+
6
+ def _parse(text: str):
7
+ return parse_bib_text(text, source=Path("test.bib"))
8
+
9
+
10
+ def test_simple_entry():
11
+ entries = _parse("@article{smith2020, title={Hello}, author={Smith}}\n")
12
+ assert len(entries) == 1
13
+ assert entries[0].key == "smith2020"
14
+ assert entries[0].entry_type == "article"
15
+ assert entries[0].line == 1
16
+
17
+
18
+ def test_multiple_entries_track_lines():
19
+ text = (
20
+ "@book{a2000, title={A}}\n"
21
+ "\n"
22
+ "@inproceedings{b2001,\n"
23
+ " title={B},\n"
24
+ "}\n"
25
+ )
26
+ entries = _parse(text)
27
+ assert [(e.key, e.line) for e in entries] == [("a2000", 1), ("b2001", 3)]
28
+
29
+
30
+ def test_string_preamble_and_comment_entries_are_ignored():
31
+ text = (
32
+ "@string{me = \"John Doe\"}\n"
33
+ "@preamble{\"hi\"}\n"
34
+ "@comment{ignored}\n"
35
+ "@article{real2020, title={X}}\n"
36
+ )
37
+ entries = _parse(text)
38
+ assert [e.key for e in entries] == ["real2020"]
39
+
40
+
41
+ def test_line_comment_entries_are_stripped():
42
+ text = "% @article{hidden, title={x}}\n@article{visible, title={y}}\n"
43
+ entries = _parse(text)
44
+ assert [e.key for e in entries] == ["visible"]
45
+
46
+
47
+ def test_entry_type_is_lowercased():
48
+ entries = _parse("@ARTICLE{k, title={T}}\n")
49
+ assert entries[0].entry_type == "article"
tests/test_citation_scan.py ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from qalmsw.parse import scan_bib_resources, scan_citations
2
+
3
+
4
+ def test_single_cite():
5
+ refs = scan_citations("See \\cite{smith2020} for details.\n")
6
+ assert [(r.key, r.line) for r in refs] == [("smith2020", 1)]
7
+
8
+
9
+ def test_multiple_keys_in_one_cite():
10
+ refs = scan_citations("\\cite{a,b, c}\n")
11
+ assert [r.key for r in refs] == ["a", "b", "c"]
12
+
13
+
14
+ def test_citep_with_pre_and_post_notes():
15
+ refs = scan_citations("\\citep[see][p.~42]{jones1999}\n")
16
+ assert [(r.key, r.line) for r in refs] == [("jones1999", 1)]
17
+
18
+
19
+ def test_line_tracking_across_multiple():
20
+ source = "first line\n\\cite{a}\n\n\\citet{b}\n"
21
+ refs = scan_citations(source)
22
+ assert [(r.key, r.line) for r in refs] == [("a", 2), ("b", 4)]
23
+
24
+
25
+ def test_cite_in_comment_is_ignored():
26
+ refs = scan_citations("real text %\\cite{hidden}\n\\cite{visible}\n")
27
+ assert [r.key for r in refs] == ["visible"]
28
+
29
+
30
+ def test_bibliography_directive():
31
+ names = scan_bib_resources("\\bibliography{refs,extra}\n")
32
+ assert names == ["refs", "extra"]
33
+
34
+
35
+ def test_addbibresource_directive():
36
+ names = scan_bib_resources("\\addbibresource{bibliography.bib}\n")
37
+ assert names == ["bibliography.bib"]
tests/test_citations_checker.py ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from pathlib import Path
2
+
3
+ from qalmsw.bib import BibEntry
4
+ from qalmsw.checkers import CitationChecker, Severity
5
+ from qalmsw.document import Document
6
+
7
+
8
+ def _doc(source: str) -> Document:
9
+ return Document(path=Path("paper.tex"), source=source, paragraphs=[])
10
+
11
+
12
+ def _entry(key: str, line: int = 1, file: str = "refs.bib") -> BibEntry:
13
+ return BibEntry(key=key, entry_type="article", file=Path(file), line=line)
14
+
15
+
16
+ def test_missing_cite_key_is_error():
17
+ doc = _doc("Hello \\cite{ghost} world\n")
18
+ findings = CitationChecker([]).check(doc)
19
+ missing = [f for f in findings if "not found" in f.message]
20
+ assert len(missing) == 1
21
+ assert missing[0].severity == Severity.error
22
+ assert missing[0].line == 1
23
+
24
+
25
+ def test_cited_key_present_in_bib_is_silent():
26
+ doc = _doc("Hello \\cite{smith2020} world\n")
27
+ findings = CitationChecker([_entry("smith2020")]).check(doc)
28
+ assert [f for f in findings if "not found" in f.message] == []
29
+
30
+
31
+ def test_unused_entry_is_info():
32
+ doc = _doc("Hello world\n")
33
+ findings = CitationChecker([_entry("orphan")]).check(doc)
34
+ assert len(findings) == 1
35
+ assert findings[0].severity == Severity.info
36
+ assert "Unused" in findings[0].message
37
+ assert findings[0].file == "refs.bib"
38
+
39
+
40
+ def test_duplicate_entries_produce_warnings_for_each_after_first():
41
+ doc = _doc("\\cite{k}\n")
42
+ entries = [_entry("k", line=10), _entry("k", line=20), _entry("k", line=30)]
43
+ findings = CitationChecker(entries).check(doc)
44
+ dups = [f for f in findings if "Duplicate" in f.message]
45
+ assert [(f.line, f.severity) for f in dups] == [
46
+ (20, Severity.warning),
47
+ (30, Severity.warning),
48
+ ]
tests/test_grammar.py CHANGED
@@ -1,7 +1,14 @@
 
 
1
  from qalmsw.checkers import GrammarChecker, Severity
 
2
  from qalmsw.parse import Paragraph
3
 
4
 
 
 
 
 
5
  class FakeLLM:
6
  def __init__(self, response: dict) -> None:
7
  self.response = response
@@ -30,7 +37,7 @@ def test_finding_shape_and_line_mapping():
30
  ]
31
  }
32
  )
33
- findings = GrammarChecker(llm).check([para])
34
  assert len(findings) == 1
35
  f = findings[0]
36
  assert f.checker == "grammar"
@@ -47,24 +54,43 @@ def test_excerpt_on_later_line_maps_correctly():
47
  end_line=6,
48
  )
49
  llm = FakeLLM({"issues": [{"excerpt": "issue here", "message": "x", "severity": "info"}]})
50
- findings = GrammarChecker(llm).check([para])
51
  assert findings[0].line == 6
52
 
53
 
54
  def test_empty_response_yields_no_findings():
55
- para = Paragraph(text="Clean.", start_line=1, end_line=1)
56
- assert GrammarChecker(FakeLLM({"issues": []})).check([para]) == []
 
 
 
 
57
 
58
 
59
  def test_unknown_severity_defaults_to_warning():
60
- para = Paragraph(text="x", start_line=1, end_line=1)
61
- llm = FakeLLM({"issues": [{"excerpt": "x", "message": "m", "severity": "BOGUS"}]})
62
- findings = GrammarChecker(llm).check([para])
 
 
 
 
63
  assert findings[0].severity == Severity.warning
64
 
65
 
66
  def test_missing_excerpt_falls_back_to_paragraph_start():
67
- para = Paragraph(text="content", start_line=7, end_line=7)
 
 
 
 
68
  llm = FakeLLM({"issues": [{"message": "vague", "severity": "info"}]})
69
- findings = GrammarChecker(llm).check([para])
70
  assert findings[0].line == 7
 
 
 
 
 
 
 
 
1
+ from pathlib import Path
2
+
3
  from qalmsw.checkers import GrammarChecker, Severity
4
+ from qalmsw.document import Document
5
  from qalmsw.parse import Paragraph
6
 
7
 
8
+ def _doc(paragraphs: list[Paragraph]) -> Document:
9
+ return Document(path=Path("test.tex"), source="", paragraphs=paragraphs)
10
+
11
+
12
  class FakeLLM:
13
  def __init__(self, response: dict) -> None:
14
  self.response = response
 
37
  ]
38
  }
39
  )
40
+ findings = GrammarChecker(llm).check(_doc([para]))
41
  assert len(findings) == 1
42
  f = findings[0]
43
  assert f.checker == "grammar"
 
54
  end_line=6,
55
  )
56
  llm = FakeLLM({"issues": [{"excerpt": "issue here", "message": "x", "severity": "info"}]})
57
+ findings = GrammarChecker(llm).check(_doc([para]))
58
  assert findings[0].line == 6
59
 
60
 
61
  def test_empty_response_yields_no_findings():
62
+ para = Paragraph(
63
+ text="The text here reads reasonably well and needs no changes.",
64
+ start_line=1,
65
+ end_line=1,
66
+ )
67
+ assert GrammarChecker(FakeLLM({"issues": []})).check(_doc([para])) == []
68
 
69
 
70
  def test_unknown_severity_defaults_to_warning():
71
+ para = Paragraph(
72
+ text="The first line contains some reasonably long prose content.",
73
+ start_line=1,
74
+ end_line=1,
75
+ )
76
+ llm = FakeLLM({"issues": [{"excerpt": "prose", "message": "m", "severity": "BOGUS"}]})
77
+ findings = GrammarChecker(llm).check(_doc([para]))
78
  assert findings[0].severity == Severity.warning
79
 
80
 
81
  def test_missing_excerpt_falls_back_to_paragraph_start():
82
+ para = Paragraph(
83
+ text="This paragraph contains enough words to qualify as prose.",
84
+ start_line=7,
85
+ end_line=7,
86
+ )
87
  llm = FakeLLM({"issues": [{"message": "vague", "severity": "info"}]})
88
+ findings = GrammarChecker(llm).check(_doc([para]))
89
  assert findings[0].line == 7
90
+
91
+
92
+ def test_prose_less_paragraphs_skip_llm_call():
93
+ para = Paragraph(text="\\maketitle", start_line=3, end_line=3)
94
+ llm = FakeLLM({"issues": [{"excerpt": "x", "message": "m"}]})
95
+ assert GrammarChecker(llm).check(_doc([para])) == []
96
+ assert llm.calls == []