pebaryan commited on
Commit
a438ef8
·
1 Parent(s): 63f36cc

Semantic Scholar backend, JSON output, image checker, batch mode

Browse files

- Add Semantic Scholar retrieval backend (free API, no auth, no CAPTCHAs)
as the default for --enable-claims. Google Scholar kept as opt-in via
--retrieval google-scholar.
- Add --json flag for machine-readable CI output.
- Add images checker: verifies \includegraphics files exist on disk,
tries common extensions when none given.
- Add batch mode: qalmsw check accepts multiple paths and globs.
- Add set_backend() for runtime retrieval switching.
- Remove old test_scholar.py (replaced by test_semantic_scholar.py).
- Update README and CLAUDE.md for all new features.

CLAUDE.md CHANGED
@@ -53,12 +53,16 @@ report.render_findings # rich-formatted terminal output
53
 
54
  ### Checker status
55
 
56
- | Checker | State | Shape |
57
- |------------|------------|-------------------------------------------------------------|
58
- | `grammar` | working | Per-paragraph LLM call, parallelizable, cheap |
59
- | `citations`| working | Deterministic `.bib` vs `\cite` cross-check (MISSING / UNUSED / DUPLICATE). No LLM. |
60
- | `reviewer` | working | One LLM call per `\section{}` (or whole body if none); over-long sections are truncated |
61
- | `claims` | working, opt-in | Two LLM calls per paragraph-with-citation (extract, then judge per (claim, cite)). Scholar abstracts cached per bib key within a run. Opt in with `--enable-claims` — slow and rate-limited. |
 
 
 
 
62
 
63
  When adding a checker: drop a file into `src/qalmsw/checkers/`, register it in `checkers/__init__.py`, wire it into `cli.py`'s `checkers` list, and add tests with a `FakeLLM` — don't hit the real server from tests.
64
 
@@ -72,5 +76,5 @@ When adding a checker: drop a file into `src/qalmsw/checkers/`, register it in `
72
 
73
  - No multi-file `\input{}` / `\include{}` resolution yet — single-file only.
74
  - No LLM-assisted citation verification (does this citation actually support this claim?). That's the `claims` checker's territory.
75
- - Retrieval starts with `src/qalmsw/retrieval/scholar.py` (Google Scholar via `scholarly`). **Scraping-based**; rate-limits and CAPTCHAs are expected under sustained use. Keep it for personal/interactive runs; fall back to Semantic Scholar or arXiv when CI-scale reliability matters.
76
- - No SARIF/JSON report formats yet only `report/text.py`. The `Finding` pydantic model is the serialization seam when those arrive.
 
53
 
54
  ### Checker status
55
 
56
+ | Checker | State | Shape |
57
+ |--------------|------------|-------------------------------------------------------------|
58
+ | `artifacts` | working | Deterministic regex scan for LLM meta-comments, placeholders, self-awareness, phantom refs. No LLM. Always runs. |
59
+ | `figures` | working | Deterministic scan for missing/placeholder captions, orphan labels, empty floats. No LLM. Always runs. |
60
+ | `images` | working | Verifies `\\includegraphics` files exist on disk relative to the .tex file. No LLM. Always runs. |
61
+ | `grammar` | working | Per-paragraph LLM call, parallelizable, cheap |
62
+ | `citations` | working | Deterministic `.bib` vs `\\cite` cross-check (MISSING / UNUSED / DUPLICATE). No LLM. |
63
+ | `references` | working | Verifies arXiv eprint IDs and DOIs resolve to real papers via live API calls. Network-backed. |
64
+ | `reviewer` | working | One LLM call per `\\section{}` (or whole body if none); over-long sections are truncated |
65
+ | `claims` | working, opt-in | Two LLM calls per paragraph-with-citation (extract, then judge per (claim, cite)). Paper abstracts fetched via retrieval backend, cached per bib key within a run. Opt in with `--enable-claims`. Retrieval backend selectable with `--retrieval` (default: `semantic-scholar`, alt: `google-scholar`). |
66
 
67
  When adding a checker: drop a file into `src/qalmsw/checkers/`, register it in `checkers/__init__.py`, wire it into `cli.py`'s `checkers` list, and add tests with a `FakeLLM` — don't hit the real server from tests.
68
 
 
76
 
77
  - No multi-file `\input{}` / `\include{}` resolution yet — single-file only.
78
  - No LLM-assisted citation verification (does this citation actually support this claim?). That's the `claims` checker's territory.
79
+ - Retrieval uses **Semantic Scholar** by default (free API, no auth, no CAPTCHAs). Google Scholar (`scholarly`) is available as an opt-in backend via `--retrieval google-scholar` but is scraping-based and may hit CAPTCHAs under sustained use. Backend switching is done at runtime via `qalmsw.retrieval.set_backend()`.
80
+ - JSON report output is available via `--json` for CI integration. The `Finding` pydantic model is the serialization seam for future SARIF support.
README.md CHANGED
@@ -5,7 +5,7 @@ Catches the artefacts that get you a 1-year arXiv ban — before you submit.
5
 
6
  ## What it catches
7
 
8
- qalmsw now checks for all three categories of "incontrovertible evidence" that arXiv's
9
  Code of Conduct penalises:
10
 
11
  | What arXiv flags | How qalmsw catches it |
@@ -17,9 +17,11 @@ Code of Conduct penalises:
17
  | **LLM self-awareness artifacts** ("as an AI language model", "I cannot provide") | `artifacts` checker |
18
  | **LLM-generated LaTeX boilerplate** (\lipsum, \blindtext, TODO/FIXME) | `artifacts` checker |
19
  | **Unreferenced figures/tables** | `figures` checker — warns on labels never \ref'd |
 
20
  | **Grammar & style issues** | `grammar` checker — per-paragraph LLM pass |
21
  | **Missing citations** | `citations` checker — cross-references \cite vs .bib |
22
  | **Substantive reviewer concerns** | `reviewer` checker — per-section LLM critique |
 
23
 
24
  The first three rows are the ones that get you banned. qalmsw catches all of them.
25
 
@@ -34,10 +36,14 @@ pip install -e '.[dev]'
34
  qalmsw check path/to/paper.tex # run all checkers
35
  qalmsw check --skip-grammar --skip-reviewer paper.tex # deterministic checks only (fast)
36
  qalmsw check --skip-grammar path/to/paper.tex # reviewer + citations + artifacts + references
37
- qalmsw check -j 4 path/to/paper.tex # fan out 4 parallel LLM calls (match server --parallel N)
38
  qalmsw check --bib refs.bib path/to/paper.tex # override .bib auto-discovery
 
 
 
39
 
40
- qalmsw scholar "Attention Is All You Need" # Google Scholar lookup (scraping; expect CAPTCHAs on bulk use)
 
41
  ```
42
 
43
  Environment variables:
@@ -51,6 +57,7 @@ Environment variables:
51
  |---|---|---|---|
52
  | `artifacts` | No | Instant | Scans for LLM meta-comments, placeholders, self-awareness, boilerplate |
53
  | `figures` | No | Instant | Checks captions, labels, refs, empty floats |
 
54
  | `citations` | No | Instant | `.bib` vs `\cite` cross-check: MISSING, UNUSED, DUPLICATE |
55
  | `references` | Network | Slow | Verifies arXiv IDs and DOIs resolve to real papers |
56
  | `grammar` | LLM | Per-paragraph | Grammar, spelling, punctuation |
@@ -61,6 +68,17 @@ Exit code is `1` only when an `error`-severity finding is present (missing citat
61
  LLM artifact, hallucinated reference), so unused-bib-entry `info`s or duplicate-key
62
  `warning`s don't fail CI.
63
 
 
 
 
 
 
 
 
 
 
 
 
64
  ## arXiv Code of Conduct
65
 
66
  arXiv's Code of Conduct (May 2026) states:
@@ -94,8 +112,9 @@ Document.load
94
  checkers.* # each implements check(doc) -> list[Finding]
95
 
96
 
97
- report.render_findings # rich-formatted terminal output
98
  ```
99
 
100
- Deterministic checkers (artifacts, figures, citations) run first and always.
101
  LLM checkers (grammar, reviewer, claims) run only when a server is available.
 
 
5
 
6
  ## What it catches
7
 
8
+ qalmsw checks for all three categories of "incontrovertible evidence" that arXiv's
9
  Code of Conduct penalises:
10
 
11
  | What arXiv flags | How qalmsw catches it |
 
17
  | **LLM self-awareness artifacts** ("as an AI language model", "I cannot provide") | `artifacts` checker |
18
  | **LLM-generated LaTeX boilerplate** (\lipsum, \blindtext, TODO/FIXME) | `artifacts` checker |
19
  | **Unreferenced figures/tables** | `figures` checker — warns on labels never \ref'd |
20
+ | **Missing image files** (\includegraphics pointing to non-existent files) | `images` checker — verifies referenced images exist on disk |
21
  | **Grammar & style issues** | `grammar` checker — per-paragraph LLM pass |
22
  | **Missing citations** | `citations` checker — cross-references \cite vs .bib |
23
  | **Substantive reviewer concerns** | `reviewer` checker — per-section LLM critique |
24
+ | **Unsupported claims** *(opt-in)* | `claims` checker — checks each \cite-backed claim against the cited paper's abstract |
25
 
26
  The first three rows are the ones that get you banned. qalmsw catches all of them.
27
 
 
36
  qalmsw check path/to/paper.tex # run all checkers
37
  qalmsw check --skip-grammar --skip-reviewer paper.tex # deterministic checks only (fast)
38
  qalmsw check --skip-grammar path/to/paper.tex # reviewer + citations + artifacts + references
39
+ qalmsw check -j 4 path/to/paper.tex # fan out 4 parallel LLM calls
40
  qalmsw check --bib refs.bib path/to/paper.tex # override .bib auto-discovery
41
+ qalmsw check --json paper.tex # JSON output for CI
42
+ qalmsw check ch1.tex ch2.tex ch3.tex # batch mode: check multiple files
43
+ qalmsw check "src/**/*.tex" # glob expansion
44
 
45
+ qalmsw scholar "Attention Is All You Need" # Semantic Scholar lookup (default, free API)
46
+ qalmsw check --enable-claims --retrieval google-scholar paper.tex # claims check via Google Scholar
47
  ```
48
 
49
  Environment variables:
 
57
  |---|---|---|---|
58
  | `artifacts` | No | Instant | Scans for LLM meta-comments, placeholders, self-awareness, boilerplate |
59
  | `figures` | No | Instant | Checks captions, labels, refs, empty floats |
60
+ | `images` | No | Instant | Verifies \includegraphics files exist on disk |
61
  | `citations` | No | Instant | `.bib` vs `\cite` cross-check: MISSING, UNUSED, DUPLICATE |
62
  | `references` | Network | Slow | Verifies arXiv IDs and DOIs resolve to real papers |
63
  | `grammar` | LLM | Per-paragraph | Grammar, spelling, punctuation |
 
68
  LLM artifact, hallucinated reference), so unused-bib-entry `info`s or duplicate-key
69
  `warning`s don't fail CI.
70
 
71
+ ### Retrieval backends
72
+
73
+ The `claims` checker needs to fetch paper abstracts. Two backends are available:
74
+
75
+ | Backend | Flag | Auth | Reliability | Speed |
76
+ |---|---|---|---|---|
77
+ | **Semantic Scholar** (default) | `--retrieval semantic-scholar` | None | High (real API) | ~1 req/sec |
78
+ | **Google Scholar** | `--retrieval google-scholar` | None | Low (scraping, CAPTCHAs) | ~1 req/sec |
79
+
80
+ Default is Semantic Scholar — no auth, no CAPTCHAs, works in CI.
81
+
82
  ## arXiv Code of Conduct
83
 
84
  arXiv's Code of Conduct (May 2026) states:
 
112
  checkers.* # each implements check(doc) -> list[Finding]
113
 
114
 
115
+ report.render_findings # rich-formatted terminal output (or --json for CI)
116
  ```
117
 
118
+ Deterministic checkers (artifacts, figures, images, citations) run first and always.
119
  LLM checkers (grammar, reviewer, claims) run only when a server is available.
120
+ Network checkers (references, claims) make live API calls.
src/qalmsw/checkers/__init__.py CHANGED
@@ -4,6 +4,7 @@ from qalmsw.checkers.citations import CitationChecker
4
  from qalmsw.checkers.claims import ClaimsChecker
5
  from qalmsw.checkers.figures import FigureTableChecker
6
  from qalmsw.checkers.grammar import GrammarChecker
 
7
  from qalmsw.checkers.references import ReferenceChecker
8
  from qalmsw.checkers.reviewer import ReviewerChecker
9
 
@@ -15,6 +16,7 @@ __all__ = [
15
  "FigureTableChecker",
16
  "Finding",
17
  "GrammarChecker",
 
18
  "ReferenceChecker",
19
  "ReviewerChecker",
20
  "Severity",
 
4
  from qalmsw.checkers.claims import ClaimsChecker
5
  from qalmsw.checkers.figures import FigureTableChecker
6
  from qalmsw.checkers.grammar import GrammarChecker
7
+ from qalmsw.checkers.images import ImageChecker
8
  from qalmsw.checkers.references import ReferenceChecker
9
  from qalmsw.checkers.reviewer import ReviewerChecker
10
 
 
16
  "FigureTableChecker",
17
  "Finding",
18
  "GrammarChecker",
19
+ "ImageChecker",
20
  "ReferenceChecker",
21
  "ReviewerChecker",
22
  "Severity",
src/qalmsw/checkers/images.py ADDED
@@ -0,0 +1,109 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Image file existence checker.
2
+
3
+ Scans for ``\\includegraphics`` commands in the LaTeX source and verifies
4
+ the referenced image files exist on disk relative to the document directory.
5
+
6
+ Missing images are a common sign of uncurated LLM output — the model generates
7
+ ``\\includegraphics{results.png}`` without actually having created the file.
8
+
9
+ Tries common image extensions (.pdf, .png, .jpg, .jpeg, .eps, .svg) when
10
+ no extension is present.
11
+ """
12
+
13
+ from __future__ import annotations
14
+
15
+ import re
16
+ from pathlib import Path
17
+
18
+ from qalmsw.checkers.base import Finding, Severity
19
+ from qalmsw.document import Document
20
+
21
+ _GRAPHICS_RE = re.compile(
22
+ r"\\includegraphics(?:\s*\[[^\]]*\])?\s*\{([^}]+)\}"
23
+ )
24
+
25
+ _COMMON_EXTENSIONS: tuple[str, ...] = (
26
+ ".pdf", ".png", ".jpg", ".jpeg", ".eps", ".svg", ".gif", ".tiff",
27
+ )
28
+
29
+ # Extensions that aren't image files — skip these
30
+ _SKIP_EXTENSIONS: tuple[str, ...] = (".tex", ".cls", ".sty", ".bib", ".bst")
31
+
32
+
33
+ class ImageChecker:
34
+ """Checks that all \\includegraphics files exist relative to the document."""
35
+
36
+ name = "images"
37
+
38
+ def check(self, doc: Document) -> list[Finding]:
39
+ findings: list[Finding] = []
40
+ doc_dir = doc.path.parent
41
+
42
+ for match in _GRAPHICS_RE.finditer(doc.source):
43
+ raw_path = match.group(1).strip()
44
+ if not raw_path:
45
+ continue
46
+
47
+ line = doc.source[: match.start()].count("\n") + 1
48
+
49
+ # Skip obviously not-image paths
50
+ if any(raw_path.lower().endswith(ext) for ext in _SKIP_EXTENSIONS):
51
+ continue
52
+
53
+ resolved = _resolve_image(doc_dir, raw_path)
54
+ if resolved is None:
55
+ findings.append(
56
+ Finding(
57
+ checker=self.name,
58
+ severity=Severity.error,
59
+ line=line,
60
+ message=f"Image file not found: '{raw_path}'",
61
+ excerpt=raw_path[:80],
62
+ suggestion=(
63
+ f"Place the image at "
64
+ f"\\includegraphics{{{_suggest_path(doc_dir, raw_path)}}} "
65
+ f"or remove this reference."
66
+ ),
67
+ )
68
+ )
69
+
70
+ return findings
71
+
72
+
73
+ def _resolve_image(doc_dir: Path, raw_path: str) -> Path | None:
74
+ """Resolve an \\includegraphics path to an existing file, or None.
75
+
76
+ Tries the path as-is first, then with common extensions appended.
77
+ """
78
+ candidate = (doc_dir / raw_path).resolve()
79
+
80
+ # Try exact path first
81
+ if candidate.exists():
82
+ return candidate
83
+
84
+ # Try without extension if one is present
85
+ if candidate.suffix:
86
+ # Maybe the referenced path has an extension that doesn't match the actual file
87
+ # Try stripping the extension and re-adding common ones
88
+ stem = candidate.with_suffix("")
89
+ for ext in _COMMON_EXTENSIONS:
90
+ trial = stem.with_suffix(ext)
91
+ if trial.exists():
92
+ return trial
93
+ else:
94
+ # No extension — try all common ones
95
+ for ext in _COMMON_EXTENSIONS:
96
+ trial = candidate.with_suffix(ext)
97
+ if trial.exists():
98
+ return trial
99
+
100
+ return None
101
+
102
+
103
+ def _suggest_path(doc_dir: Path, raw_path: str) -> str:
104
+ """Return a suggested path hint for the error message."""
105
+ for ext in _COMMON_EXTENSIONS:
106
+ trial = (doc_dir / raw_path).with_suffix(ext)
107
+ if trial.exists():
108
+ return str(trial.relative_to(doc_dir))
109
+ return raw_path
src/qalmsw/cli.py CHANGED
@@ -1,6 +1,7 @@
1
  """`qalmsw` command-line entry point."""
2
  from __future__ import annotations
3
 
 
4
  from pathlib import Path
5
 
6
  import typer
@@ -16,14 +17,15 @@ from qalmsw.checkers import (
16
  FigureTableChecker,
17
  Finding,
18
  GrammarChecker,
 
19
  ReferenceChecker,
20
  ReviewerChecker,
21
  )
22
  from qalmsw.document import Document
23
  from qalmsw.llm import LlamaCppClient
24
  from qalmsw.parse import scan_bib_resources
25
- from qalmsw.report import render_findings
26
- from qalmsw.retrieval import search_by_title
27
 
28
  app = typer.Typer(no_args_is_help=True, add_completion=False)
29
  console = Console()
@@ -37,7 +39,10 @@ def version() -> None:
37
 
38
  @app.command()
39
  def check(
40
- file: Path = typer.Argument(..., exists=True, readable=True, help="Path to a .tex file"),
 
 
 
41
  bib: list[Path] = typer.Option(
42
  [],
43
  "--bib",
@@ -49,7 +54,13 @@ def check(
49
  enable_claims: bool = typer.Option(
50
  False,
51
  "--enable-claims",
52
- help="Enable claim-to-reference check via Google Scholar. Slow; may hit CAPTCHAs.",
 
 
 
 
 
 
53
  ),
54
  concurrency: int = typer.Option(
55
  1,
@@ -60,8 +71,54 @@ def check(
60
  ),
61
  base_url: str | None = typer.Option(None, "--base-url", envvar="QALMSW_BASE_URL"),
62
  model: str | None = typer.Option(None, "--model", envvar="QALMSW_MODEL"),
 
 
 
 
 
63
  ) -> None:
64
- """Run QA checks on a LaTeX document."""
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
65
  doc = Document.load(file)
66
  console.print(f"[dim]{len(doc.paragraphs)} paragraph(s) parsed[/]")
67
 
@@ -75,14 +132,22 @@ def check(
75
  )
76
  if inline:
77
  bib_entries = inline
78
- console.print(f"[dim]{len(inline)} bib entries from inline \\begin{{thebibliography}}[/]")
 
 
 
79
  else:
80
  console.print(
81
  "[yellow]warning[/]: no bib entries found (no --bib, no \\bibliography{}, "
82
  "no inline \\begin{thebibliography}); citation checks will be limited."
83
  )
84
 
85
- checkers: list[Checker] = [ArtifactChecker(), FigureTableChecker(), CitationChecker(bib_entries)]
 
 
 
 
 
86
  if bib_entries:
87
  checkers.append(ReferenceChecker(bib_entries))
88
  if not skip_grammar or not skip_reviewer or enable_claims:
@@ -98,17 +163,22 @@ def check(
98
  for c in checkers:
99
  findings.extend(c.check(doc))
100
 
101
- render_findings(console, file, findings)
102
- raise typer.Exit(code=1 if any(f.severity.value == "error" for f in findings) else 0)
 
 
 
 
103
 
104
 
105
  @app.command()
106
  def scholar(
107
  query: list[str] = typer.Argument(..., help="Title query (may be unquoted)"),
108
  ) -> None:
109
- """Look up the first Google Scholar match for a title query.
110
 
111
- Scraping-based; expect CAPTCHAs under sustained use.
 
112
  """
113
  text = " ".join(query)
114
  result = search_by_title(text)
@@ -122,6 +192,21 @@ def scholar(
122
  console.print(f"[bold]abstract:[/] {result.abstract or '(no abstract)'}")
123
 
124
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
125
  def _discover_bib_files(doc: Document) -> list[Path]:
126
  names = scan_bib_resources(doc.source)
127
  base_dir = doc.path.parent
@@ -140,3 +225,6 @@ def _load_bib_entries(paths: list[Path]) -> list[BibEntry]:
140
  for p in paths:
141
  entries.extend(parse_bib_file(p))
142
  return entries
 
 
 
 
1
  """`qalmsw` command-line entry point."""
2
  from __future__ import annotations
3
 
4
+ from glob import glob
5
  from pathlib import Path
6
 
7
  import typer
 
17
  FigureTableChecker,
18
  Finding,
19
  GrammarChecker,
20
+ ImageChecker,
21
  ReferenceChecker,
22
  ReviewerChecker,
23
  )
24
  from qalmsw.document import Document
25
  from qalmsw.llm import LlamaCppClient
26
  from qalmsw.parse import scan_bib_resources
27
+ from qalmsw.report import render_findings, render_findings_json
28
+ from qalmsw.retrieval import search_by_title, set_backend
29
 
30
  app = typer.Typer(no_args_is_help=True, add_completion=False)
31
  console = Console()
 
39
 
40
  @app.command()
41
  def check(
42
+ files: list[Path] = typer.Argument(
43
+ ...,
44
+ help="Path(s) to .tex file(s). Supports glob patterns (e.g. src/**/*.tex).",
45
+ ),
46
  bib: list[Path] = typer.Option(
47
  [],
48
  "--bib",
 
54
  enable_claims: bool = typer.Option(
55
  False,
56
  "--enable-claims",
57
+ help="Enable claim-to-reference check via Semantic Scholar.",
58
+ ),
59
+ retrieval: str = typer.Option(
60
+ "semantic-scholar",
61
+ "--retrieval",
62
+ help="Retrieval backend for --enable-claims: 'semantic-scholar' (default, free API) "
63
+ "or 'google-scholar' (scraping, may hit CAPTCHAs).",
64
  ),
65
  concurrency: int = typer.Option(
66
  1,
 
71
  ),
72
  base_url: str | None = typer.Option(None, "--base-url", envvar="QALMSW_BASE_URL"),
73
  model: str | None = typer.Option(None, "--model", envvar="QALMSW_MODEL"),
74
+ json_output: bool = typer.Option(
75
+ False,
76
+ "--json",
77
+ help="Output results as JSON (for CI integration).",
78
+ ),
79
  ) -> None:
80
+ """Run QA checks on one or more LaTeX documents."""
81
+ resolved = _resolve_files(files)
82
+ if not resolved:
83
+ console.print("[red]error[/]: no .tex files found matching the given paths")
84
+ raise typer.Exit(code=1)
85
+
86
+ # Select retrieval backend for claims checker
87
+ set_backend(retrieval)
88
+
89
+ if len(resolved) > 1:
90
+ console.print(f"[dim]{len(resolved)} file(s) to check[/]")
91
+
92
+ any_errors = False
93
+ for file in resolved:
94
+ if not json_output:
95
+ if len(resolved) > 1:
96
+ console.print(f"\n[bold]{file}[/]")
97
+ else:
98
+ console.print(f"[bold]{file}[/]")
99
+
100
+ has_errors = _check_single(
101
+ file, bib, skip_grammar, skip_reviewer, enable_claims,
102
+ concurrency, base_url, model, json_output,
103
+ )
104
+ if has_errors:
105
+ any_errors = True
106
+
107
+ raise typer.Exit(code=1 if any_errors else 0)
108
+
109
+
110
+ def _check_single(
111
+ file: Path,
112
+ bib: list[Path],
113
+ skip_grammar: bool,
114
+ skip_reviewer: bool,
115
+ enable_claims: bool,
116
+ concurrency: int,
117
+ base_url: str | None,
118
+ model: str | None,
119
+ json_output: bool,
120
+ ) -> bool:
121
+ """Run checks on a single file. Returns True if any errors found."""
122
  doc = Document.load(file)
123
  console.print(f"[dim]{len(doc.paragraphs)} paragraph(s) parsed[/]")
124
 
 
132
  )
133
  if inline:
134
  bib_entries = inline
135
+ console.print(
136
+ f"[dim]{len(inline)} bib entries from inline "
137
+ f"\\begin{{thebibliography}}[/]"
138
+ )
139
  else:
140
  console.print(
141
  "[yellow]warning[/]: no bib entries found (no --bib, no \\bibliography{}, "
142
  "no inline \\begin{thebibliography}); citation checks will be limited."
143
  )
144
 
145
+ checkers: list[Checker] = [
146
+ ArtifactChecker(),
147
+ FigureTableChecker(),
148
+ ImageChecker(),
149
+ CitationChecker(bib_entries),
150
+ ]
151
  if bib_entries:
152
  checkers.append(ReferenceChecker(bib_entries))
153
  if not skip_grammar or not skip_reviewer or enable_claims:
 
163
  for c in checkers:
164
  findings.extend(c.check(doc))
165
 
166
+ if json_output:
167
+ console.print(render_findings_json(file, findings))
168
+ else:
169
+ render_findings(console, file, findings)
170
+
171
+ return any(f.severity.value == "error" for f in findings)
172
 
173
 
174
  @app.command()
175
  def scholar(
176
  query: list[str] = typer.Argument(..., help="Title query (may be unquoted)"),
177
  ) -> None:
178
+ """Look up the first Semantic Scholar match for a title query.
179
 
180
+ Default backend is Semantic Scholar (free API, no CAPTCHAs).
181
+ Use --retrieval google-scholar for the scraping-based backend.
182
  """
183
  text = " ".join(query)
184
  result = search_by_title(text)
 
192
  console.print(f"[bold]abstract:[/] {result.abstract or '(no abstract)'}")
193
 
194
 
195
+ def _resolve_files(paths: list[Path]) -> list[Path]:
196
+ """Expand globs and filter to existing .tex files."""
197
+ resolved: list[Path] = []
198
+ for p in paths:
199
+ if p.is_file():
200
+ resolved.append(p)
201
+ else:
202
+ # Try glob expansion on the string form
203
+ for match in sorted(glob(str(p))):
204
+ candidate = Path(match)
205
+ if candidate.is_file() and candidate.suffix == ".tex":
206
+ resolved.append(candidate)
207
+ return resolved
208
+
209
+
210
  def _discover_bib_files(doc: Document) -> list[Path]:
211
  names = scan_bib_resources(doc.source)
212
  base_dir = doc.path.parent
 
225
  for p in paths:
226
  entries.extend(parse_bib_file(p))
227
  return entries
228
+
229
+
230
+
src/qalmsw/report/__init__.py CHANGED
@@ -1,3 +1,4 @@
 
1
  from qalmsw.report.text import render_findings
2
 
3
- __all__ = ["render_findings"]
 
1
+ from qalmsw.report.json import render_findings_json
2
  from qalmsw.report.text import render_findings
3
 
4
+ __all__ = ["render_findings", "render_findings_json"]
src/qalmsw/report/json.py ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """JSON report output."""
2
+ from __future__ import annotations
3
+
4
+ import json
5
+ from pathlib import Path
6
+ from typing import Any
7
+
8
+ from qalmsw.checkers import Finding
9
+
10
+
11
+ def render_findings_json(file: Path, findings: list[Finding]) -> str:
12
+ """Return findings as a JSON string, one object with file info and results list.
13
+
14
+ Structure:
15
+ {
16
+ "file": "paper.tex",
17
+ "total": 3,
18
+ "by_severity": {"error": 1, "warning": 1, "info": 1},
19
+ "findings": [ ... Finding.dict() ... ]
20
+ }
21
+ """
22
+ by_severity: dict[str, int] = {}
23
+ for f in findings:
24
+ by_severity[f.severity.value] = by_severity.get(f.severity.value, 0) + 1
25
+
26
+ payload: dict[str, Any] = {
27
+ "file": str(file),
28
+ "total": len(findings),
29
+ "by_severity": by_severity,
30
+ "findings": [_finding_dict(f) for f in findings],
31
+ }
32
+ return json.dumps(payload, indent=2, default=str)
33
+
34
+
35
+ def _finding_dict(f: Finding) -> dict[str, Any]:
36
+ d: dict[str, Any] = {
37
+ "checker": f.checker,
38
+ "severity": f.severity.value,
39
+ "line": f.line,
40
+ "message": f.message,
41
+ }
42
+ if f.suggestion is not None:
43
+ d["suggestion"] = f.suggestion
44
+ if f.excerpt is not None:
45
+ d["excerpt"] = f.excerpt
46
+ if f.file is not None:
47
+ d["file"] = str(f.file)
48
+ return d
src/qalmsw/retrieval/__init__.py CHANGED
@@ -1,3 +1,33 @@
1
- from qalmsw.retrieval.scholar import ScholarResult, search_by_title
2
 
3
- __all__ = ["ScholarResult", "search_by_title"]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Retrieval backends for looking up paper metadata.
2
 
3
+ Default backend is Semantic Scholar (free API, no CAPTCHAs, no auth).
4
+ Switch to Google Scholar at runtime via ``set_backend('google-scholar')``
5
+ or use the CLI ``--retrieval`` flag.
6
+ """
7
+
8
+ from qalmsw.retrieval.scholar import ScholarResult
9
+ from qalmsw.retrieval.semantic_scholar import search_by_title # noqa: F401 — re-exported
10
+
11
+ __all__ = ["ScholarResult", "search_by_title", "set_backend"]
12
+
13
+
14
+ def set_backend(name: str) -> None:
15
+ """Switch the active retrieval backend at runtime.
16
+
17
+ This patches the module-level ``search_by_title`` so existing imports
18
+ like ``from qalmsw.retrieval import search_by_title`` pick up the change.
19
+
20
+ Parameters
21
+ ----------
22
+ name
23
+ ``'semantic-scholar'`` (default) or ``'google-scholar'``.
24
+ """
25
+ import qalmsw.retrieval as mod
26
+
27
+ if name == "google-scholar":
28
+ from qalmsw.retrieval.scholar import search_by_title as _fn
29
+ elif name == "semantic-scholar":
30
+ from qalmsw.retrieval.semantic_scholar import search_by_title as _fn
31
+ else:
32
+ raise ValueError(f"Unknown retrieval backend: {name!r}")
33
+ mod.search_by_title = _fn
src/qalmsw/retrieval/semantic_scholar.py ADDED
@@ -0,0 +1,123 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Semantic Scholar retrieval via the free API.
2
+
3
+ Replaces Google Scholar scraping (`scholarly`) for the claims checker.
4
+ Semantic Scholar has a free, no-auth-required API with generous rate limits.
5
+
6
+ API:
7
+ GET /graph/v1/paper/search?query={title}&limit=3
8
+ &fields=title,authors,year,abstract,externalIds,url
9
+
10
+ Docs:
11
+ https://api.semanticscholar.org/api-docs/graph
12
+ #tag/Paper-Data/operation/get_graph_search_papers
13
+
14
+ Rate limit: ~1 req/sec for free tier. We throttle to one per second.
15
+ """
16
+
17
+ from __future__ import annotations
18
+
19
+ import json
20
+ import time
21
+ import urllib.error
22
+ import urllib.parse
23
+ import urllib.request
24
+ from typing import Any
25
+
26
+ from qalmsw.retrieval.scholar import ScholarResult
27
+
28
+ _SEARCH_URL = "https://api.semanticscholar.org/graph/v1/paper/search"
29
+ _FIELDS = "title,authors,year,abstract,externalIds,url"
30
+ _USER_AGENT = "qalmsw/0.0.1"
31
+
32
+ _last_call = 0.0
33
+
34
+
35
+ def search_by_title(title: str) -> ScholarResult | None:
36
+ """Return the first Semantic Scholar match for ``title``, or ``None``.
37
+
38
+ Uses stdlib only (no external deps). Throttled to 1 req/sec.
39
+ """
40
+ global _last_call
41
+
42
+ encoded = urllib.parse.quote(title)
43
+ url = f"{_SEARCH_URL}?query={encoded}&limit=3&fields={_FIELDS}"
44
+
45
+ # Rate limit: at least 1 second between calls
46
+ now = time.time()
47
+ elapsed = now - _last_call
48
+ if elapsed < 1.0:
49
+ time.sleep(1.0 - elapsed)
50
+
51
+ try:
52
+ req = urllib.request.Request(url, headers={"User-Agent": _USER_AGENT})
53
+ with urllib.request.urlopen(req, timeout=15.0) as resp:
54
+ data = json.loads(resp.read().decode("utf-8"))
55
+ _last_call = time.time()
56
+ except (urllib.error.HTTPError, urllib.error.URLError, json.JSONDecodeError, OSError):
57
+ _last_call = time.time()
58
+ return None
59
+
60
+ results = data.get("data", [])
61
+ if not results:
62
+ return None
63
+
64
+ best = _pick_best(results, title)
65
+ if best is None:
66
+ return None
67
+ return _to_result(best)
68
+
69
+
70
+ def _pick_best(results: list[dict[str, Any]], query: str) -> dict[str, Any] | None:
71
+ """Return the best match from a list of results.
72
+
73
+ Tries exact title match first (case-insensitive), then falls back to first result.
74
+ """
75
+ query_lower = query.lower().strip()
76
+ for r in results:
77
+ candidate = (r.get("title") or "").lower().strip()
78
+ if candidate == query_lower:
79
+ return r
80
+ # Fall back to first result
81
+ return results[0]
82
+
83
+
84
+ def _to_result(raw: dict[str, Any]) -> ScholarResult | None:
85
+ """Convert a Semantic Scholar API response to a ScholarResult."""
86
+ title = _coerce_str(raw.get("title"))
87
+ if not title:
88
+ return None
89
+
90
+ authors_raw = raw.get("authors") or []
91
+ authors = [a.get("name", "") for a in authors_raw if a.get("name")]
92
+
93
+ abstract = _coerce_str(raw.get("abstract"))
94
+ year = _coerce_year(raw.get("year"))
95
+
96
+ # Prefer Semantic Scholar URL, fall back to external IDs
97
+ url = raw.get("url") or ""
98
+ external_ids = raw.get("externalIds") or {}
99
+ if not url:
100
+ url = external_ids.get("ArXiv", "") or external_ids.get("DOI", "")
101
+
102
+ return ScholarResult(
103
+ title=title,
104
+ authors=authors,
105
+ year=year,
106
+ abstract=abstract,
107
+ url=url or None,
108
+ )
109
+
110
+
111
+ def _coerce_str(value: Any) -> str:
112
+ if value is None:
113
+ return ""
114
+ return str(value).strip()
115
+
116
+
117
+ def _coerce_year(value: Any) -> int | None:
118
+ if value in (None, ""):
119
+ return None
120
+ try:
121
+ return int(value)
122
+ except (TypeError, ValueError):
123
+ return None
tests/test_cli.py ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Tests for CLI batch mode and retrieval backend switching."""
2
+ from pathlib import Path
3
+
4
+ from qalmsw.retrieval import search_by_title, set_backend
5
+ from qalmsw.retrieval.scholar import search_by_title as gs_search
6
+ from qalmsw.retrieval.semantic_scholar import search_by_title as ss_search
7
+
8
+
9
+ def test_default_backend_is_semantic_scholar():
10
+ """Module-level search_by_title should point to Semantic Scholar by default."""
11
+ assert search_by_title is ss_search
12
+
13
+
14
+ def test_set_backend_google_scholar():
15
+ """Switching to google-scholar patches the module-level function."""
16
+ import qalmsw.retrieval as mod
17
+ original = mod.search_by_title
18
+ set_backend("google-scholar")
19
+ try:
20
+ assert mod.search_by_title is gs_search
21
+ finally:
22
+ # Restore
23
+ set_backend("semantic-scholar")
24
+ assert mod.search_by_title is original
25
+
26
+
27
+ def test_set_backend_semantic_scholar():
28
+ """Switching back to semantic-scholar restores the original."""
29
+ import qalmsw.retrieval as mod
30
+ set_backend("google-scholar")
31
+ set_backend("semantic-scholar")
32
+ assert mod.search_by_title is ss_search
33
+
34
+
35
+ def test_set_backend_unknown_raises():
36
+ """Unknown backend name raises ValueError."""
37
+ import pytest
38
+ with pytest.raises(ValueError, match="Unknown retrieval backend"):
39
+ set_backend("bing")
40
+
41
+
42
+ def test_resolve_files_single(tmp_path: Path):
43
+ """Single existing .tex file is returned as-is."""
44
+ tex = tmp_path / "paper.tex"
45
+ tex.write_text(r"\documentclass{article}\begin{document}hello\end{document}")
46
+ from qalmsw.cli import _resolve_files
47
+ resolved = _resolve_files([tex])
48
+ assert len(resolved) == 1
49
+ assert resolved[0] == tex
50
+
51
+
52
+ def test_resolve_files_glob(tmp_path: Path):
53
+ """Glob pattern expands to matching .tex files."""
54
+ (tmp_path / "ch1.tex").write_text("chapter 1")
55
+ (tmp_path / "ch2.tex").write_text("chapter 2")
56
+ (tmp_path / "notes.txt").write_text("not a tex file")
57
+ from qalmsw.cli import _resolve_files
58
+ resolved = _resolve_files([tmp_path / "ch*.tex"])
59
+ assert len(resolved) == 2
60
+ names = {f.name for f in resolved}
61
+ assert names == {"ch1.tex", "ch2.tex"}
62
+
63
+
64
+ def test_resolve_files_no_match(tmp_path: Path):
65
+ """Glob with no matches returns empty list."""
66
+ from qalmsw.cli import _resolve_files
67
+ resolved = _resolve_files([tmp_path / "nonexistent*.tex"])
68
+ assert resolved == []
tests/test_images.py ADDED
@@ -0,0 +1,102 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Tests for the Image file existence checker."""
2
+ from pathlib import Path
3
+
4
+ from qalmsw.checkers import ImageChecker, Severity
5
+ from qalmsw.document import Document
6
+
7
+
8
+ def _doc(source: str, path: Path) -> Document:
9
+ return Document(path=path, source=source, paragraphs=[])
10
+
11
+
12
+ def test_image_exists(tmp_path: Path):
13
+ """An existing image file is silent."""
14
+ img = tmp_path / "results.png"
15
+ img.write_text("fake png")
16
+ tex = tmp_path / "paper.tex"
17
+ source = rf"\begin{{document}}\includegraphics{{{img.name}}}\end{{document}}"
18
+ tex.write_text(source)
19
+ findings = ImageChecker().check(_doc(source, tex))
20
+ assert findings == []
21
+
22
+
23
+ def test_image_missing_is_error(tmp_path: Path):
24
+ """A missing image file is an error."""
25
+ tex = tmp_path / "paper.tex"
26
+ source = r"\begin{document}\includegraphics{missing_plot.png}\end{document}"
27
+ tex.write_text(source)
28
+ findings = ImageChecker().check(_doc(source, tex))
29
+ errors = [f for f in findings if f.severity == Severity.error]
30
+ assert len(errors) >= 1
31
+ assert "missing_plot.png" in errors[0].message
32
+
33
+
34
+ def test_no_extension_checks_common_formats(tmp_path: Path):
35
+ """When no extension is given, try common image extensions."""
36
+ img = tmp_path / "results.pdf" # exists as .pdf
37
+ img.write_text("fake pdf")
38
+ tex = tmp_path / "paper.tex"
39
+ source = r"\begin{document}\includegraphics{results}\end{document}"
40
+ tex.write_text(source)
41
+ findings = ImageChecker().check(_doc(source, tex))
42
+ assert findings == []
43
+
44
+
45
+ def test_multiple_images_some_missing(tmp_path: Path):
46
+ """Only missing images are flagged."""
47
+ img = tmp_path / "good.png"
48
+ img.write_text("fake png")
49
+ tex = tmp_path / "paper.tex"
50
+ source = (
51
+ r"\begin{document}"
52
+ r"\includegraphics{good.png}"
53
+ r"\includegraphics{bad.jpg}"
54
+ r"\includegraphics{also_missing.png}"
55
+ r"\end{document}"
56
+ )
57
+ tex.write_text(source)
58
+ findings = ImageChecker().check(_doc(source, tex))
59
+ missing = [f for f in findings if "not found" in f.message]
60
+ assert len(missing) == 2
61
+ assert all(f.severity == Severity.error for f in missing)
62
+
63
+
64
+ def test_skips_non_image_extensions(tmp_path: Path):
65
+ """Don't flag .tex, .cls, .bib paths as missing images."""
66
+ tex = tmp_path / "paper.tex"
67
+ source = r"\begin{document}\includegraphics{appendix.tex}\end{document}"
68
+ tex.write_text(source)
69
+ findings = ImageChecker().check(_doc(source, tex))
70
+ assert findings == []
71
+
72
+
73
+ def test_empty_graphics_path_is_ignored(tmp_path: Path):
74
+ """Empty {} inside `includegraphics` is skipped."""
75
+ tex = tmp_path / "paper.tex"
76
+ source = r"\begin{document}\includegraphics{}\end{document}"
77
+ tex.write_text(source)
78
+ findings = ImageChecker().check(_doc(source, tex))
79
+ assert findings == []
80
+
81
+
82
+ def test_path_with_subdirectory(tmp_path: Path):
83
+ """Images in subdirectories relative to the .tex file."""
84
+ figures = tmp_path / "figures"
85
+ figures.mkdir()
86
+ img = figures / "arch.pdf"
87
+ img.write_text("fake pdf")
88
+ tex = tmp_path / "paper.tex"
89
+ source = r"\begin{document}\includegraphics{figures/arch.pdf}\end{document}"
90
+ tex.write_text(source)
91
+ findings = ImageChecker().check(_doc(source, tex))
92
+ assert findings == []
93
+
94
+
95
+ def test_path_with_subdirectory_missing(tmp_path: Path):
96
+ """Missing image in subdirectory is flagged."""
97
+ tex = tmp_path / "paper.tex"
98
+ source = r"\begin{document}\includegraphics{figures/missing.pdf}\end{document}"
99
+ tex.write_text(source)
100
+ findings = ImageChecker().check(_doc(source, tex))
101
+ errors = [f for f in findings if f.severity == Severity.error]
102
+ assert len(errors) == 1
tests/test_scholar.py DELETED
@@ -1,70 +0,0 @@
1
- from unittest.mock import patch
2
-
3
- from qalmsw.retrieval import ScholarResult, search_by_title
4
-
5
-
6
- def _fake_pub(**bib_overrides) -> dict:
7
- bib = {
8
- "title": "Attention Is All You Need",
9
- "author": ["Ashish Vaswani", "Noam Shazeer"],
10
- "pub_year": "2017",
11
- "abstract": "We propose a new architecture, the Transformer.",
12
- }
13
- bib.update(bib_overrides)
14
- return {"bib": bib, "pub_url": "https://arxiv.org/abs/1706.03762"}
15
-
16
-
17
- def test_search_by_title_returns_first_match():
18
- with patch(
19
- "qalmsw.retrieval.scholar.scholarly.search_pubs", return_value=iter([_fake_pub()])
20
- ):
21
- result = search_by_title("attention transformer")
22
- assert isinstance(result, ScholarResult)
23
- assert result.title == "Attention Is All You Need"
24
- assert result.authors == ["Ashish Vaswani", "Noam Shazeer"]
25
- assert result.year == 2017
26
- assert "Transformer" in result.abstract
27
- assert result.url == "https://arxiv.org/abs/1706.03762"
28
-
29
-
30
- def test_search_by_title_returns_none_when_no_results():
31
- with patch("qalmsw.retrieval.scholar.scholarly.search_pubs", return_value=iter([])):
32
- assert search_by_title("no such paper xyzzy") is None
33
-
34
-
35
- def test_author_string_is_split_on_and():
36
- raw = _fake_pub(author="Ashish Vaswani and Noam Shazeer and Niki Parmar")
37
- with patch("qalmsw.retrieval.scholar.scholarly.search_pubs", return_value=iter([raw])):
38
- result = search_by_title("x")
39
- assert result is not None
40
- assert result.authors == ["Ashish Vaswani", "Noam Shazeer", "Niki Parmar"]
41
-
42
-
43
- def test_missing_fields_produce_safe_defaults():
44
- raw = {"bib": {}}
45
- with patch("qalmsw.retrieval.scholar.scholarly.search_pubs", return_value=iter([raw])):
46
- result = search_by_title("x")
47
- assert result is not None
48
- assert result.title == ""
49
- assert result.authors == []
50
- assert result.year is None
51
- assert result.abstract == ""
52
- assert result.url is None
53
-
54
-
55
- def test_non_numeric_year_coerces_to_none():
56
- raw = _fake_pub(pub_year="in press")
57
- with patch("qalmsw.retrieval.scholar.scholarly.search_pubs", return_value=iter([raw])):
58
- result = search_by_title("x")
59
- assert result is not None
60
- assert result.year is None
61
-
62
-
63
- def test_eprint_url_used_when_pub_url_missing():
64
- raw = _fake_pub()
65
- raw.pop("pub_url")
66
- raw["eprint_url"] = "https://example.org/eprint.pdf"
67
- with patch("qalmsw.retrieval.scholar.scholarly.search_pubs", return_value=iter([raw])):
68
- result = search_by_title("x")
69
- assert result is not None
70
- assert result.url == "https://example.org/eprint.pdf"
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
tests/test_semantic_scholar.py ADDED
@@ -0,0 +1,101 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Tests for Semantic Scholar retrieval module."""
2
+ from unittest.mock import patch
3
+
4
+ from qalmsw.retrieval import ScholarResult, search_by_title
5
+
6
+
7
+ def _mock_response(data: list[dict]) -> dict:
8
+ return {"data": data}
9
+
10
+
11
+ def _paper(
12
+ title: str = "Attention Is All You Need",
13
+ authors: list[str] | None = None,
14
+ year: int | None = 2017,
15
+ abstract: str = "We propose the Transformer architecture.",
16
+ arxiv_id: str = "1706.03762",
17
+ url: str = "https://api.semanticscholar.org/CorpusID:1",
18
+ ) -> dict:
19
+ return {
20
+ "title": title,
21
+ "authors": [{"name": a} for a in (authors or ["Ashish Vaswani", "Noam Shazeer"])],
22
+ "year": year,
23
+ "abstract": abstract,
24
+ "externalIds": {"ArXiv": arxiv_id} if arxiv_id else {},
25
+ "url": url,
26
+ }
27
+
28
+
29
+ def _mock_urlopen(json_data: dict):
30
+ """Return a mock urlopen context manager."""
31
+ import json as _json
32
+ from unittest.mock import MagicMock
33
+
34
+ cm = MagicMock()
35
+ cm.__enter__.return_value.read.return_value = _json.dumps(json_data).encode("utf-8")
36
+ return cm
37
+
38
+
39
+ def test_search_by_title_returns_first_match():
40
+ data = _mock_response([_paper()])
41
+ with patch("qalmsw.retrieval.semantic_scholar.urllib.request.urlopen",
42
+ return_value=_mock_urlopen(data)):
43
+ result = search_by_title("attention transformer")
44
+ assert isinstance(result, ScholarResult)
45
+ assert result.title == "Attention Is All You Need"
46
+ assert result.authors == ["Ashish Vaswani", "Noam Shazeer"]
47
+ assert result.year == 2017
48
+ assert "Transformer" in result.abstract
49
+ assert "semanticscholar" in (result.url or "")
50
+
51
+
52
+ def test_search_by_title_returns_none_when_no_results():
53
+ data = _mock_response([])
54
+ with patch("qalmsw.retrieval.semantic_scholar.urllib.request.urlopen",
55
+ return_value=_mock_urlopen(data)):
56
+ assert search_by_title("no such paper xyzzy") is None
57
+
58
+
59
+ def test_search_by_title_returns_none_on_http_error():
60
+ import urllib.error
61
+
62
+ def _raise(*args, **kwargs):
63
+ raise urllib.error.HTTPError(
64
+ "http://example.com", 429, "Too Many", {}, None
65
+ )
66
+
67
+ with patch("qalmsw.retrieval.semantic_scholar.urllib.request.urlopen", _raise):
68
+ assert search_by_title("any title") is None
69
+
70
+
71
+ def test_missing_fields_produce_safe_defaults():
72
+ data = _mock_response([
73
+ _paper(
74
+ title="", authors=None, year=None,
75
+ abstract="", arxiv_id="", url="",
76
+ )
77
+ ])
78
+ with patch("qalmsw.retrieval.semantic_scholar.urllib.request.urlopen",
79
+ return_value=_mock_urlopen(data)):
80
+ result = search_by_title("x")
81
+ assert result is None # empty title → None
82
+
83
+
84
+ def test_empty_title_returns_none():
85
+ data = _mock_response([_paper(title="", authors=None)])
86
+ with patch("qalmsw.retrieval.semantic_scholar.urllib.request.urlopen",
87
+ return_value=_mock_urlopen(data)):
88
+ assert search_by_title("x") is None
89
+
90
+
91
+ def test_exact_title_match_is_preferred():
92
+ data = _mock_response([
93
+ _paper(title="Some Related Work", year=2020),
94
+ _paper(title="Attention Is All You Need", year=2017),
95
+ ])
96
+ with patch("qalmsw.retrieval.semantic_scholar.urllib.request.urlopen",
97
+ return_value=_mock_urlopen(data)):
98
+ result = search_by_title("Attention Is All You Need")
99
+ assert result is not None
100
+ assert result.title == "Attention Is All You Need"
101
+ assert result.year == 2017