VibecoderMcSwaggins commited on
Commit
1cd600b
Β·
1 Parent(s): 68ead43

docs: add SPEC_03 for OpenAlex integration (TDD plan)

Browse files

Ironclad spec for adding OpenAlex as 4th data source:

Why OpenAlex:
- Citation counts (prioritize authoritative papers)
- Concept tagging (hierarchical classification)
- Open access links (direct PDF URLs)
- FREE API, 209M+ works indexed

Architecture:
- Implements SearchTool Protocol (DIP)
- Returns Evidence with rich metadata
- Follows existing tool patterns (SOLID/DRY)

TDD Plan:
- 10 unit tests covering all functionality
- Integration test with real API
- Tests written before implementation

Groundwork ready:
- SourceName already includes "openalex"
- Evidence.metadata designed for citation data

Closes #51 when implemented.

docs/specs/SPEC_03_OPENALEX_INTEGRATION.md ADDED
@@ -0,0 +1,638 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SPEC 03: OpenAlex Integration
2
+
3
+ ## Priority: P1 (Feature Enhancement)
4
+
5
+ ## Problem Statement
6
+
7
+ We search 3 sources (PubMed, Europe PMC, ClinicalTrials.gov) but have **no citation metrics**. We can't distinguish a highly-cited landmark paper from an obscure one. OpenAlex provides:
8
+
9
+ 1. **Citation counts** - Prioritize authoritative papers
10
+ 2. **Citation networks** - "Who cites whom"
11
+ 3. **Concept tagging** - Hierarchical categorization
12
+ 4. **Related works** - ML-powered similarity
13
+ 5. **Open access links** - Direct PDF URLs
14
+
15
+ **FREE API. No key required. 209M+ works indexed.**
16
+
17
+ ## Groundwork Already Done
18
+
19
+ ```python
20
+ # src/utils/models.py:9
21
+ SourceName = Literal["pubmed", "clinicaltrials", "europepmc", "preprint", "openalex", "web"]
22
+
23
+ # src/utils/models.py:39-42
24
+ metadata: dict[str, Any] = Field(
25
+ default_factory=dict,
26
+ description="Additional metadata (e.g., cited_by_count, concepts, is_open_access)",
27
+ )
28
+ ```
29
+
30
+ The infrastructure is ready. We just need to build the tool.
31
+
32
+ ## OpenAlex API Reference
33
+
34
+ ### Endpoint
35
+
36
+ ```
37
+ GET https://api.openalex.org/works
38
+ ```
39
+
40
+ ### Key Parameters
41
+
42
+ | Parameter | Description |
43
+ |-----------|-------------|
44
+ | `search` | Full-text search across title, abstract, fulltext |
45
+ | `filter` | Constrain results (e.g., `type:article`) |
46
+ | `sort` | Order results (e.g., `cited_by_count:desc`) |
47
+ | `per_page` | Results per page (max 200) |
48
+ | `mailto` | Email for polite pool (higher rate limits) |
49
+
50
+ ### Example Request
51
+
52
+ ```bash
53
+ GET https://api.openalex.org/works?search=metformin%20cancer&filter=type:article&sort=cited_by_count:desc&per_page=10&mailto=deepboner@example.com
54
+ ```
55
+
56
+ ### Response Structure
57
+
58
+ ```json
59
+ {
60
+ "results": [
61
+ {
62
+ "id": "https://openalex.org/W2741809807",
63
+ "doi": "https://doi.org/10.1234/example",
64
+ "display_name": "Paper Title",
65
+ "publication_year": 2024,
66
+ "cited_by_count": 150,
67
+ "abstract_inverted_index": {
68
+ "word1": [0],
69
+ "word2": [1, 5]
70
+ },
71
+ "concepts": [
72
+ {"display_name": "Metformin", "score": 0.95, "level": 2}
73
+ ],
74
+ "authorships": [
75
+ {"author": {"display_name": "John Smith"}}
76
+ ],
77
+ "open_access": {
78
+ "is_oa": true,
79
+ "oa_url": "https://example.com/pdf"
80
+ },
81
+ "best_oa_location": {
82
+ "pdf_url": "https://example.com/paper.pdf"
83
+ }
84
+ }
85
+ ]
86
+ }
87
+ ```
88
+
89
+ ### Abstract Reconstruction
90
+
91
+ OpenAlex stores abstracts as inverted index for compression. Must reconstruct:
92
+
93
+ ```python
94
+ def _reconstruct_abstract(self, inverted_index: dict[str, list[int]] | None) -> str:
95
+ """Rebuild abstract from {"word": [positions]} format."""
96
+ if not inverted_index:
97
+ return ""
98
+
99
+ position_word: dict[int, str] = {}
100
+ for word, positions in inverted_index.items():
101
+ for pos in positions:
102
+ position_word[pos] = word
103
+
104
+ if not position_word:
105
+ return ""
106
+
107
+ max_pos = max(position_word.keys())
108
+ return " ".join(position_word.get(i, "") for i in range(max_pos + 1))
109
+ ```
110
+
111
+ ## Architecture
112
+
113
+ ### Class Diagram
114
+
115
+ ```
116
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
117
+ β”‚ SearchTool (Protocol) β”‚
118
+ β”‚ ───────────────────────────────── β”‚
119
+ β”‚ + name: str β”‚
120
+ β”‚ + search(query, max_results) β†’ list[Evidence] β”‚
121
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
122
+ β”‚ implements
123
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
124
+ β”‚ OpenAlexTool β”‚
125
+ β”‚ ───────────────────────────────── β”‚
126
+ β”‚ - BASE_URL: str β”‚
127
+ β”‚ - _polite_email: str β”‚
128
+ β”‚ ───────────────────────────────── β”‚
129
+ β”‚ + name β†’ "openalex" β”‚
130
+ β”‚ + search(query, max_results) β†’ list[Evidence] β”‚
131
+ β”‚ - _reconstruct_abstract(inverted_index) β†’ str β”‚
132
+ β”‚ - _to_evidence(work) β†’ Evidence β”‚
133
+ β”‚ - _extract_authors(authorships) β†’ list[str] β”‚
134
+ β”‚ - _extract_concepts(concepts) β†’ list[str] β”‚
135
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
136
+ ```
137
+
138
+ ### Integration Flow
139
+
140
+ ```
141
+ SearchHandler
142
+ β”‚
143
+ β”œβ”€β”€ PubMedTool
144
+ β”œβ”€β”€ EuropePMCTool
145
+ β”œβ”€β”€ ClinicalTrialsTool
146
+ └── OpenAlexTool ◄── NEW
147
+ β”‚
148
+ β–Ό
149
+ Evidence(
150
+ content=abstract[:2000],
151
+ citation=Citation(source="openalex", ...),
152
+ metadata={
153
+ "cited_by_count": 150,
154
+ "concepts": ["Metformin", "Cancer"],
155
+ "is_open_access": True,
156
+ "pdf_url": "https://..."
157
+ }
158
+ )
159
+ ```
160
+
161
+ ## TDD Implementation Plan
162
+
163
+ ### Red Phase: Write Failing Tests First
164
+
165
+ **File: `tests/unit/tools/test_openalex.py`**
166
+
167
+ ```python
168
+ """Unit tests for OpenAlex tool - TDD RED phase."""
169
+
170
+ from unittest.mock import AsyncMock, MagicMock, patch
171
+
172
+ import pytest
173
+
174
+ from src.tools.openalex import OpenAlexTool
175
+ from src.utils.models import Evidence
176
+
177
+
178
+ # Sample OpenAlex response
179
+ SAMPLE_OPENALEX_RESPONSE = {
180
+ "results": [
181
+ {
182
+ "id": "https://openalex.org/W12345",
183
+ "doi": "https://doi.org/10.1234/test",
184
+ "display_name": "Metformin in Cancer Treatment",
185
+ "publication_year": 2024,
186
+ "cited_by_count": 150,
187
+ "abstract_inverted_index": {
188
+ "Metformin": [0],
189
+ "shows": [1],
190
+ "promise": [2],
191
+ "in": [3],
192
+ "cancer": [4],
193
+ "treatment": [5],
194
+ },
195
+ "concepts": [
196
+ {"display_name": "Metformin", "score": 0.95, "level": 2},
197
+ {"display_name": "Cancer", "score": 0.88, "level": 1},
198
+ ],
199
+ "authorships": [
200
+ {"author": {"display_name": "John Smith"}},
201
+ {"author": {"display_name": "Jane Doe"}},
202
+ ],
203
+ "open_access": {"is_oa": True, "oa_url": "https://example.com/oa"},
204
+ "best_oa_location": {"pdf_url": "https://example.com/paper.pdf"},
205
+ }
206
+ ]
207
+ }
208
+
209
+
210
+ @pytest.mark.unit
211
+ class TestOpenAlexTool:
212
+ """Tests for OpenAlexTool."""
213
+
214
+ @pytest.fixture
215
+ def tool(self) -> OpenAlexTool:
216
+ return OpenAlexTool()
217
+
218
+ def test_tool_name(self, tool: OpenAlexTool) -> None:
219
+ """Tool name should be 'openalex'."""
220
+ assert tool.name == "openalex"
221
+
222
+ @pytest.mark.asyncio
223
+ async def test_search_returns_evidence(self, tool: OpenAlexTool) -> None:
224
+ """Search should return Evidence objects."""
225
+ with patch("httpx.AsyncClient") as mock_client:
226
+ mock_instance = AsyncMock()
227
+ mock_client.return_value.__aenter__.return_value = mock_instance
228
+
229
+ mock_resp = MagicMock()
230
+ mock_resp.json.return_value = SAMPLE_OPENALEX_RESPONSE
231
+ mock_resp.raise_for_status.return_value = None
232
+ mock_instance.get.return_value = mock_resp
233
+
234
+ results = await tool.search("metformin cancer", max_results=5)
235
+
236
+ assert len(results) == 1
237
+ assert isinstance(results[0], Evidence)
238
+ assert results[0].citation.source == "openalex"
239
+
240
+ @pytest.mark.asyncio
241
+ async def test_search_includes_citation_count(self, tool: OpenAlexTool) -> None:
242
+ """Evidence metadata should include cited_by_count."""
243
+ with patch("httpx.AsyncClient") as mock_client:
244
+ mock_instance = AsyncMock()
245
+ mock_client.return_value.__aenter__.return_value = mock_instance
246
+
247
+ mock_resp = MagicMock()
248
+ mock_resp.json.return_value = SAMPLE_OPENALEX_RESPONSE
249
+ mock_resp.raise_for_status.return_value = None
250
+ mock_instance.get.return_value = mock_resp
251
+
252
+ results = await tool.search("metformin cancer", max_results=5)
253
+
254
+ assert results[0].metadata["cited_by_count"] == 150
255
+
256
+ @pytest.mark.asyncio
257
+ async def test_search_includes_concepts(self, tool: OpenAlexTool) -> None:
258
+ """Evidence metadata should include concepts."""
259
+ with patch("httpx.AsyncClient") as mock_client:
260
+ mock_instance = AsyncMock()
261
+ mock_client.return_value.__aenter__.return_value = mock_instance
262
+
263
+ mock_resp = MagicMock()
264
+ mock_resp.json.return_value = SAMPLE_OPENALEX_RESPONSE
265
+ mock_resp.raise_for_status.return_value = None
266
+ mock_instance.get.return_value = mock_resp
267
+
268
+ results = await tool.search("metformin cancer", max_results=5)
269
+
270
+ assert "Metformin" in results[0].metadata["concepts"]
271
+ assert "Cancer" in results[0].metadata["concepts"]
272
+
273
+ @pytest.mark.asyncio
274
+ async def test_search_includes_open_access_info(self, tool: OpenAlexTool) -> None:
275
+ """Evidence metadata should include open access info."""
276
+ with patch("httpx.AsyncClient") as mock_client:
277
+ mock_instance = AsyncMock()
278
+ mock_client.return_value.__aenter__.return_value = mock_instance
279
+
280
+ mock_resp = MagicMock()
281
+ mock_resp.json.return_value = SAMPLE_OPENALEX_RESPONSE
282
+ mock_resp.raise_for_status.return_value = None
283
+ mock_instance.get.return_value = mock_resp
284
+
285
+ results = await tool.search("metformin cancer", max_results=5)
286
+
287
+ assert results[0].metadata["is_open_access"] is True
288
+ assert results[0].metadata["pdf_url"] == "https://example.com/paper.pdf"
289
+
290
+ def test_reconstruct_abstract(self, tool: OpenAlexTool) -> None:
291
+ """Abstract reconstruction from inverted index."""
292
+ inverted_index = {
293
+ "Hello": [0],
294
+ "world": [1],
295
+ "this": [2],
296
+ "is": [3],
297
+ "a": [4],
298
+ "test": [5],
299
+ }
300
+
301
+ result = tool._reconstruct_abstract(inverted_index)
302
+
303
+ assert result == "Hello world this is a test"
304
+
305
+ def test_reconstruct_abstract_empty(self, tool: OpenAlexTool) -> None:
306
+ """Handle None/empty inverted index."""
307
+ assert tool._reconstruct_abstract(None) == ""
308
+ assert tool._reconstruct_abstract({}) == ""
309
+
310
+ @pytest.mark.asyncio
311
+ async def test_search_empty_results(self, tool: OpenAlexTool) -> None:
312
+ """Handle empty results gracefully."""
313
+ with patch("httpx.AsyncClient") as mock_client:
314
+ mock_instance = AsyncMock()
315
+ mock_client.return_value.__aenter__.return_value = mock_instance
316
+
317
+ mock_resp = MagicMock()
318
+ mock_resp.json.return_value = {"results": []}
319
+ mock_resp.raise_for_status.return_value = None
320
+ mock_instance.get.return_value = mock_resp
321
+
322
+ results = await tool.search("xyznonexistent123", max_results=5)
323
+
324
+ assert results == []
325
+
326
+ @pytest.mark.asyncio
327
+ async def test_search_sorts_by_citations(self, tool: OpenAlexTool) -> None:
328
+ """Verify API call requests citation-sorted results."""
329
+ with patch("httpx.AsyncClient") as mock_client:
330
+ mock_instance = AsyncMock()
331
+ mock_client.return_value.__aenter__.return_value = mock_instance
332
+
333
+ mock_resp = MagicMock()
334
+ mock_resp.json.return_value = {"results": []}
335
+ mock_resp.raise_for_status.return_value = None
336
+ mock_instance.get.return_value = mock_resp
337
+
338
+ await tool.search("test query", max_results=5)
339
+
340
+ # Verify call params
341
+ call_args = mock_instance.get.call_args
342
+ params = call_args[1]["params"]
343
+ assert params["sort"] == "cited_by_count:desc"
344
+
345
+ @pytest.mark.asyncio
346
+ async def test_search_uses_polite_pool(self, tool: OpenAlexTool) -> None:
347
+ """Verify mailto parameter for polite pool."""
348
+ with patch("httpx.AsyncClient") as mock_client:
349
+ mock_instance = AsyncMock()
350
+ mock_client.return_value.__aenter__.return_value = mock_instance
351
+
352
+ mock_resp = MagicMock()
353
+ mock_resp.json.return_value = {"results": []}
354
+ mock_resp.raise_for_status.return_value = None
355
+ mock_instance.get.return_value = mock_resp
356
+
357
+ await tool.search("test query", max_results=5)
358
+
359
+ call_args = mock_instance.get.call_args
360
+ params = call_args[1]["params"]
361
+ assert "mailto" in params
362
+ ```
363
+
364
+ ### Green Phase: Implement to Pass Tests
365
+
366
+ **File: `src/tools/openalex.py`**
367
+
368
+ ```python
369
+ """OpenAlex search tool - citation-aware scholarly search."""
370
+
371
+ from typing import Any
372
+
373
+ import httpx
374
+ from tenacity import retry, stop_after_attempt, wait_exponential
375
+
376
+ from src.utils.exceptions import SearchError
377
+ from src.utils.models import Citation, Evidence
378
+
379
+
380
+ class OpenAlexTool:
381
+ """
382
+ Search OpenAlex for scholarly works with citation metrics.
383
+
384
+ OpenAlex indexes 209M+ works and provides:
385
+ - Citation counts (prioritize influential papers)
386
+ - Concept tagging (hierarchical classification)
387
+ - Open access links (direct PDF URLs)
388
+ - Related works (ML-powered similarity)
389
+
390
+ API Docs: https://docs.openalex.org
391
+ Rate Limits: Polite pool with mailto = 100k/day
392
+ """
393
+
394
+ BASE_URL = "https://api.openalex.org/works"
395
+ POLITE_EMAIL = "deepboner-research@proton.me"
396
+
397
+ @property
398
+ def name(self) -> str:
399
+ return "openalex"
400
+
401
+ @retry(
402
+ stop=stop_after_attempt(3),
403
+ wait=wait_exponential(multiplier=1, min=1, max=10),
404
+ reraise=True,
405
+ )
406
+ async def search(self, query: str, max_results: int = 10) -> list[Evidence]:
407
+ """
408
+ Search OpenAlex, sorted by citation count.
409
+
410
+ Args:
411
+ query: Search terms
412
+ max_results: Maximum results to return
413
+
414
+ Returns:
415
+ List of Evidence objects with citation metadata
416
+ """
417
+ params: dict[str, str | int] = {
418
+ "search": query,
419
+ "filter": "type:article", # Only peer-reviewed articles
420
+ "sort": "cited_by_count:desc", # Most cited first
421
+ "per_page": min(max_results, 100),
422
+ "mailto": self.POLITE_EMAIL,
423
+ }
424
+
425
+ async with httpx.AsyncClient(timeout=30.0) as client:
426
+ try:
427
+ response = await client.get(self.BASE_URL, params=params)
428
+ response.raise_for_status()
429
+
430
+ data = response.json()
431
+ works = data.get("results", [])
432
+
433
+ return [self._to_evidence(work) for work in works[:max_results]]
434
+
435
+ except httpx.HTTPStatusError as e:
436
+ raise SearchError(f"OpenAlex API error: {e}") from e
437
+ except httpx.RequestError as e:
438
+ raise SearchError(f"OpenAlex connection failed: {e}") from e
439
+
440
+ def _to_evidence(self, work: dict[str, Any]) -> Evidence:
441
+ """Convert OpenAlex work to Evidence with rich metadata."""
442
+ # Extract basic fields
443
+ title = work.get("display_name", "Untitled")
444
+ doi = work.get("doi", "")
445
+ year = work.get("publication_year", "Unknown")
446
+ cited_by_count = work.get("cited_by_count", 0)
447
+
448
+ # Reconstruct abstract from inverted index
449
+ abstract = self._reconstruct_abstract(work.get("abstract_inverted_index"))
450
+ if not abstract:
451
+ abstract = f"[No abstract available. Cited by {cited_by_count} works.]"
452
+
453
+ # Extract authors (limit to 5)
454
+ authors = self._extract_authors(work.get("authorships", []))
455
+
456
+ # Extract concepts (top 5 by score)
457
+ concepts = self._extract_concepts(work.get("concepts", []))
458
+
459
+ # Open access info
460
+ oa_info = work.get("open_access", {})
461
+ is_oa = oa_info.get("is_oa", False)
462
+
463
+ # Get PDF URL (prefer best_oa_location)
464
+ best_oa = work.get("best_oa_location", {})
465
+ pdf_url = best_oa.get("pdf_url") if best_oa else None
466
+
467
+ # Build URL
468
+ if doi:
469
+ url = doi if doi.startswith("http") else f"https://doi.org/{doi}"
470
+ else:
471
+ openalex_id = work.get("id", "")
472
+ url = openalex_id if openalex_id else "https://openalex.org"
473
+
474
+ # Prepend citation badge to content
475
+ citation_badge = f"[Cited by {cited_by_count}] " if cited_by_count > 0 else ""
476
+ content = f"{citation_badge}{abstract[:1900]}"
477
+
478
+ return Evidence(
479
+ content=content[:2000],
480
+ citation=Citation(
481
+ source="openalex",
482
+ title=title[:500],
483
+ url=url,
484
+ date=str(year),
485
+ authors=authors,
486
+ ),
487
+ relevance=min(1.0, cited_by_count / 1000), # Normalize to 0-1
488
+ metadata={
489
+ "cited_by_count": cited_by_count,
490
+ "concepts": concepts,
491
+ "is_open_access": is_oa,
492
+ "pdf_url": pdf_url,
493
+ },
494
+ )
495
+
496
+ def _reconstruct_abstract(self, inverted_index: dict[str, list[int]] | None) -> str:
497
+ """Rebuild abstract from {"word": [positions]} format."""
498
+ if not inverted_index:
499
+ return ""
500
+
501
+ position_word: dict[int, str] = {}
502
+ for word, positions in inverted_index.items():
503
+ for pos in positions:
504
+ position_word[pos] = word
505
+
506
+ if not position_word:
507
+ return ""
508
+
509
+ max_pos = max(position_word.keys())
510
+ return " ".join(position_word.get(i, "") for i in range(max_pos + 1))
511
+
512
+ def _extract_authors(self, authorships: list[dict[str, Any]]) -> list[str]:
513
+ """Extract author names from authorships array."""
514
+ authors = []
515
+ for authorship in authorships[:5]:
516
+ author = authorship.get("author", {})
517
+ name = author.get("display_name")
518
+ if name:
519
+ authors.append(name)
520
+ return authors
521
+
522
+ def _extract_concepts(self, concepts: list[dict[str, Any]]) -> list[str]:
523
+ """Extract concept names, sorted by score."""
524
+ sorted_concepts = sorted(concepts, key=lambda c: c.get("score", 0), reverse=True)
525
+ return [c.get("display_name", "") for c in sorted_concepts[:5] if c.get("display_name")]
526
+ ```
527
+
528
+ ### Refactor Phase: Clean Integration
529
+
530
+ **Update: `src/tools/__init__.py`**
531
+
532
+ ```python
533
+ """Search tools package."""
534
+
535
+ from src.tools.base import SearchTool
536
+ from src.tools.clinicaltrials import ClinicalTrialsTool
537
+ from src.tools.europepmc import EuropePMCTool
538
+ from src.tools.openalex import OpenAlexTool
539
+ from src.tools.pubmed import PubMedTool
540
+ from src.tools.search_handler import SearchHandler
541
+
542
+ __all__ = [
543
+ "ClinicalTrialsTool",
544
+ "EuropePMCTool",
545
+ "OpenAlexTool",
546
+ "PubMedTool",
547
+ "SearchHandler",
548
+ "SearchTool",
549
+ ]
550
+ ```
551
+
552
+ ## Test Matrix
553
+
554
+ | Test | What It Validates | Priority |
555
+ |------|------------------|----------|
556
+ | `test_tool_name` | Returns "openalex" | P0 |
557
+ | `test_search_returns_evidence` | Returns `list[Evidence]` | P0 |
558
+ | `test_search_includes_citation_count` | `metadata["cited_by_count"]` populated | P0 |
559
+ | `test_search_includes_concepts` | `metadata["concepts"]` populated | P0 |
560
+ | `test_search_includes_open_access_info` | `metadata["is_open_access"]`, `pdf_url` | P1 |
561
+ | `test_reconstruct_abstract` | Inverted index β†’ text | P0 |
562
+ | `test_reconstruct_abstract_empty` | Handle None/empty | P0 |
563
+ | `test_search_empty_results` | Return `[]` for no matches | P0 |
564
+ | `test_search_sorts_by_citations` | API param `sort=cited_by_count:desc` | P0 |
565
+ | `test_search_uses_polite_pool` | API param `mailto` present | P1 |
566
+
567
+ ## Integration Test
568
+
569
+ ```python
570
+ @pytest.mark.integration
571
+ class TestOpenAlexIntegration:
572
+ """Integration tests with real OpenAlex API."""
573
+
574
+ @pytest.mark.asyncio
575
+ async def test_real_api_returns_results(self) -> None:
576
+ """Test actual API returns relevant results."""
577
+ tool = OpenAlexTool()
578
+ results = await tool.search("metformin cancer treatment", max_results=3)
579
+
580
+ assert len(results) > 0
581
+ # Should have citation counts
582
+ assert results[0].metadata["cited_by_count"] >= 0
583
+ # Should have concepts
584
+ assert len(results[0].metadata["concepts"]) > 0
585
+ ```
586
+
587
+ ## Acceptance Criteria
588
+
589
+ - [ ] `OpenAlexTool` implements `SearchTool` Protocol
590
+ - [ ] `search()` returns `list[Evidence]` sorted by citation count
591
+ - [ ] `metadata["cited_by_count"]` populated for each result
592
+ - [ ] `metadata["concepts"]` contains top 5 concept names
593
+ - [ ] `metadata["is_open_access"]` and `pdf_url` populated
594
+ - [ ] Abstract correctly reconstructed from inverted index
595
+ - [ ] Empty results handled gracefully
596
+ - [ ] Tool exported from `src/tools/__init__.py`
597
+ - [ ] All unit tests pass with mocked API
598
+ - [ ] Integration test passes with real API
599
+ - [ ] `make check` passes (lint + type + test)
600
+
601
+ ## Files to Create/Modify
602
+
603
+ ### Create
604
+
605
+ 1. `src/tools/openalex.py` - Main tool class
606
+ 2. `tests/unit/tools/test_openalex.py` - Unit tests
607
+
608
+ ### Modify
609
+
610
+ 1. `src/tools/__init__.py` - Add export
611
+ 2. `src/orchestrator.py` - Add to tool list (if using default tools)
612
+
613
+ ## SOLID Principles Applied
614
+
615
+ | Principle | Application |
616
+ |-----------|------------|
617
+ | **S**ingle Responsibility | `OpenAlexTool` only handles OpenAlex API |
618
+ | **O**pen/Closed | New tool without modifying existing ones |
619
+ | **L**iskov Substitution | Implements `SearchTool` Protocol, interchangeable |
620
+ | **I**nterface Segregation | Uses minimal `SearchTool` interface |
621
+ | **D**ependency Inversion | `SearchHandler` depends on Protocol, not concrete class |
622
+
623
+ ## DRY Applied
624
+
625
+ - Reuses `Evidence`, `Citation`, `SearchError` from `src/utils/models.py`
626
+ - Follows identical patterns as `PubMedTool`, `EuropePMCTool`
627
+ - Same retry decorator pattern as other tools
628
+
629
+ ## Related Issues
630
+
631
+ - #51: feat: Add OpenAlex as 4th data source
632
+
633
+ ## Effort Estimate
634
+
635
+ - Tests (TDD Red): 30 min
636
+ - Implementation (Green): 1 hour
637
+ - Refactor + Integration: 30 min
638
+ - **Total: ~2 hours**