Deploy URAAS — African Research Archival & Analytics System

docs/PARTNER_API.md

@@ -0,0 +1,174 @@
+# URAAS Partner API
+
+Server-to-server read access to URAAS (University of Lagos Academic Archive &
+Special Collections) data — built for external partners like the Africa PID
+Alliance / DOCiD to integrate against, without needing a browser session.
+
+This is separate from the dashboard's human login (username/password +
+session cookie). Partner access uses a long-lived API key instead.
+
+---
+
+## Base URL
+
+```
+https://lordkiki-apa-uraas.hf.space
+```
+
+## Authentication
+
+Every request must include the key in a header:
+
+```
+X-API-Key: uraas_live_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+```
+
+- Missing or invalid key → `401 Unauthorized`
+- Valid key used against an endpoint outside this document → `403 Forbidden`
+  (partner keys are read-only by construction — no key can reach crawler
+  controls, admin exports, or anything that mutates data, regardless of what
+  path is requested)
+- Keys are issued manually by a URAAS admin. To request one, contact the
+  URAAS team directly — there is no self-service signup.
+
+## Rate limits
+
+**120 requests per rolling 60-second window, per key.** Exceeding it returns
+`429 Too Many Requests`. Contact us if your integration needs a higher limit.
+
+## Errors
+
+All errors are JSON:
+
+```json
+{ "status": "error", "message": "..." }
+```
+
+| Status | Meaning |
+|---|---|
+| `401` | Missing or invalid/revoked API key |
+| `403` | Valid key, but this endpoint isn't in the partner allowlist |
+| `429` | Rate limit exceeded |
+| `404` | Resource not found (e.g. unknown paper id) |
+| `500` | Server error — please report it to us |
+
+---
+
+## Endpoints
+
+### `GET /api/stats`
+Network/collaboration summary: co-authorship edges between departments, and
+top authors by paper count.
+
+```json
+{
+  "status": "success",
+  "network_edges": [{ "source": "Botany", "target": "Chemistry", "weight": 1 }],
+  "top_authors": [{ "author": "...", "count": 2, "orcid": "", "ror": "" }]
+}
+```
+
+### `GET /api/papers/tree`
+Every paper, grouped by faculty then department.
+
+```json
+{
+  "data": {
+    "College of Medicine": {
+      "Community Health and Primary Care": [
+        {
+          "id": 22,
+          "title": "...",
+          "doi": "10.1093/hmg/ddw104",
+          "url": "https://doi.org/10.1093/hmg/ddw104",
+          "docid": "",
+          "download_url": null,
+          "has_local_pdf": false,
+          "access_policy": null
+        }
+      ]
+    }
+  }
+}
+```
+`docid` is populated once a paper has a real, registered DOCiD — empty string
+otherwise. This is the field to poll if you want to detect which URAAS papers
+already carry a DOCiD vs. which don't yet.
+
+### `GET /api/papers/<id>`
+Full metadata for one paper — title, abstract, authors, DOI, publication
+date, Special Collections category/score, and identifiers (`docid`, `ark`,
+`ror`) where present.
+
+### `GET /api/papers/<id>/download`
+Redirects to or streams the PDF, only when the item is open access.
+
+### `GET /api/papers/<id>/bibtex`
+BibTeX citation export for one paper (`Content-Type: text/plain`).
+
+### `GET /api/analytics/special-collections`
+The Special Collections subset specifically — the papers classified into one
+or more of the 8 categories (Indigenous Knowledge, Cultural Heritage, Oral
+Tradition, Ethnomusicology, African Philosophy, Traditional Medicine, Ethnic
+Languages & Groups, Pan-African Studies).
+
+### `GET /api/analytics/special-collections/overview`
+Aggregate view over that same set: category co-occurrence matrix, country
+breakdown, custodian institutions, and the most-cited/influential papers per
+category.
+
+```json
+{
+  "co_occurrence": { "labels": [...], "matrix": [[...]] },
+  "countries": [{ "code": "NG", "name": "Nigeria", "papers": 54 }],
+  "custodians": [{ "institution": "University of Lagos", "count": 54 }],
+  "influential": [{ "id": 1, "title": "...", "categories": [...], "citations": 856, "sc_score": 3.0 }]
+}
+```
+
+### `GET /api/university-registry`
+The full registry of institutions URAAS crawls against (52 African
+universities as of this writing) — name, ROR, country.
+
+### `GET /api/institution/info?institution=unilag`
+Summary for one institution from the registry above. `institution` is the
+registry key (e.g. `unilag`); omitting it returns null fields.
+
+```json
+{ "status": "success", "data": { "institution_name": "University of Lagos", "country_name": "Nigeria", "country_code": "NG" } }
+```
+
+---
+
+## Example
+
+```bash
+curl -H "X-API-Key: uraas_live_..." \
+  "https://lordkiki-apa-uraas.hf.space/api/analytics/special-collections/overview"
+```
+
+---
+
+## What this API does *not* do (yet)
+
+There is currently no inbound path — nothing lets DOCID's platform push data
+or notifications back into URAAS (e.g. "here's the DocID we just assigned,
+sync your record" or a webhook on registration). Every endpoint above is
+URAAS serving data out. If two-way sync is needed, that's new work to scope
+separately, not something already exposed here.
+
+## For URAAS admins — issuing/revoking keys
+
+```bash
+python scripts/manage_api_keys.py create --name "Africa PID Alliance / DOCiD"
+python scripts/manage_api_keys.py list
+python scripts/manage_api_keys.py revoke --prefix uraas_live_AbCd1234
+```
+
+The full key value is shown exactly once, at creation — only its hash is
+stored. If a key is lost or compromised, revoke it and issue a new one; there
+is no way to recover a lost key's value.
+
+To add a new endpoint to partner access, add its Flask endpoint (function)
+name to `PARTNER_ENDPOINTS` in `uraas/dashboard/app.py` — nothing is reachable
+via API key unless explicitly listed there.

scripts/manage_api_keys.py

@@ -0,0 +1,117 @@
+"""Manage API keys for external partner access to the URAAS dashboard API
+(e.g. Africa PID Alliance / DOCiD reading papers/analytics endpoints
+server-to-server, instead of using the browser session-cookie login).
+
+The plaintext key is shown exactly once, at creation — it is never stored;
+only its SHA-256 hash is, so a stolen database dump can't be used to
+authenticate as a partner. If a key is lost, revoke it and create a new one.
+
+Usage:
+    python scripts/manage_api_keys.py create --name "Africa PID Alliance / DOCiD"
+    python scripts/manage_api_keys.py list
+    python scripts/manage_api_keys.py revoke --prefix uraas_live_AbCd
+"""
+
+import argparse
+import hashlib
+import os
+import secrets
+import sys
+from datetime import datetime
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from uraas.database import ApiKey, SessionLocal
+
+_PREFIX = "uraas_live_"
+
+
+def _hash(key: str) -> str:
+    return hashlib.sha256(key.encode("utf-8")).hexdigest()
+
+
+def cmd_create(args):
+    raw = _PREFIX + secrets.token_urlsafe(32)
+    session = SessionLocal()
+    try:
+        row = ApiKey(
+            name=args.name,
+            key_hash=_hash(raw),
+            key_prefix=raw[: len(_PREFIX) + 8],
+            scope="read",
+            created_by=args.created_by or "cli",
+        )
+        session.add(row)
+        session.commit()
+        print("Created API key — this is the ONLY time the full value is shown:\n")
+        print(f"    {raw}\n")
+        print(f"Name:   {row.name}")
+        print(f"Prefix: {row.key_prefix}  (safe to log/reference later)")
+        print(f"Scope:  {row.scope}")
+        print("\nGive the partner this header format:")
+        print(f'    X-API-Key: {raw}')
+    finally:
+        session.close()
+
+
+def cmd_list(args):
+    session = SessionLocal()
+    try:
+        rows = session.query(ApiKey).order_by(ApiKey.created_at.desc()).all()
+        if not rows:
+            print("No API keys yet.")
+            return
+        for r in rows:
+            status = "REVOKED" if r.revoked else "active"
+            last_used = r.last_used_at.isoformat() if r.last_used_at else "never"
+            print(f"[{r.id}] {r.key_prefix}...  {r.name!r}  scope={r.scope}  "
+                  f"status={status}  created={r.created_at}  last_used={last_used}")
+    finally:
+        session.close()
+
+
+def cmd_revoke(args):
+    session = SessionLocal()
+    try:
+        q = session.query(ApiKey)
+        row = None
+        if args.id:
+            row = q.filter(ApiKey.id == args.id).first()
+        elif args.prefix:
+            row = q.filter(ApiKey.key_prefix == args.prefix).first()
+        if not row:
+            print("No matching key found.")
+            return
+        row.revoked = True
+        row.revoked_at = datetime.utcnow()
+        session.commit()
+        print(f"Revoked key [{row.id}] {row.key_prefix}... ({row.name!r})")
+    finally:
+        session.close()
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    sub = parser.add_subparsers(dest="command", required=True)
+
+    p_create = sub.add_parser("create", help="Issue a new partner API key")
+    p_create.add_argument("--name", required=True, help="Partner label, e.g. 'Africa PID Alliance / DOCiD'")
+    p_create.add_argument("--created-by", default=None)
+    p_create.set_defaults(func=cmd_create)
+
+    p_list = sub.add_parser("list", help="List all keys (never shows the full value)")
+    p_list.set_defaults(func=cmd_list)
+
+    p_revoke = sub.add_parser("revoke", help="Revoke a key")
+    p_revoke.add_argument("--id", type=int, default=None)
+    p_revoke.add_argument("--prefix", default=None)
+    p_revoke.set_defaults(func=cmd_revoke)
+
+    args = parser.parse_args()
+    if args.command == "revoke" and not (args.id or args.prefix):
+        parser.error("revoke requires --id or --prefix")
+    args.func(args)
+
+
+if __name__ == "__main__":
+    main()

scripts/push_to_hf.py

@@ -123,16 +123,39 @@ def main():
     print(_SEP)
 
     # ── Auth ───────────────────────────────────────────────────────────────
+    # Live-verified 2026-08-04: with no HF_TOKEN env var, this used to always
+    # call login() with no token, which prompts interactively for one — in
+    # any non-interactive shell (CI, a background task, this script run
+    # without a TTY) that blocks forever waiting for input that can never
+    # arrive, even when a valid token is already cached from a previous
+    # `huggingface-cli login` on the same machine. Try the cached credential
+    # first; only fall back to an interactive prompt when actually attached
+    # to a terminal.
     token = os.getenv("HF_TOKEN")
     if token:
         login(token=token, add_to_git_credential=True)
         print("  Logged in via HF_TOKEN env var.")
     else:
-        print()
-        print("  Paste your HF write token below.")
-        print("  (Get one at: https://huggingface.co/settings/tokens)")
-        print()
-        login(add_to_git_credential=True)
+        from huggingface_hub import HfFolder
+        cached = HfFolder.get_token()
+        if cached:
+            try:
+                HfApi().whoami(token=cached)
+                print("  Logged in via cached HF credential.")
+            except Exception:
+                cached = None
+        if not cached:
+            if not sys.stdin.isatty():
+                print()
+                print("  No HF_TOKEN env var, no cached credential, and no terminal to prompt on.")
+                print("  Set HF_TOKEN=hf_... and re-run, e.g.:")
+                print(f"    HF_TOKEN=hf_... python {os.path.basename(__file__)}")
+                sys.exit(1)
+            print()
+            print("  Paste your HF write token below.")
+            print("  (Get one at: https://huggingface.co/settings/tokens)")
+            print()
+            login(add_to_git_credential=True)
 
     api = HfApi()
 

uraas/dashboard/app.py

@@ -1,11 +1,15 @@
 import csv
+import hashlib
 import io
 import json
 import logging
 import os
 import re
+import secrets
 import subprocess
 import threading
+import time
+from datetime import datetime
 
 from flask import (
     Flask,
@@ -25,6 +29,7 @@ from sqlalchemy.orm import selectinload
 from uraas.analytics.engine import analytics
 from uraas.config import config
 from uraas.database import (
+    ApiKey,
     Author,
     Collection,
     Community,
@@ -109,8 +114,71 @@ ADMIN_ENDPOINTS = {
     "citations_velocity_csv",
     "staff_directory",
     "clear_half_and_recrawl",
+    "admin_list_api_keys",
+    "admin_create_api_key",
+    "admin_revoke_api_key",
 }
 
+# Read-only data endpoints external partners (e.g. Africa PID Alliance /
+# DOCiD) may reach with a server-to-server API key instead of a browser
+# session — see uraas.dashboard.app._check_api_key and
+# scripts/manage_api_keys.py. Deliberately a small, explicit allowlist: a
+# valid key never grants anything outside it, including ADMIN_ENDPOINTS —
+# partner keys are read-only by construction, not just by convention.
+PARTNER_ENDPOINTS = {
+    "get_stats",
+    "papers_tree",
+    "get_paper",
+    "download_paper",
+    "export_single_bibtex",
+    "special_collections",
+    "special_collections_overview",
+    "get_university_registry",
+    "institution_info",
+}
+
+# Simple in-memory sliding-window limiter for API-key traffic, separate from
+# the IP-keyed `limiter` above (a partner integration calls from a small,
+# fixed set of IPs the per-IP limiter wasn't designed to distinguish from
+# abuse). Process-local only — fine for a handful of trusted partner keys on
+# a single dashboard instance; would need a shared store (e.g. Redis) behind
+# multiple worker processes.
+_API_KEY_RATE_LIMIT = 120  # requests per rolling 60s window, per key
+_api_key_hits: dict[str, list] = {}
+
+
+def _check_api_key():
+    """Validate the X-API-Key header against uraas.database.ApiKey.
+
+    Returns (True, partner_name) if valid, unrevoked, in-quota, and the
+    caller updates ApiKey.last_used_at as a side effect. Returns
+    (False, (response, status)) otherwise, ready to return directly from
+    the before_request hook.
+    """
+    raw_key = request.headers.get("X-API-Key")
+    if not raw_key:
+        return False, None
+
+    key_hash = hashlib.sha256(raw_key.encode("utf-8")).hexdigest()
+    session_db = SessionLocal()
+    try:
+        row = session_db.query(ApiKey).filter(ApiKey.key_hash == key_hash).first()
+        if not row or row.revoked:
+            return False, (jsonify({"status": "error", "message": "Invalid or revoked API key"}), 401)
+
+        now = time.time()
+        hits = [t for t in _api_key_hits.get(key_hash, []) if now - t < 60]
+        if len(hits) >= _API_KEY_RATE_LIMIT:
+            return False, (jsonify({"status": "error", "message": "Rate limit exceeded"}), 429)
+        hits.append(now)
+        _api_key_hits[key_hash] = hits
+
+        row.last_used_at = datetime.utcnow()
+        session_db.commit()
+        return True, row.name
+    finally:
+        session_db.close()
+
 
 @app.before_request
 def _enforce_authentication():
@@ -118,6 +186,18 @@ def _enforce_authentication():
     # Unknown endpoint (404s) and explicit public routes pass through.
     if endpoint is None or endpoint in PUBLIC_ENDPOINTS:
         return None
+
+    if request.headers.get("X-API-Key"):
+        # Check the allowlist before touching the DB/rate-limit counter at
+        # all — an API key is never a path to an endpoint outside
+        # PARTNER_ENDPOINTS, so there's nothing to gain by validating first.
+        if endpoint not in PARTNER_ENDPOINTS:
+            return jsonify({"status": "error", "message": "This endpoint is not available via API key"}), 403
+        ok, result = _check_api_key()
+        if not ok:
+            return result
+        return None  # valid partner key, allowlisted endpoint
+
     role = current_role()
     if not role:
         if request.path.startswith("/api/"):
@@ -1827,6 +1907,79 @@ def clear_half_and_recrawl():
     )
 
 
+@app.route("/api/admin/api-keys", methods=["GET"])
+def admin_list_api_keys():
+    """List partner API keys (admin only). Never returns full key values —
+    only the prefix stored at creation, same as scripts/manage_api_keys.py."""
+    session_db = SessionLocal()
+    try:
+        rows = session_db.query(ApiKey).order_by(ApiKey.created_at.desc()).all()
+        return api_ok({
+            "keys": [
+                {
+                    "id": r.id,
+                    "name": r.name,
+                    "prefix": r.key_prefix,
+                    "scope": r.scope,
+                    "revoked": r.revoked,
+                    "created_at": r.created_at.isoformat() if r.created_at else None,
+                    "last_used_at": r.last_used_at.isoformat() if r.last_used_at else None,
+                }
+                for r in rows
+            ]
+        })
+    finally:
+        session_db.close()
+
+
+@app.route("/api/admin/api-keys", methods=["POST"])
+def admin_create_api_key():
+    """Issue a new partner API key (admin only). The plaintext value is
+    returned exactly once, in this response — only its hash is persisted, so
+    it can never be retrieved again after this call."""
+    data = request.get_json(silent=True) or {}
+    name = (data.get("name") or "").strip()
+    if not name:
+        return api_error("name is required", status=400)
+
+    raw = "uraas_live_" + secrets.token_urlsafe(32)
+    session_db = SessionLocal()
+    try:
+        row = ApiKey(
+            name=name,
+            key_hash=hashlib.sha256(raw.encode("utf-8")).hexdigest(),
+            key_prefix=raw[:20],
+            scope="read",
+            created_by=session.get("user"),
+        )
+        session_db.add(row)
+        session_db.commit()
+        return api_ok({
+            "id": row.id,
+            "name": row.name,
+            "prefix": row.key_prefix,
+            "key": raw,
+        }, narrative="Save this key now — it will not be shown again.")
+    finally:
+        session_db.close()
+
+
+@app.route("/api/admin/api-keys/<int:key_id>/revoke", methods=["POST"])
+def admin_revoke_api_key(key_id):
+    """Revoke a partner API key (admin only)."""
+    session_db = SessionLocal()
+    try:
+        row = session_db.query(ApiKey).filter(ApiKey.id == key_id).first()
+        if not row:
+            return api_error("Key not found", status=404)
+        row.revoked = True
+        row.revoked_at = datetime.utcnow()
+        session_db.commit()
+        return api_ok({"id": row.id, "revoked": True})
+    finally:
+        session_db.close()
+
+
 @app.route("/api/author/<int:author_id>/metrics")
 def get_author_metrics(author_id):
     """Get bibliometric indicators for an author (h-index, citations, etc.)."""

uraas/database.py

@@ -368,6 +368,30 @@ class DepositBatch(Base):
     deposit_log = Column(Text, default="[]")
 
 
+class ApiKey(Base):
+    """Server-to-server credential for external partners (e.g. Africa PID
+    Alliance / DOCiD) reading URAAS data programmatically — distinct from the
+    dashboard's session-cookie login, which only suits a human in a browser.
+
+    The plaintext key is shown exactly once at creation time (see
+    scripts/manage_api_keys.py) and never stored — only its SHA-256 hash is
+    persisted, so a stolen database dump can't be used to authenticate.
+    """
+
+    __tablename__ = "api_keys"
+
+    id = Column(Integer, primary_key=True)
+    name = Column(String(255), nullable=False)  # partner label, e.g. "Africa PID Alliance / DOCiD"
+    key_hash = Column(String(64), unique=True, index=True, nullable=False)  # sha256 hex digest
+    key_prefix = Column(String(12), nullable=False)  # first chars only, for display/audit
+    scope = Column(String(20), default="read", nullable=False)  # "read" is the only scope for now — never admin
+    created_at = Column(DateTime, default=datetime.utcnow)
+    created_by = Column(String(100))  # admin session username that issued it
+    last_used_at = Column(DateTime)
+    revoked = Column(Boolean, default=False, nullable=False)
+    revoked_at = Column(DateTime)
+
+
 # ── Indexes for query performance ─────────────────────────────────────────────
 # ix_items_docid is auto-created by index=True on Item.docid — no duplicate needed
 Index("ix_items_language", Item.language_code)