# PLAN - Worcestershire Libraries Live Assistant

The blueprint for the build: what it is, how it's wired, what's done, and what's next.
Companion docs: [README](README.md) · [SUBMISSION](SUBMISSION.md) · [IDEAS](IDEAS.md) · [FIELD_NOTES](FIELD_NOTES.md).

## Goal

Give Worcestershire Libraries one conversational voice that answers any resident
question from **official, source-cited data at every granularity** - down to
*which branch has the copy on its shelf right now* and *exactly what every page
of the Hive offers* - tells them **exactly what they need to sign up**, and
**surfaces the services they never knew existed** - on a ≤32B model,
locally-capable. The DCMS-evidenced answer to "the library doesn't shout about
what it offers." Built with/for Jack's library-resources team.

The granularity ladder:

| Level | Data | Tool |
|---|---|---|
| Service | 87 council service pages, eligibility-first | KB tools |
| Page | all 57 pages of thehiveworcester.org, exact offerings + provenance | `hive_info` |
| Item | live catalogue (Atom feed), incl. BorrowBox deep links | `search_catalogue` |
| Copy | per-branch holdings + status from the item's detail page | `where_to_get` |

## Architecture

```
                       ┌─────────────────────────────────────────┐
   user question ──▶ │ route() LLM JSON router + keyword │
                       │ fallback (no-token safe) │
                       └───────────────┬────────────────────────────┘
                                       ▼
        ┌──────────────── 12 tools ─────────────────┐ ┌── GraphRAG ──┐
        │ where_to_get (copy-level: shelf @ │ │ local_search │
        │ branch → reserve → │ │ global_search│
        │ BorrowBox tonight) │ │ multi-hop │
        │ hive_info (57 Hive pages, exact │◀──│ over graph │
        │ offer + provenance) │ └──────────────┘
        │ search_catalogue (SirsiDynix Atom feed) │ library_graph.json
        │ whats_new (newest titles) │ 349 nodes/502 edges
        │ find_library (hours/open-now/facils) │ incl. 26 HiveService
        │ mobile_library (154 villages) │
        │ library_events (live events) │
        │ online_hub (PressReader/BorrowBox…)│
        │ libraries_unlocked · printing · membership │
        └───────────────────┬────────────────────────┘
                            ▼
        synthesise (≤32B) + eligibility + £-receipt + EAST nudge
                            + source link + open agent trace
                            ▼
                    answer + quick-reply chips
```

## Components & files

| Layer | File | Role |
|---|---|---|
| Ingestion | `build_kb.py` → `library_kb.json` | Crawl all 218 library URLs → 87 services / 23 branches / 17 resources, with eligibility & facilities |
| Hive ingestion | `build_hive_kb.py` → `hive_kb.json` | Crawl all 57 Hive pages → exact per-page offerings, what-you-need, prices, levels; curated `hive_profile` of 26 extended capabilities, every fact source-tagged |
| Graph | `graph_build.py` → `library_graph.json` | Deterministic GraphRAG over BOTH KBs: entities → relationships → communities → reports (incl. 26 HiveService nodes; offline rebuild keeps villages) |
| Sources | `library_sources.py` | 12 live/KB tools + curated hub access + membership tiers + copy-level availability parser |
| Retrieval | `graph_rag.py` | local/global graph search (multi-hop) |
| Traces | `trace.py` → `traces.jsonl` | per-turn structured agent trace (Open Trace badge) |
| App | `app.py` | router, renders, behaviour-change layer, Gradio UI |
| Custom UI | `server.py` + `index.html` | `gradio.Server` custom frontend (Off-Brand badge) |

## Build phases

- [x] **P1 - Live tools.** Catalogue (Atom), mobile, events, printing. Verified live.
- [x] **P2 - Comprehensive KB.** Full council-site crawl; eligibility + facilities + hours.
- [x] **P3 - New tools.** find_library (open-now), online_hub (+curated access), libraries_unlocked, membership_help, whats_new.
- [x] **P4 - GraphRAG.** 320-node graph; multi-hop "late library + café + meeting rooms" → Malvern.
- [x] **P5 - Behaviour-change layer.** EAST nudges, £-saved value receipt, quick-reply chips (DCMS/COM-B grounded).
- [x] **P6 - Traces.** JSONL logging + in-chat "how I answered" panel.
- [x] **P7 - KB refinement.** Curate all 17 hub resources, filter time-bound junk, tag seasonal pages.
- [x] **P8 - The Hive, page-level.** Reverse the exclusion: crawl all 57 pages
  with provenance (`build_hive_kb.py`), `hive_info` tool, Hive in the graph
  (open-late, archives, 26 capabilities), council-wins-on-conflict rule.
- [x] **P9 - Copy-level "where to get it".** `where_to_get`: per-branch
  holdings/status from the item detail page (two parse strategies, fail-soft),
  free-reservation steps, BorrowBox deep links from the Atom feed's
  Electronic Access field. *Live-verify with `python library_sources.py`.*
- [ ] **P10 - Deploy** to the Space + `HF_TOKEN` + smoke-test (user action),
  incl. the new self-test §5 (copy-level) against the live catalogue.
- [ ] **P11 - Stretch:** custom `gr.Server` frontend; a small fine-tune (badge); llama.cpp local run ().

## The behaviour-change layer (why, not just what)

Grounded in [DCMS *What works to engage non-users*](https://www.gov.uk/government/publications/what-works-to-engage-library-non-users) (COM-B) + the
Behavioural Insights Team **EAST** framework:

- **Easy** → quick-reply chips, exact "what you need", deep links.
- **Attractive** → £-saved "value receipt" (DCMS's strongest reframe: money-saving).
- **Social/Timely** → one contextual "did you know?" nudge at the moment of contact.
- **Awareness of breadth** (the #1 barrier) → the nudge engine surfaces hidden gems.
- **Measurement loop** → trace logs which nudges convert → evidence for Jack's campaigns.

## Trace & eval strategy (Open Trace)

Every turn writes one JSON object to `traces.jsonl` (route → steps → answer → sources →
timing), close to the hackathon's own trace-dataset schema. `trace.push_to_hub(repo_id)`
uploads it as a dataset. The same trace renders in-chat as a "how I answered" panel
(Best Agent evidence). Future: a golden-question eval set scoring route accuracy +
source correctness.

## Badge roadmap

Best Agent · Open Trace · Field Notes · Off-Brand ◑ (server.py) ·
Off-the-Grid / Llama Champion ◑ (local llama.cpp) · Well-Tuned (next).

## Risks & mitigations

| Risk | Mitigation |
|---|---|
| HF Inference flaky / no token | **No-LLM fallback** renders raw live data - demo never breaks |
| Council site HTML changes | Re-run `build_kb.py` (re-crawl) any time; tools fail soft |
| SirsiDynix slow/timeouts | per-call try/except; app degrades gracefully |
| Hive site goes stale again | every Hive fact shows its crawl date; council wins on conflict; `build_hive_kb.py` re-crawls in minutes |
| Detail-page holdings markup shifts | two independent parse strategies (table + branch-name text scan); if both miss, the answer says so and links the item page - never guesses |
| Gradio 6 needs Py3.10+ | tested logic locally; boot-test on the Space first |
| Facility data sparse (e.g. "study space") | multi-hop honest about coverage; lead demo on café+meeting |

## Future work

Fine-tune a tiny model on Q→tool routing; live PressReader title search; FOI ingestion
(WhatDoTheyKnow) as a transparency feature; auto-generated "Shelf Life" podcast +
share-cards (see [IDEAS.md](IDEAS.md)); per-branch service mapping.