| --- |
| title: Beacon |
| emoji: 🕯️ |
| colorFrom: yellow |
| colorTo: red |
| sdk: docker |
| app_port: 7860 |
| pinned: false |
| --- |
| |
| # Handoff: Astroparse — annotated astronomy paper reader |
|
|
| ## Overview |
|
|
| Astroparse turns an astronomy PDF into a cleanly typeset reading view with **paragraph-level AI marginalia**. The pipeline: PDF → text extraction (pymupdf4llm) → header/footer cleaning → paragraph segmentation → SPECTER2 embeddings + BM25 index → literature retrieval (Pathfinder / NASA ADS) → per-paragraph LLM commentary rendered as Tufte-style margin notes beside the text. Users pick a commentary mode per paragraph (historical context, referee critique, related-literature observations, plain-language explainer, methods comparison), inspect the retrieved papers behind each note (abstract, relevance scores, ADS/arXiv links), choose their LLM provider (Claude / OpenAI / Gemini) with their own API key, and save/reload the whole annotated state as a JSON session. |
|
|
| Primary user: a single researcher (the project owner), with intent to expand to a public web app if useful. |
|
|
| ## About the Design Files |
|
|
| The files in this bundle are **design references created in HTML** — a working prototype showing the intended look and behavior, not production code to copy directly. The task is to **recreate this design in a real application environment**. There is no existing codebase; the recommended stack (chosen to fit the Python-native pipeline) is: |
|
|
| - **Frontend**: React + Vite (the prototype's JSX components port almost directly; they currently run via in-browser Babel, which must not ship to production) |
| - **Backend**: Python + FastAPI (pymupdf4llm, SPECTER2, BM25, and Pathfinder/ADS access are all Python) |
| - **Hosting**: Hugging Face Spaces (Docker) for the first deployment; Fly.io/Render later if a custom domain is needed |
|
|
| Important: the prototype **simulates** parsing and retrieval (it ships one hard-coded paper, Mowla & Iyer 2024, and a curated 10-paper retrieval set in `data.js`). The pipeline screen plays a scripted animation. In production these become real backend calls — see "Backend pipeline specification" below. The LLM commentary generation in the prototype is REAL (it calls providers from the browser) and its prompt design should be kept. |
|
|
| ## Fidelity |
|
|
| **High-fidelity.** Colors, typography, spacing, and interactions are final design intent. Recreate the UI pixel-faithfully. The only placeholder behaviors are the simulated parsing/retrieval described above. |
|
|
| ## Design language (read first) |
|
|
| "Annotated encyclopedia / manuscript marginalia." A parchment folio on a darker desk; EB Garamond for all reading text; IBM Plex Mono for machine traces (pipeline logs, scores, file names, buttons); rubrication red for marks of structure (pilcrows, drop caps, rules, active states); five mode colors at matched oklch lightness/chroma. No rounded-corner cards, no gradients, no icons — typography does all the work. Buttons are underlined mono text, not pills. |
|
|
| ## Design tokens |
|
|
| ### Colors |
| Theme is driven by 3 CSS variables (user-switchable palettes via Tweaks): |
|
|
| | Token | Default ("Parchment") | Alt 1 ("Ivory") | Alt 2 ("Chalk") | |
| |---|---|---|---| |
| | `--paper` (background) | `#f6f1e6` | `#fbf8f1` | `#f3f2ee` | |
| | `--ink` (text) | `#2b2418` | `#231f17` | `#22241f` | |
| | `--rubric` (accent) | `#9a3b20` | `#8a5a18` | `#3f5a96` | |
|
|
| Desk (page surround): `color-mix(in oklab, var(--paper) 86%, var(--ink) 14%)`. |
| All secondary grays are mixes of ink into paper (e.g. 70% ink for secondary text, 50% for captions, 25% for hairline borders, 14% for rules, 8% for faint dividers) — never neutral hex grays, so every tint stays warm with the palette. |
|
|
| Mode colors (fixed, all `oklch(0.52 C H)`): |
|
|
| | Mode | Color | Glyph | |
| |---|---|---| |
| | Historical context | `oklch(0.52 0.10 70)` amber | H | |
| | Referee mode | `oklch(0.52 0.13 28)` red | R | |
| | From the literature | `oklch(0.52 0.10 250)` blue | L | |
| | Plain-language | `oklch(0.52 0.10 150)` green | P | |
| | Methods comparison | `oklch(0.52 0.10 310)` violet | M | |
|
|
| ### Typography |
| - **EB Garamond** (Google Fonts, 400/500/600 + italics): all reading text, headings, wordmark |
| - **IBM Plex Mono** (400/500): logs, scores, chips, text-buttons, hints, metadata |
| - Body: 19px default (user-adjustable 16–23px via Tweaks), line-height 1.58, justified + `hyphens: auto` (toggleable) |
| - Paper title: 31px / 600 / line-height 1.22, centered, `text-wrap: balance` |
| - Section headings: 13px / 500 / uppercase / letter-spacing 0.18em |
| - Margin notes: 13.5px / line-height 1.5; note mode label: 12.5px small-caps, letter-spacing 0.08em, weight 500, in mode color |
| - Mono UI text: 10–12px, letter-spacing 0.04–0.14em |
| - Wordmark "astroparse": uppercase, letter-spacing 0.28–0.32em, 12–14px, rubric color |
|
|
| ### Spacing & chrome |
| - Folio: max-width 1180px, centered, padding 64px 72px 48px, 1px border (ink 18%), shadow `0 1px 2px rgba(40,30,10,0.08), 0 18px 50px -24px rgba(40,30,10,0.4)` |
| - Border radius: 0 everywhere except 2px on chips/inputs and 50% on mode dots |
| - Text column: flex `0 1 660px`, 48px gap to margin column |
| - Margin column: flex `1 0 300px`, max-width 330px, 1px left rule (ink 14%), notes inset 28px from the rule |
| - Paragraph spacing: 1.15em; note-to-note minimum gap: 26px |
|
|
| ## Screens / Views |
|
|
| ### 1. Upload ("Place a manuscript here") |
| Centered column on the desk background: wordmark, then a folio drop-zone card (min(620px, 92vw); padding 58px 48px 44px) with a double border — 1px solid outer (ink 18%) plus 1px **dashed** inset outline (`outline-offset: -10px`, ink 30%; turns rubric on hover/drag, card scales 1.01 on drag-over). Contents, centered: a 58px rubric pilcrow ¶; "Place a manuscript here" (26px / 500); italic subline "drop an astronomy PDF, or click to browse" (16px, ink 70%); mono caption "parsed with pymupdf4llm · annotated against the literature via Pathfinder" (10.5px, ink 52%, 26px above-gap). |
| Below the card: two mono text-buttons — "open ‹last/sample manuscript›" and "reload a saved session (.json)" — separated by "·". Bottom mono caption explains prototype limitations (drop in production). |
| Behavior: click anywhere on card opens file picker (accept `application/pdf`); drag-and-drop accepted; choosing a `.json` via the second button restores a session directly. |
|
|
| ### 2. Pipeline ("in preparation") |
| Same desk + wordmark. A folio card (min(600px, 92vw), padding 30px 38px 24px). Header row: italic filename (17px) left, "IN PREPARATION" mono uppercase rubric right; hairline rule below. Then six stages as a list, each separated by faint rules (ink 8%): |
| - Row: roman numeral (mono 11px, rubric, right-aligned 26px col) · stage name (17px / 500) · tool name (mono 10.5px, ink 50%, right) · status mark (✓ done / … active, rubric) |
| - Pending stages at 38% opacity; active stage name in rubric |
| - Under active/done stages: mono log lines (11px, ink 58%, indented 36px) appear one-by-one (~420ms apart, fade+2px rise 300ms) |
|
|
| Stages (names are part of the design voice): i. Reading manuscript (pymupdf4llm) · ii. Cleaning the leaves (header/footer removal) · iii. Ruling the columns (paragraph segmentation) · iv. Committing to memory (SPECTER2 embeddings) · v. Indexing the lexicon (BM25) · vi. Consulting the library (Pathfinder). Log lines report real counts from the backend in production. |
| Footer: right-aligned "skip to the manuscript →" text-button. Auto-advance ~1.1s after the last stage. In production, drive stage transitions from real pipeline progress (SSE or polling). |
|
|
| ### 3. Reader (the core screen) |
| **Toolbar** (sticky, backdrop-blur paper at 88% opacity, hairline bottom border, padding 13px 28px): wordmark · filename (mono 10.5px ink 50%) · spacer · text-buttons: "annotator: ‹provider›", "save session ↓", "load", "new manuscript". |
|
|
| **Folio** (margin 36px auto 70px): centered paper header — title, italic author line (16px ink 75%), mono metadata line "arXiv:2402.08696 · 20 pp · Nature 636, 332–338 (2024)" (10.5px ink 50%). |
|
|
| **Mode legend**: a centered row between hairline rules; five items (8px color dot + small-caps label, 13px, ink 70%, hover → mode color). Clicking a legend item sets **every** note to that mode (tooltip: "set every note to …"). |
|
|
| **Two-column body**: text column + margin column (rule between). Notes can flip to the left side via Tweaks (`flex-direction: row-reverse`; rule flips side). |
|
|
| Text column: |
| - Section headings as specced above |
| - Paragraphs with a rubric pilcrow ¶ hanging in the left gutter (absolute, `left: -1.5em`, 0.85em, 55% opacity) |
| - First paragraph of each section gets a drop cap (`::first-letter`: float left, 3.05em, line-height 0.82, weight 500, rubric) when the Tweak is on |
| - Hovering a paragraph tints it AND its note with the note's mode color at 7% (`color-mix(... 7%, transparent)`), and vice versa — this cross-highlight is the key affordance linking the columns |
|
|
| Margin column — each paragraph has exactly one note, absolutely positioned: |
| - **Alignment rule (critical)**: every note's top is pinned EXACTLY level with its paragraph's top. If a paragraph is shorter than its note, the text column adds extra space below that paragraph (extra `margin-bottom` in px) so the next paragraph/note pair is also level. Algorithm: measure natural paragraph tops (subtracting previously applied spacers), walk pairs in order, `spacer[i] = max(0, noteTop[i] + noteHeight[i] + 26 − nextParagraphNaturalTop)`. Re-run on resize, font load, and any note content/mode change; must be idempotent (no oscillation). Disable spacers and absolute positioning below 900px viewport (notes become static blocks under their paragraphs). |
| - Note anatomy, top to bottom: 1.5px top rule in mode color; header row — small-caps mode label in mode color + "✦" marker if LLM-generated + "↻" regenerate button (hidden until note hover; ink 40% → rubric on hover); body text; citation chips; mode dots. |
| - Citation chips: mono 10px, `[n] AuthorYY` (rank number in mode color), 1px border (ink 22%), 2px radius, padding 2px 7px 2px 5px; hover → mode color border/text. Click opens the citation popover. |
| - Mode dots: five 19px circles, mono 9px glyph letters (H R L P M); inactive: 1px border ink 25%, text ink 50%; hover: mode color; active: filled with mode color, paper-colored glyph. Clicking switches that note's mode; if no commentary exists yet for (paragraph, mode), generation starts. |
| - Loading state: italic "consulting the literature…" with an animated ellipsis (steps(4), 1.2s) |
| - Error state: italic message (include the provider error, e.g. "no API key set — open the annotator settings") + "try again" text-button |
|
|
| Folio footer (rule + centered mono 10px, ink 45%): "annotations are machine-generated against retrieved literature — verify before citing". Keep this disclaimer. |
|
|
| ### 4. Citation popover |
| Fixed-position card (360px; paper bg; 1px border ink 30%; shadow `0 16px 44px -16px rgba(40,30,10,0.5)`; padding 18px 20px 14px) anchored under the clicked chip, clamped to viewport (flips above if needed); invisible full-screen backdrop click-to-close; Esc closes; × button top-right. |
| Contents: mono overline "RETRIEVED · RANK n" (9.5px, rubric, letter-spacing 0.1em) · title (16.5px / 600) · italic authors+year+journal (13px, ink 65%) · abstract (13.5px / 1.5) · score bars (when "Retrieval scores" tweak is on): rows of `label | track | value` — mono 10px, 3px track (ink 12%) with rubric fill; "specter" normalized /1.0, "bm25" /20 · footer links "ADS ↗" and "arXiv ↗" (mono 11px rubric) → `https://ui.adsabs.harvard.edu/abs/{bibcode}/abstract` and `https://arxiv.org/abs/{arxiv}`, new tab. |
|
|
| ### 5. Annotator settings |
| Fixed panel, top-right (top 64px, right 24px, width 360px; same chrome as popover). Title "The Annotator" (19px / 600), italic subtitle "which mind writes in your margins". Fields: |
| - **Provider**: four toggle chips (mono 10.5px, 2px radius) — Built-in (Claude) / Anthropic / OpenAI / Gemini; active chip filled rubric. (In production, drop "Built-in" unless you ship a server-funded default.) Switching provider resets the model field to that provider's default: `claude-haiku-4-5` / `gpt-4o-mini` / `gemini-2.0-flash`. |
| - **Model**: mono text input (12px; 1px border ink 22%; focus border rubric; bg ink 4%). |
| - **API key** (hidden for built-in): password input + mono hint "‹console URL› · stored only in this browser, sent only to ‹provider›". |
| - Footer rule + right-aligned "done". |
| Field labels: 11px / 500 / uppercase / letter-spacing 0.14em / ink 70%. |
|
|
| ### 6. Tweaks panel (prototype-only) |
| The prototype has a design-review Tweaks panel (palette, body size, drop caps, justification, notes side, score visibility). In production, promote **notes side**, **body size**, **justification**, and **palette** to a small user-facing "display" preferences menu; persist with the session/app preferences. |
|
|
| ## Interactions & Behavior |
|
|
| - Upload → pipeline → reader is a linear state machine (`screen: 'upload' | 'pipeline' | 'reader'`). |
| - Mode switch (dot click): updates `noteModes[paraId]`; if `commentary[paraId+':'+mode]` is missing, fire generation (show loading state). Concurrent requests for the same key must be deduped; multiple different notes may generate in parallel. |
| - Regenerate (↻): always re-fires generation for the current (paragraph, mode), overwriting. |
| - Legend click: sets all paragraphs' modes (generating any missing ones). |
| - Hover cross-highlight paragraph ↔ note (7% mode-color tint both sides). |
| - Toast: fixed bottom-center, ink bg / paper text, mono 11px, padding 9px 18px, ~2.6s, for session events ("session written to …", "session restored", "could not read that session file"). |
| - No animations on note re-positioning (snap, don't tween — tweening reads as drift). |
| - Responsive: below 900px, single column; notes render as static blocks beneath their paragraphs; folio padding 40px 26px. |
|
|
| ## State Management |
|
|
| ``` |
| screen 'upload' | 'pipeline' | 'reader' |
| fileName string |
| paper { title, authors, venue, arxivId, pages, paragraphs: [{ id, section, firstOfSection?, text }] } |
| litPapers { [litId]: { id, short, title, authors, year, journal, bibcode, arxiv, abstract, |
| scores: { specter, bm25, rank } } } |
| litByPara { [paraId]: litId[] } // top-3, ranked |
| noteModes { [paraId]: modeId } // per-paragraph current mode |
| commentary { [`${paraId}:${modeId}`]: { text, cites: litId[], source: 'canned'|'llm', |
| error?, message? } } |
| generating { [`${paraId}:${modeId}`]: true } // in-flight |
| annotator { provider: 'anthropic'|'openai'|'gemini', model: string, key: string } |
| ``` |
|
|
| Autosave the session to localStorage on every commentary/mode change; restore on load. Annotator config persists separately (`localStorage`, key never leaves the browser in the prototype — see security note below). |
|
|
| ### Session JSON schema (export/import — keep stable) |
| ```json |
| { |
| "version": 1, |
| "savedAt": "ISO-8601", |
| "fileName": "string", |
| "screen": "reader", |
| "noteModes": { "p1": "explainer", ... }, |
| "commentary": { "p1:explainer": { "text": "...", "cites": ["mowla22"], "source": "llm" }, ... }, |
| "paper": { ...full paper object... } |
| } |
| ``` |
| Production additions: a content hash of the source PDF (to re-anchor sessions), and `litPapers`/`litByPara` (the prototype omits them from import since they're static). Never serialize the API key into sessions. |
|
|
| ## LLM commentary generation |
|
|
| Prompt template (keep; it produces good notes): |
|
|
| ``` |
| You are an erudite astronomical annotator writing a marginal note beside a paragraph of a |
| research paper, in the tradition of an annotated encyclopedia. Scholarly, precise, a little |
| wry. Write 55–85 words, a single paragraph, no markdown, no preamble. Refer to the retrieved |
| works by bracketed number, e.g. [1], where relevant. |
| |
| {MODE_INSTRUCTION} |
| |
| PARAGRAPH (from section "{section}"): |
| {paragraph text} |
| |
| RETRIEVED LITERATURE (via Pathfinder): |
| [1] {short} — "{title}" ({journal}). {abstract} |
| [2] ... |
| |
| Marginal note: |
| ``` |
|
|
| Mode instructions (verbatim in `app.jsx`): history = how the claim emerged, which earlier works set it up; referee = strongest assumption / unstated caveat / a check to report, specific not generic; literature = how retrieved works support/contradict/extend, compare numbers; explainer = plain language for a beginning grad student, unpack jargon, don't dumb down; methods = contrast technique with retrieved works' approaches, note trade-offs. |
|
|
| Provider routing (see `annotator.jsx` for working request shapes): Anthropic `POST /v1/messages` (max_tokens 400), OpenAI `POST /v1/chat/completions`, Gemini `POST /v1beta/models/{model}:generateContent`. Surface provider errors verbatim-but-truncated in the note error state. |
| |
| **Security note**: the prototype calls provider APIs directly from the browser with the user's key (acceptable for a personal tool). For the public app, proxy LLM calls through the backend: accept the user's key per-request over HTTPS, hold it in memory only, never log or persist it. Cache completions server-side by `hash(paragraph + mode + retrievedSetIds + model)`. |
| |
| ## Backend pipeline specification (replaces the simulation) |
| |
| FastAPI service; suggested endpoints: |
| |
| 1. `POST /api/parse` (multipart PDF) → `{ paper }`. |
| - `pymupdf4llm.to_markdown()` for extraction |
| - Cleaning: drop lines repeating across ≥ half the pages (running headers/footers), page numbers, arXiv/DOI stamps; repair hyphenated line breaks; set aside figure/table captions and the references section |
| - Segment into paragraphs (min length ~200 chars; merge continuation fragments across page breaks); detect section headings; mark `firstOfSection` |
| 2. `POST /api/index` → embeds paragraphs with SPECTER2 (`sentence-transformers`, CPU fine at this scale) and builds a `rank_bm25` index. Can be merged into /parse. |
| 3. `POST /api/retrieve` → per-paragraph top-3 papers `{ litByPara, litPapers }`. |
| - Pathfinder (the authors' astronomy literature retrieval tool) or equivalent hybrid search over NASA ADS: SPECTER cosine + BM25, normalized as in the popover's score display. Requires an ADS API token (server-side secret) |
| 4. `POST /api/annotate` `{ paragraph, mode, lit, provider, model, key }` → `{ text }` (the proxy above) |
| 5. Progress for the pipeline screen via SSE or a polled job-status endpoint, mapping 1:1 to the six stage rows (with real counts in the log lines). |
|
|
| ## Assets |
|
|
| No images or icon fonts. Everything is typography: ¶ (U+00B6), ✦ (U+2726), ↻ (U+21BB), ✓, ↗, →, ×. Fonts from Google Fonts: EB Garamond, IBM Plex Mono. The bundled manuscript content in `data.js` (Mowla & Iyer 2024 + 10 retrieval papers with real bibcodes/arXiv IDs) is useful as a test fixture for the real pipeline. |
|
|
| ## Files |
|
|
| - `Astroparse.html` — shell; ALL CSS lives here (the canonical style reference) |
| - `app.jsx` — root state machine, session save/load, generation orchestration, prompt + mode instructions, Tweaks wiring |
| - `reader.jsx` — reader screen: note alignment algorithm (`recompute`), MarginNote, CitePopover, score bars, legend |
| - `pipeline.jsx` — upload screen + staged pipeline animation |
| - `annotator.jsx` — provider settings panel + `apComplete()` multi-provider router (working request shapes for all three APIs) |
| - `data.js` — data model exemplar: modes, paper, litPapers, litByPara, defaultModes, canned commentary, pipeline stage copy |
| - `tweaks-panel.jsx` — prototype-only design-review scaffolding; do not port |
|
|
| Open `Astroparse.html` in a browser to interact with the reference implementation. |
|
|
