hirokamitai Claude Opus 4.7 (1M context) commited on
Commit
2d0df89
Β·
1 Parent(s): b32c6df

Sync _project docs to deployed reality

Browse files

- ARCHITECTURE: root route, parse() signature change, response shape, bucket
mount, 5-min sleep, public-Space sharp edge.
- CURRENT_STATE: deployed Space verified live; remaining task is
ANTHROPIC_API_KEY + first real PDF.
- CHANGELOG: dated v0 deploy entry; consolidated Unreleased into shipped.
- RUNBOOK: secret-setting recipe via huggingface_hub; dev-mode debug step;
hf spaces info command.
- INFRASTRUCTURE: public visibility, dev-mode off, deploy date.
- ENVIRONMENT: marks API_TOKEN as set, ANTHROPIC_API_KEY as pending.
- DECISIONS: new entry for public-Space + bearer-token auth model.
- ROADMAP: filled in (was placeholder).
- TODO: deploy items moved to Done; ANTHROPIC_API_KEY surfaced as next.
- SPEC: bucket replaces persistent-storage tier; sleep 15β†’5 min.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

_project/ARCHITECTURE.md CHANGED
@@ -17,25 +17,30 @@
17
  - **Frontend:** none. JSON API only.
18
  - **Build / package manager:** `uv` for dependency resolution; `pip` inside the
19
  Docker image (smaller layer than installing `uv` into the runtime).
20
- - **Hosting:** Hugging Face Space, **Docker SDK**, A10 small GPU, persistent
21
- storage mounted at `/data`.
 
22
 
23
  ## Components
24
 
25
- - **`app/main.py`** β€” FastAPI app. Exposes `GET /health` and `POST /parse`.
26
- Bearer-token auth via `Authorization: Bearer <secret>` header, secret read from
27
- the `API_TOKEN` Space secret.
28
- - **`app/parser.py`** β€” wraps marker-pdf. Loads the marker model converter once at
29
- startup (module-level singleton) so subsequent requests reuse weights on GPU.
30
- Exposes `parse(pdf_bytes, mode) -> str`. `mode="quality"` flips marker's
31
- LLM-enhancement flag on; `mode="fast"` leaves it off.
32
- - **`Dockerfile`** β€” CUDA-enabled base image, installs system deps marker needs,
33
- `pip install marker-pdf` + FastAPI stack, sets `HF_HOME=/data/hf` and
34
- `TORCH_HOME=/data/torch` so model weights persist across cold starts. Exposes
35
- port 7860 (HF Space convention).
36
- - **`README.md`** (Space root) β€” HF Space frontmatter (`sdk: docker`, `app_port:
37
- 7860`, `sleep_time: 900`, `suggested_hardware: a10g-small`), plus a short usage
38
- blurb.
 
 
 
 
39
 
40
  ## Data flow
41
 
@@ -45,18 +50,26 @@
45
  file: the PDF
46
  parse_mode: "fast" (default) | "quality"
47
 
48
- 2. FastAPI handler validates the bearer token + form fields.
 
49
 
50
- 3. Handler reads the PDF bytes into memory and calls parser.parse(bytes, mode).
51
 
52
  4. parser.parse runs marker-pdf on the GPU. On "quality" mode it also calls out
53
  to Claude (Sonnet 4.6 by default) via marker's `ClaudeService` for layout,
54
  table, and equation cleanup. See _project/ENVIRONMENT.md for env vars.
55
 
56
- 5. Handler returns JSON: { "markdown": "<string>", "mode": "fast|quality" }.
57
-
58
- 6. Space autosleeps after 15 min of no requests. Next request triggers a cold
59
- start; model weights load from /data instead of re-downloading.
 
 
 
 
 
 
 
60
  ```
61
 
62
  ## Key directories
@@ -64,12 +77,13 @@
64
  | Path | Purpose |
65
  |---|---|
66
  | `app/` | FastAPI app code (`main.py`, `parser.py`). |
67
- | `/data/` (runtime, on the Space) | HF persistent storage. Holds `hf/` (HF model cache) and `torch/` (Torch hub cache). Not in this repo. |
68
 
69
  ## External touchpoints
70
 
71
- - **Hugging Face Hub** β€” marker pulls its layout / OCR models from the Hub on first
72
- cold start; cached to `/data/hf` thereafter. No auth needed for public models.
 
73
  - **Anthropic API (Claude)** β€” only called when `parse_mode=quality`. Marker
74
  uses its built-in `ClaudeService`; we pin `claude-sonnet-4-6`. API key
75
  stored as a Space secret. See [DEPENDENCIES.md](DEPENDENCIES.md).
@@ -79,10 +93,14 @@
79
  - **Cold start is slow.** First request after sleep waits on container boot +
80
  model load from `/data` (~30–60s). This is by design for cost; don't try to
81
  paper over it with a keepalive.
82
- - **Model cache MUST land on `/data`.** Without persistent storage, every cold
83
- start re-downloads ~5 GB of weights, which both slows cold starts and burns
84
- bandwidth. The Dockerfile sets `HF_HOME=/data/hf` and `TORCH_HOME=/data/torch`
85
- for this reason.
 
 
 
 
86
  - **Package name vs import name.** `pip install marker-pdf`, but `import marker`.
87
  Easy to get wrong.
88
  - **Single-process, GPU-bound.** Don't run multiple Uvicorn workers β€” they'd each
 
17
  - **Frontend:** none. JSON API only.
18
  - **Build / package manager:** `uv` for dependency resolution; `pip` inside the
19
  Docker image (smaller layer than installing `uv` into the runtime).
20
+ - **Hosting:** Hugging Face Space (`hirokamitai/trenchlesspm`, **public**),
21
+ Docker SDK, A10 small GPU, HF Bucket `hirokamitai/trenchlesspm-cache`
22
+ mounted at `/data` for the model cache.
23
 
24
  ## Components
25
 
26
+ - **`app/main.py`** β€” FastAPI app. Exposes `GET /` (endpoint blurb, public),
27
+ `GET /health` (liveness + gpu/cpu, public), and `POST /parse` (bearer-gated).
28
+ Bearer-token auth via `Authorization: Bearer <secret>`; secret comes from the
29
+ `API_TOKEN` Space secret. Streams uploads to a temp file (no RAM buffering).
30
+ - **`app/parser.py`** β€” wraps marker-pdf. Loads the marker model dict once at
31
+ import time (module-level singleton) so subsequent requests reuse weights on
32
+ GPU. Exposes `parse(pdf_path, mode) -> tuple[str, int]` returning
33
+ `(markdown, page_count)`. `mode="quality"` flips marker's LLM-enhancement
34
+ flag on and routes through `marker.services.claude.ClaudeService`
35
+ (defaults to `claude-sonnet-4-6`).
36
+ - **`Dockerfile`** β€” CUDA-enabled base image, installs system deps marker
37
+ needs, `pip install` of marker-pdf + FastAPI stack, sets `HF_HOME=/data/hf`
38
+ and `TORCH_HOME=/data/torch` so model weights persist on the bucket. CMD
39
+ runs Uvicorn with `--timeout-keep-alive 1800` so long parses don't drop
40
+ the connection. Exposes port 7860 (HF Space convention).
41
+ - **`README.md`** (Space root) β€” HF Space frontmatter (`sdk: docker`,
42
+ `app_port: 7860`, `sleep_time: 300`, `suggested_hardware: a10g-small`,
43
+ `suggested_storage: small`), plus a short usage blurb.
44
 
45
  ## Data flow
46
 
 
50
  file: the PDF
51
  parse_mode: "fast" (default) | "quality"
52
 
53
+ 2. FastAPI handler validates the bearer token + form fields. Upload streams
54
+ in 1 MiB chunks to a NamedTemporaryFile so 100+ MB PDFs don't OOM.
55
 
56
+ 3. Handler times parser.parse(tmp_path, mode) with time.perf_counter().
57
 
58
  4. parser.parse runs marker-pdf on the GPU. On "quality" mode it also calls out
59
  to Claude (Sonnet 4.6 by default) via marker's `ClaudeService` for layout,
60
  table, and equation cleanup. See _project/ENVIRONMENT.md for env vars.
61
 
62
+ 5. Handler returns JSON:
63
+ {
64
+ "markdown": "...",
65
+ "mode": "fast"|"quality",
66
+ "page_count": <int>,
67
+ "duration_ms": <int>
68
+ }
69
+ page_count comes from marker's rendered.metadata, falling back to pypdf.
70
+
71
+ 6. Space autosleeps after 5 min of no requests. Next request triggers a cold
72
+ start; model weights load from /data (HF Bucket) instead of re-downloading.
73
  ```
74
 
75
  ## Key directories
 
77
  | Path | Purpose |
78
  |---|---|
79
  | `app/` | FastAPI app code (`main.py`, `parser.py`). |
80
+ | `/data/` (runtime, on the Space) | Mount point for HF Bucket `hirokamitai/trenchlesspm-cache`. Holds `hf/` (HF Hub cache) and `torch/` (Torch hub cache). Not in this repo. |
81
 
82
  ## External touchpoints
83
 
84
+ - **Hugging Face Hub** β€” marker pulls its layout / OCR models from the Hub on
85
+ first cold start; cached to `/data/hf` (HF Bucket mount) thereafter. No auth
86
+ needed for public models.
87
  - **Anthropic API (Claude)** β€” only called when `parse_mode=quality`. Marker
88
  uses its built-in `ClaudeService`; we pin `claude-sonnet-4-6`. API key
89
  stored as a Space secret. See [DEPENDENCIES.md](DEPENDENCIES.md).
 
93
  - **Cold start is slow.** First request after sleep waits on container boot +
94
  model load from `/data` (~30–60s). This is by design for cost; don't try to
95
  paper over it with a keepalive.
96
+ - **Model cache MUST land on `/data`.** If the bucket isn't mounted, every
97
+ cold start re-downloads ~5 GB of weights into ephemeral container disk
98
+ (lost on the next sleep). The Dockerfile sets `HF_HOME=/data/hf` and
99
+ `TORCH_HOME=/data/torch` so marker writes through the bucket.
100
+ - **Public Space, bearer auth.** The Space URL is reachable without HF auth.
101
+ Real access control is the bearer-token check in `app/main.py` β€” `/parse`
102
+ 401s without `Authorization: Bearer <API_TOKEN>`. `/` and `/health` are
103
+ intentionally open and leak nothing.
104
  - **Package name vs import name.** `pip install marker-pdf`, but `import marker`.
105
  Easy to get wrong.
106
  - **Single-process, GPU-bound.** Don't run multiple Uvicorn workers β€” they'd each
_project/CHANGELOG.md CHANGED
@@ -8,12 +8,19 @@ Newest first. Format: `## YYYY-MM-DD β€” short title`, then bullets.
8
 
9
  ## Unreleased
10
 
11
- - Persistence switched from the older per-Space "Persistent Storage" tier to an HF Bucket (`hirokamitai/trenchlesspm-cache`, private, mounted at `/data`) for the marker model cache. No code changes β€” Dockerfile already pins `HF_HOME`/`TORCH_HOME` under `/data`. See DECISIONS for the cost rationale.
12
- - `/parse` response shape extended with `page_count` and `duration_ms`; uploads stream to a temp file (no longer buffered in RAM) so 100+ MB construction PDFs don't OOM the A10 small. Uvicorn keep-alive bumped to 30 min for long parses. Space autosleep lowered from 15 min β†’ 5 min to halve idle billing. Added `pypdf` as a fallback page-count source when marker's metadata key shifts between versions. Caller for these changes: TrenchlessPM, replacing LlamaParse.
13
- - Initial FastAPI app: `GET /health`, `POST /parse` (multipart upload, bearer-token auth, `parse_mode` toggle for marker's LLM-enhancement flag).
14
- - `quality` mode wired to Claude (Sonnet 4.6 via marker's `ClaudeService`); reads `ANTHROPIC_API_KEY`, optional `CLAUDE_MODEL` override.
15
- - Dockerfile: CUDA-enabled base, marker-pdf installed, `HF_HOME` and `TORCH_HOME` pinned to `/data` so model cache survives cold starts.
16
- - Root `README.md` carries the HF Space frontmatter (Docker SDK, A10 small, 15-min autosleep, persistent storage).
17
- - Project docs filled in (SPEC, ARCHITECTURE, DECISIONS, ENVIRONMENT, DEPENDENCIES, INFRASTRUCTURE, RUNBOOK).
18
 
19
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
 
9
  ## Unreleased
10
 
11
+ - (nothing yet)
 
 
 
 
 
 
12
 
13
  ---
14
+
15
+ ## 2026-04-28 β€” v0 deploy to HF Space
16
+
17
+ - First deploy of marker to `hirokamitai/trenchlesspm` (public, Docker SDK, A10 small, dev-mode off, autosleep 5 min). Push via `git push origin master:main`.
18
+ - HF Bucket `hirokamitai/trenchlesspm-cache` (private, RW) mounted at `/data` for marker's ~5 GB model cache. Replaces the older fixed-fee Persistent Storage tier with pay-per-byte billing. No code change β€” `HF_HOME`/`TORCH_HOME` already point at `/data`. See DECISIONS for rationale.
19
+ - `API_TOKEN` set as a Space secret (via `huggingface_hub.add_space_secret`). `ANTHROPIC_API_KEY` still pending β€” `parse_mode=quality` will 400 until set.
20
+ - Verified live: `GET /` 200 (endpoint blurb), `GET /health` 200 with `mode: "gpu"`, `POST /parse` without auth 401.
21
+ - FastAPI app shipped: `GET /` (blurb, public), `GET /health` (liveness, public), `POST /parse` (bearer-gated, multipart, `parse_mode` toggle).
22
+ - `parse_mode=quality` routes through Claude Sonnet 4.6 via marker's `ClaudeService`; reads `ANTHROPIC_API_KEY`, optional `CLAUDE_MODEL` override.
23
+ - `/parse` response shape: `{markdown, mode, page_count, duration_ms}`. Uploads stream to a temp file (no RAM buffering) so 100+ MB construction PDFs don't OOM the A10 small. Uvicorn `--timeout-keep-alive 1800` so long parses don't drop. Added `pypdf` as a fallback page-count source when marker's metadata key shifts between versions. Caller these changes were sized for: TrenchlessPM, replacing LlamaParse.
24
+ - Dockerfile: CUDA 12.4 base, Python 3.11, marker-pdf installed, `HF_HOME=/data/hf` and `TORCH_HOME=/data/torch` so the bucket holds the model cache across cold starts.
25
+ - Root `README.md` carries the HF Space frontmatter (Docker SDK, A10 small, 5-min autosleep, persistent storage suggested).
26
+ - Project docs filled in (SPEC, ARCHITECTURE, DECISIONS, ENVIRONMENT, DEPENDENCIES, INFRASTRUCTURE, RUNBOOK, CURRENT_STATE, TODO, ROADMAP, CHANGELOG).
_project/CURRENT_STATE.md CHANGED
@@ -8,25 +8,37 @@
8
 
9
  ## What's built and working
10
 
11
- - Project docs filled in: SPEC, ARCHITECTURE, DECISIONS, ENVIRONMENT, DEPENDENCIES, INFRASTRUCTURE, RUNBOOK.
12
- - FastAPI app skeleton: `GET /health`, `POST /parse` with bearer-token auth and `parse_mode` (`fast` | `quality`).
13
- - Dockerfile targeting CUDA + marker-pdf, with `HF_HOME` / `TORCH_HOME` pinned to `/data` for persistent model cache.
14
- - HF Space frontmatter in root `README.md` (Docker SDK, A10 small, 15-min autosleep, persistent storage).
 
 
 
 
 
 
15
 
16
  ## What's deployed
17
 
18
- - **Environment:** Hugging Face Space `hirokamitai/trenchlesspm` (Docker SDK, A10 small, persistent `/data`).
19
- - **Version / commit:** see latest push to `origin master:main`.
20
- - **Deployed at:** 2026-04-28 (first deploy).
 
 
 
 
 
21
 
22
  ## What's in progress
23
 
24
- - First push to the Space; verifying cold-start + model cache behavior on real GPU.
25
- - Setting `API_TOKEN` and `ANTHROPIC_API_KEY` as Space secrets.
 
26
 
27
  ## What's known broken / flaky
28
 
29
- - Nothing observed yet β€” nothing has been deployed.
30
 
31
  ## Half-finished or abandoned
32
 
 
8
 
9
  ## What's built and working
10
 
11
+ - FastAPI app with three routes:
12
+ - `GET /` β€” endpoint blurb (public, no auth).
13
+ - `GET /health` β€” liveness + reports `gpu`/`cpu` (public, no auth).
14
+ - `POST /parse` β€” multipart upload β†’ `{markdown, mode, page_count, duration_ms}` (bearer-gated).
15
+ - Bearer-token auth via `Authorization: Bearer <API_TOKEN>`; constant-time compare.
16
+ - Streamed uploads to a temp file (1 MiB chunks) β€” handles 100+ MB PDFs without OOM.
17
+ - `parse_mode=fast` runs marker-pdf alone; `parse_mode=quality` adds Claude Sonnet 4.6 LLM-enhancement via marker's `ClaudeService`.
18
+ - Page count from marker's rendered metadata, falling back to `pypdf`; duration timed with `time.perf_counter()`.
19
+ - Dockerfile (CUDA 12.4 base, Python 3.11) with `HF_HOME` and `TORCH_HOME` pinned to `/data` so marker's ~5 GB of model weights persist via the HF Bucket mount.
20
+ - Uvicorn `--timeout-keep-alive 1800` so long parses don't drop.
21
 
22
  ## What's deployed
23
 
24
+ - **Environment:** Hugging Face Space `hirokamitai/trenchlesspm` β€” public, Docker SDK, A10 small, dev-mode off, autosleep 5 min.
25
+ - **Persistence:** HF Bucket `hirokamitai/trenchlesspm-cache` (private, RW) mounted at `/data`.
26
+ - **Secrets set:** `API_TOKEN` βœ“. `ANTHROPIC_API_KEY` is **not yet set** β€” so `parse_mode=quality` will 400 until it is.
27
+ - **Verified live (2026-04-28):**
28
+ - `GET /` β†’ 200, returns endpoint blurb.
29
+ - `GET /health` β†’ 200, `{"status":"ok","mode":"gpu"}` (GPU detected).
30
+ - `POST /parse` without auth β†’ 401 (auth enforced).
31
+ - **Version / commit:** see latest `origin/main`. End-to-end PDF smoke test still pending (next item below).
32
 
33
  ## What's in progress
34
 
35
+ - Set `ANTHROPIC_API_KEY` on the Space (only blocks `parse_mode=quality`).
36
+ - End-to-end smoke test: upload a real contract PDF, verify both modes return sensible markdown and that cold start after autosleep loads weights from the bucket (no 5 GB re-download).
37
+ - Wire TrenchlessPM (`/Users/michael/Documents/apps/web/TrenchlessPM`) to call `/parse`, replacing LlamaParse.
38
 
39
  ## What's known broken / flaky
40
 
41
+ - Nothing observed yet at runtime.
42
 
43
  ## Half-finished or abandoned
44
 
_project/DECISIONS.md CHANGED
@@ -27,6 +27,30 @@ Write decisions where someone smart would reasonably pick differently.
27
 
28
  ## Entries
29
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
30
  ## 2026-04-28 β€” Claude Sonnet 4.6 for marker's LLM-enhancement pass
31
 
32
  **Decision:** point marker's `quality` mode at `marker.services.claude.ClaudeService`
 
27
 
28
  ## Entries
29
 
30
+ ## 2026-04-28 β€” Public Space, bearer-token auth (not HF private + HF token)
31
+
32
+ **Decision:** the Space is **public**; access control to `/parse` is the
33
+ single shared bearer token in `app/main.py` (`API_TOKEN`, set as a Space
34
+ secret). `/` and `/health` are intentionally open and leak nothing.
35
+ **Context:** TrenchlessPM and any other internal caller needs to invoke
36
+ `/parse` from non-interactive code. HF Spaces gate private Spaces behind
37
+ the caller's HF user token, on top of any app-level auth.
38
+ **Alternatives considered:** keep the Space private and require callers
39
+ to send `Authorization: Bearer <HF_USER_TOKEN>` *plus* our app-level
40
+ `API_TOKEN`. Workable, but doubles the number of secrets each integrator
41
+ manages, and HF user tokens are coarse-grained (they grant access to the
42
+ whole HF account, not just this Space) β€” leaking one is a much bigger
43
+ incident than leaking the marker bearer.
44
+ **Tradeoffs:** the Space URL is publicly discoverable. Anyone can hit it,
45
+ and unauthenticated `/parse` calls will burn a tiny bit of edge bandwidth
46
+ returning 401. Acceptable β€” a 256-bit shared secret is the actual
47
+ enforcement boundary, and HF's edge handles abuse.
48
+ **Revisit if:** (a) we need fine-grained per-caller revocation (right now
49
+ rotating the single token forces every caller to update), (b) compliance
50
+ ever requires that the URL itself not be discoverable, or (c) we add
51
+ endpoints that are unsafe to expose at all (in which case route-level
52
+ auth is the answer, not Space-level).
53
+
54
  ## 2026-04-28 β€” Claude Sonnet 4.6 for marker's LLM-enhancement pass
55
 
56
  **Decision:** point marker's `quality` mode at `marker.services.claude.ClaudeService`
_project/ENVIRONMENT.md CHANGED
@@ -12,7 +12,7 @@ do **not** live in this file β€” only pointers to them.
12
 
13
  | Name | Purpose | Where the real value lives | Used by |
14
  |---|---|---|---|
15
- | `API_TOKEN` | Bearer token clients send in `Authorization: Bearer …` to call `/parse`. Single shared secret, no per-user tokens. | 1Password > marker > API_TOKEN | `app/main.py` |
16
 
17
  ## Optional
18
 
@@ -20,7 +20,7 @@ do **not** live in this file β€” only pointers to them.
20
  |---|---|---|---|
21
  | `HF_HOME` | Where the Hugging Face client caches model weights. Pinned to persistent storage so cold starts don't re-download ~5 GB. | `/data/hf` (set in Dockerfile) | n/a β€” set in image |
22
  | `TORCH_HOME` | Where Torch hub caches checkpoints. Same reason as `HF_HOME`. | `/data/torch` (set in Dockerfile) | n/a β€” set in image |
23
- | `ANTHROPIC_API_KEY` | Used by marker's LLM-enhancement path (Claude) when `parse_mode=quality`. Not needed if you never call quality mode. | 1Password > marker > ANTHROPIC_API_KEY | `app/parser.py` |
24
  | `CLAUDE_MODEL` | Claude model ID used for quality mode. Override only if you want a different speed/cost/quality tradeoff. | `claude-sonnet-4-6` | n/a |
25
  | `LOG_LEVEL` | Uvicorn log level. | `info` | n/a |
26
  | `PORT` | Port FastAPI binds to. HF Spaces convention is 7860. | `7860` | n/a |
@@ -28,7 +28,7 @@ do **not** live in this file β€” only pointers to them.
28
  ## Where env vars are loaded
29
 
30
  - **Local:** `.env` file at project root, loaded by `python-dotenv` from `app/main.py` on startup. Only used for development; production reads from the Space's secret store directly.
31
- - **Production (HF Space):** set in the Space's **Settings β†’ Variables and secrets** panel. `API_TOKEN` and `ANTHROPIC_API_KEY` go in **Secrets**; `HF_HOME` / `TORCH_HOME` are baked into the Dockerfile and don't need to be set here.
32
 
33
  ## Rotation notes
34
 
 
12
 
13
  | Name | Purpose | Where the real value lives | Used by |
14
  |---|---|---|---|
15
+ | `API_TOKEN` | Bearer token clients send in `Authorization: Bearer …` to call `/parse`. Single shared secret, no per-user tokens. **Set on the Space as of 2026-04-28.** | 1Password > marker > API_TOKEN | `app/main.py` |
16
 
17
  ## Optional
18
 
 
20
  |---|---|---|---|
21
  | `HF_HOME` | Where the Hugging Face client caches model weights. Pinned to persistent storage so cold starts don't re-download ~5 GB. | `/data/hf` (set in Dockerfile) | n/a β€” set in image |
22
  | `TORCH_HOME` | Where Torch hub caches checkpoints. Same reason as `HF_HOME`. | `/data/torch` (set in Dockerfile) | n/a β€” set in image |
23
+ | `ANTHROPIC_API_KEY` | Used by marker's LLM-enhancement path (Claude) when `parse_mode=quality`. Not needed if you never call quality mode. **Not yet set on the Space as of 2026-04-28** β€” `parse_mode=quality` will 400 until it is. | 1Password > marker > ANTHROPIC_API_KEY | `app/parser.py` |
24
  | `CLAUDE_MODEL` | Claude model ID used for quality mode. Override only if you want a different speed/cost/quality tradeoff. | `claude-sonnet-4-6` | n/a |
25
  | `LOG_LEVEL` | Uvicorn log level. | `info` | n/a |
26
  | `PORT` | Port FastAPI binds to. HF Spaces convention is 7860. | `7860` | n/a |
 
28
  ## Where env vars are loaded
29
 
30
  - **Local:** `.env` file at project root, loaded by `python-dotenv` from `app/main.py` on startup. Only used for development; production reads from the Space's secret store directly.
31
+ - **Production (HF Space):** set in the Space's **Settings β†’ Variables and secrets** panel, or programmatically via `huggingface_hub.HfApi().add_space_secret(...)`. `API_TOKEN` and `ANTHROPIC_API_KEY` go in **Secrets**; `HF_HOME` / `TORCH_HOME` are baked into the Dockerfile and don't need to be set here. See [RUNBOOK.md](RUNBOOK.md) for the API-based recipe (avoids putting the secret on the bash command line / in shell history).
32
 
33
  ## Rotation notes
34
 
_project/INFRASTRUCTURE.md CHANGED
@@ -19,14 +19,16 @@ and the `hf` CLI / git push to the Space repo.
19
  - **Provider:** Hugging Face Spaces
20
  - **Host name / label:** `hirokamitai/trenchlesspm`
21
  - **URL:** `https://hirokamitai-trenchlesspm.hf.space`
 
22
  - **Region:** US (HF default for GPU Spaces)
23
  - **Size / tier:** A10 small (1Γ— NVIDIA A10G, 24 GB VRAM)
24
  - **SDK:** Docker
 
25
  - **Persistent storage:** HF Bucket `hirokamitai/trenchlesspm-cache` (private, read+write) mounted at `/data`. Holds the marker model cache (`/data/hf`, `/data/torch`) so cold starts skip the ~5 GB model download.
26
  - **Sleep behavior:** autosleep after 5 min idle (`sleep_time: 300` in README frontmatter)
27
  - **Monthly cost:** A10 small billed by the second when awake (~$1.05/hr) + bucket storage (pay-per-byte, ~5 GB of marker weights). Target total <$5–10/mo at expected usage.
28
  - **Paid with:** card on file at huggingface.co/settings/billing
29
- - **Provisioned on:** pre-existing (Space was scaffolded from the sappcode template)
30
 
31
  ## How to access
32
 
 
19
  - **Provider:** Hugging Face Spaces
20
  - **Host name / label:** `hirokamitai/trenchlesspm`
21
  - **URL:** `https://hirokamitai-trenchlesspm.hf.space`
22
+ - **Visibility:** **public** (real access control is the bearer-token check in `app/main.py`; see [DECISIONS.md](DECISIONS.md))
23
  - **Region:** US (HF default for GPU Spaces)
24
  - **Size / tier:** A10 small (1Γ— NVIDIA A10G, 24 GB VRAM)
25
  - **SDK:** Docker
26
+ - **Dev mode:** off (must stay off β€” when on, HF routes the public URL to a dev IDE and every endpoint 404s)
27
  - **Persistent storage:** HF Bucket `hirokamitai/trenchlesspm-cache` (private, read+write) mounted at `/data`. Holds the marker model cache (`/data/hf`, `/data/torch`) so cold starts skip the ~5 GB model download.
28
  - **Sleep behavior:** autosleep after 5 min idle (`sleep_time: 300` in README frontmatter)
29
  - **Monthly cost:** A10 small billed by the second when awake (~$1.05/hr) + bucket storage (pay-per-byte, ~5 GB of marker weights). Target total <$5–10/mo at expected usage.
30
  - **Paid with:** card on file at huggingface.co/settings/billing
31
+ - **First deploy:** 2026-04-28
32
 
33
  ## How to access
34
 
_project/ROADMAP.md CHANGED
@@ -13,23 +13,23 @@ For why specific choices were made, see [DECISIONS.md](DECISIONS.md).
13
 
14
  ## Now (this month)
15
 
16
- - **<!-- FILL IN: theme -->** β€” <!-- one-line description. Success looks like: ... -->
17
- -
 
18
 
19
  ## Next (1–3 months)
20
 
21
- - **<!-- FILL IN: theme -->** β€” <!-- one-line description -->
22
- -
23
 
24
  ## Later (3–6+ months)
25
 
26
- - **<!-- FILL IN: theme -->** β€” <!-- one-line description -->
27
- -
28
 
29
  ## Considered, not doing
30
 
31
- Things actively evaluated and rejected. Saves you (and Claude) from
32
- re-litigating them. One-line reason each.
33
-
34
- - <!-- FILL IN: e.g., "Custom code-reviewer subagent β€” Claude Code's /review already covers it." -->
35
- -
 
13
 
14
  ## Now (this month)
15
 
16
+ - **Cut TrenchlessPM over from LlamaParse to marker** β€” finish the integration on the TrenchlessPM side and run the first real contract through `/parse`. Success: parses match or beat LlamaParse Premium quality at <$5/mo bill.
17
+ - **Verify cold-start cache hit** β€” confirm that after autosleep, the first wake reads weights from the bucket (no 5 GB redownload in the Logs tab). Success: cold-start latency settles around 30–60 s.
18
+ - **Set `ANTHROPIC_API_KEY`** so `parse_mode=quality` is usable. Success: a real contract round-trips through Sonnet 4.6 cleanup.
19
 
20
  ## Next (1–3 months)
21
 
22
+ - **Hash-keyed result caching** β€” skip the GPU when TrenchlessPM retries the same PDF. Probably an in-memory LRU or a small SQLite at `/data/cache.db`.
23
+ - **Operability** β€” wire the Space into the sapplab monitor dashboard via `.monitor.yml`; add a `429`-aware retry on the TrenchlessPM client side.
24
 
25
  ## Later (3–6+ months)
26
 
27
+ - **Multi-PDF batch endpoint** β€” only if a real workflow asks for it; sync request model is fine for v1.
28
+ - **Audit / replay** β€” externalize input+output to an R2/S3 bucket if a downstream consumer ever needs to dispute "what did marker return for this PDF?"
29
 
30
  ## Considered, not doing
31
 
32
+ - **Mounting an external S3/R2/GCS bucket for inputs/outputs** β€” see [DECISIONS.md](DECISIONS.md) "No external bucket"; speculative for v1.
33
+ - **Gradio UI on top of `/parse`** β€” JSON API only is a goal, not a v1 omission; see [SPEC.md](SPEC.md).
34
+ - **Multiple Uvicorn workers** β€” would OOM the GPU; concurrency comes from Space scaling, not in-process workers.
35
+ - **Local LLM for `quality` mode** β€” A10's VRAM is for marker; the LLM call stays external.
 
_project/RUNBOOK.md CHANGED
@@ -34,13 +34,16 @@ uvicorn app.main:app --host 0.0.0.0 --port 7860 --reload
34
  Smoke test:
35
 
36
  ```bash
 
 
 
37
  curl -s http://localhost:7860/health
38
  # => {"status":"ok","mode":"cpu"}
39
 
40
  curl -s -X POST http://localhost:7860/parse \
41
  -H "Authorization: Bearer $API_TOKEN" \
42
  -F "file=@sample.pdf" \
43
- -F "parse_mode=fast" | jq -r '.markdown' | head
44
  ```
45
 
46
  ### Run tests
@@ -61,12 +64,28 @@ The Space repo is wired up as this repo's `origin` remote β€” local branch is
61
  `master`, the Space's default branch is `main`, so all pushes are
62
  `master:main`.
63
 
64
- ### Set Space secrets (one-time, via web UI)
 
 
65
 
66
- Go to https://huggingface.co/spaces/hirokamitai/trenchlesspm/settings and set:
 
 
 
 
 
 
 
 
 
 
 
 
67
 
68
- - `API_TOKEN` β€” bearer token callers must send.
69
- - `ANTHROPIC_API_KEY` β€” only needed for `parse_mode=quality`.
 
 
70
 
71
  ### Deploy a change
72
 
@@ -103,10 +122,26 @@ curl -fsS https://hirokamitai-trenchlesspm.hf.space/health
103
  If it's been idle >5 min the first request triggers a cold start (~30–60 s
104
  wait). That's by design.
105
 
106
- ### Restart
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
107
 
108
- The Space web UI: **Settings β†’ Factory rebuild** (rebuilds image) or
109
- **Restart this Space** (just restarts the container). No CLI equivalent.
110
 
111
  ### Tail logs
112
 
@@ -124,9 +159,10 @@ container boots and marker loads model weights from `/data`.
124
 
125
  When something's broken, try these in order:
126
 
127
- 1. **Health check.** `curl https://hirokamitai-trenchlesspm.hf.space/health`. If this fails, the container is down or sleeping β€” check the Space's status badge on the web UI.
128
- 2. **Tail the Space logs** in the web UI's Logs tab. Look for Python tracebacks, OOMs ("CUDA out of memory"), or marker model-load errors.
129
- 3. **401 from `/parse`?** Verify the caller is sending `Authorization: Bearer <API_TOKEN>` and that the token matches the `API_TOKEN` Space secret. Re-set the secret if in doubt.
130
- 4. **Quality mode failing but fast mode works?** Almost certainly `ANTHROPIC_API_KEY` is missing/expired or you're rate-limited. Check the Space secrets and the Anthropic console for usage/quota.
131
- 5. **Cold starts re-download 5 GB every time?** The HF Bucket (`hirokamitai/trenchlesspm-cache`) isn't mounted, or `HF_HOME`/`TORCH_HOME` aren't pointing at `/data`. Verify the bucket is attached at `/data` in the Space's Storage Buckets settings and the Dockerfile's `ENV` lines are intact.
132
- 6. **Cross-reference [DEPENDENCIES.md](DEPENDENCIES.md)** for status pages of HF and Anthropic if multiple things are broken at once.
 
 
34
  Smoke test:
35
 
36
  ```bash
37
+ curl -s http://localhost:7860/
38
+ # => {"service":"marker","endpoints":{...}}
39
+
40
  curl -s http://localhost:7860/health
41
  # => {"status":"ok","mode":"cpu"}
42
 
43
  curl -s -X POST http://localhost:7860/parse \
44
  -H "Authorization: Bearer $API_TOKEN" \
45
  -F "file=@sample.pdf" \
46
+ -F "parse_mode=fast" | jq '{mode, page_count, duration_ms, head: (.markdown[:200])}'
47
  ```
48
 
49
  ### Run tests
 
64
  `master`, the Space's default branch is `main`, so all pushes are
65
  `master:main`.
66
 
67
+ ### Set Space secrets
68
+
69
+ Either via the web UI (https://huggingface.co/spaces/hirokamitai/trenchlesspm/settings β†’ Variables and secrets) or via the `huggingface_hub` Python API. **Never paste a secret on the bash command line directly** β€” pass it via env var so it doesn't land in shell history. Example:
70
 
71
+ ```bash
72
+ HF_PY=/opt/homebrew/Cellar/hf/*/libexec/bin/python # bundled with `brew install hf`
73
+ SECRET_VALUE='<paste here>' $HF_PY -c '
74
+ import os
75
+ from huggingface_hub import HfApi
76
+ HfApi().add_space_secret(
77
+ repo_id="hirokamitai/trenchlesspm",
78
+ key="ANTHROPIC_API_KEY", # or API_TOKEN
79
+ value=os.environ["SECRET_VALUE"],
80
+ )
81
+ '
82
+ unset SECRET_VALUE
83
+ ```
84
 
85
+ Required:
86
+
87
+ - `API_TOKEN` β€” bearer token callers must send. **Already set** as of 2026-04-28.
88
+ - `ANTHROPIC_API_KEY` β€” only needed for `parse_mode=quality`. **Pending** as of 2026-04-28.
89
 
90
  ### Deploy a change
91
 
 
122
  If it's been idle >5 min the first request triggers a cold start (~30–60 s
123
  wait). That's by design.
124
 
125
+ ### Check Space metadata (stage, dev mode, hardware, sha)
126
+
127
+ ```bash
128
+ hf spaces info hirokamitai/trenchlesspm
129
+ ```
130
+
131
+ Look at `runtime.stage` (`RUNNING` / `BUILDING` / `STOPPED` / `SLEEPING`) and
132
+ `runtime.raw.devMode`. **Dev mode must be `false` for the public URL to route
133
+ to FastAPI** β€” when it's `true`, HF routes the URL to a dev IDE and every
134
+ endpoint 404s (you'll see HF's HTML 404 page, not our JSON).
135
+
136
+ ### Restart / rebuild
137
+
138
+ ```bash
139
+ # Toggle dev mode off (also forces a restart):
140
+ hf spaces dev-mode hirokamitai/trenchlesspm --stop
141
+ ```
142
 
143
+ Web UI alternatives: **Settings β†’ Factory rebuild** (rebuilds image) or
144
+ **Restart this Space** (just restarts the container).
145
 
146
  ### Tail logs
147
 
 
159
 
160
  When something's broken, try these in order:
161
 
162
+ 1. **Health check.** `curl https://hirokamitai-trenchlesspm.hf.space/health`. If this fails, the container is down, sleeping, or dev-mode is on.
163
+ 2. **HTML 404 (not our JSON 404) on every endpoint?** `hf spaces info hirokamitai/trenchlesspm` β€” check `runtime.raw.devMode`. If `true`, run `hf spaces dev-mode hirokamitai/trenchlesspm --stop` (or toggle in the UI). Same symptom can also mean the Space is private and the caller has no HF auth β€” the Space should be public.
164
+ 3. **Tail the Space logs** in the web UI's Logs tab. Look for Python tracebacks, OOMs ("CUDA out of memory"), or marker model-load errors.
165
+ 4. **401 from `/parse`?** Verify the caller is sending `Authorization: Bearer <API_TOKEN>` and that the token matches the `API_TOKEN` Space secret. Re-set the secret if in doubt.
166
+ 5. **Quality mode failing but fast mode works?** Almost certainly `ANTHROPIC_API_KEY` is missing/expired or you're rate-limited. Check the Space secrets and the Anthropic console for usage/quota.
167
+ 6. **Cold starts re-download 5 GB every time?** The HF Bucket (`hirokamitai/trenchlesspm-cache`) isn't mounted, or `HF_HOME`/`TORCH_HOME` aren't pointing at `/data`. Verify the bucket is attached at `/data` in the Space's Storage Buckets settings and the Dockerfile's `ENV` lines are intact.
168
+ 7. **Cross-reference [DEPENDENCIES.md](DEPENDENCIES.md)** for status pages of HF and Anthropic if multiple things are broken at once.
_project/SPEC.md CHANGED
@@ -56,7 +56,7 @@ Not public, not multi-tenant.
56
 
57
  - Budget: pay-per-second GPU billing on A10 small. Target <$5/mo at expected volume.
58
  - Time: weekend build. Don't over-engineer.
59
- - Platform: Hugging Face Space, Docker SDK, A10 small GPU, persistent storage enabled
60
- and mounted at `/data`.
61
- - Other: model cache must live on `/data` (HF persistent storage), not the ephemeral
62
- container FS. Space configured to sleep after 15 min idle.
 
56
 
57
  - Budget: pay-per-second GPU billing on A10 small. Target <$5/mo at expected volume.
58
  - Time: weekend build. Don't over-engineer.
59
+ - Platform: Hugging Face Space, Docker SDK, A10 small GPU, HF Bucket
60
+ (`hirokamitai/trenchlesspm-cache`) mounted at `/data`.
61
+ - Other: model cache must live on `/data` (the HF Bucket mount), not the
62
+ ephemeral container FS. Space configured to sleep after 5 min idle.
_project/TODO.md CHANGED
@@ -9,10 +9,10 @@ For "what's broken," also see CURRENT_STATE.md's known-issues section.
9
 
10
  ## Next up (doing soon, in order)
11
 
12
- 1. Set `API_TOKEN` and `ANTHROPIC_API_KEY` as secrets on the `hirokamitai/trenchlesspm` Space (https://huggingface.co/spaces/hirokamitai/trenchlesspm/settings).
13
- 2. Watch the Logs tab on the Space for the first Docker build (~5–10 min on first push).
14
- 3. Hit `/health`, then run an end-to-end `/parse` on a sample contract in both `fast` and `quality` modes.
15
- 4. Confirm cold starts after autosleep load weights from `/data` (no 5 GB re-download).
16
 
17
  ## Backlog (not prioritized)
18
 
@@ -23,9 +23,9 @@ For "what's broken," also see CURRENT_STATE.md's known-issues section.
23
 
24
  ## Ideas / maybe
25
 
26
- - Audit-trail bucket (deferred β€” see [DECISIONS.md](DECISIONS.md) 2026-04-28 entry). Revisit if a downstream caller ever needs replay.
27
  - Multi-PDF batch endpoint, if a real workflow shows up that wants it.
28
- - Swap the LLM model used for `quality` mode (e.g. bump to Opus 4.7, or try Haiku 4.5 for cheaper) if Sonnet 4.6 output disappoints on contracts. Set via `CLAUDE_MODEL` env var, no code change.
29
 
30
  ---
31
 
@@ -33,6 +33,12 @@ For "what's broken," also see CURRENT_STATE.md's known-issues section.
33
 
34
  Keep the last ~10. Older items belong in [CHANGELOG.md](CHANGELOG.md).
35
 
 
 
 
 
 
 
36
  - 2026-04-28 β€” Filled in `_project/` docs (SPEC, ARCHITECTURE, DECISIONS, ENVIRONMENT, DEPENDENCIES, INFRASTRUCTURE, RUNBOOK).
37
  - 2026-04-28 β€” Installed `hf` CLI + the `hf-cli` skill globally for Claude Code.
38
  - 2026-04-28 β€” Scaffolded FastAPI app, Dockerfile, and HF Space README frontmatter.
 
9
 
10
  ## Next up (doing soon, in order)
11
 
12
+ 1. Set `ANTHROPIC_API_KEY` on the Space (https://huggingface.co/spaces/hirokamitai/trenchlesspm/settings β†’ Secrets). Required for `parse_mode=quality`.
13
+ 2. End-to-end smoke test: upload a real contract PDF and verify `{markdown, mode, page_count, duration_ms}` for both `fast` and `quality`.
14
+ 3. After the first parse, sleep the Space (or wait 5 min) and confirm cold start loads weights from the bucket β€” no 5 GB re-download in the Logs tab.
15
+ 4. Wire TrenchlessPM to call `/parse`, replacing LlamaParse. Document the integration on the TrenchlessPM side.
16
 
17
  ## Backlog (not prioritized)
18
 
 
23
 
24
  ## Ideas / maybe
25
 
26
+ - External audit-trail bucket (deferred β€” see [DECISIONS.md](DECISIONS.md) "No external bucket"). Revisit if a downstream caller ever needs replay.
27
  - Multi-PDF batch endpoint, if a real workflow shows up that wants it.
28
+ - Swap the Claude model used for `quality` mode (e.g. bump to Opus 4.7, or try Haiku 4.5 for cheaper) if Sonnet 4.6 output disappoints on contracts. Set via `CLAUDE_MODEL` env var, no code change.
29
 
30
  ---
31
 
 
33
 
34
  Keep the last ~10. Older items belong in [CHANGELOG.md](CHANGELOG.md).
35
 
36
+ - 2026-04-28 β€” Deployed to `hirokamitai/trenchlesspm`. Public, dev-mode off, A10 small running. `/`, `/health`, `/parse`-401 verified live.
37
+ - 2026-04-28 β€” Switched persistence to HF Bucket `hirokamitai/trenchlesspm-cache` mounted at `/data`. Cheaper than the older Persistent Storage tier.
38
+ - 2026-04-28 β€” Set `API_TOKEN` as a Space secret via `huggingface_hub.add_space_secret`.
39
+ - 2026-04-28 β€” Added `GET /` route so the Space URL doesn't 404 in a browser.
40
+ - 2026-04-28 β€” Wired `parse_mode=quality` to Claude Sonnet 4.6 via marker's `ClaudeService` (was Gemini default).
41
+ - 2026-04-28 β€” Streamed uploads to disk, added `page_count` + `duration_ms` to the `/parse` response, raised Uvicorn keep-alive to 30 min, lowered autosleep to 5 min.
42
  - 2026-04-28 β€” Filled in `_project/` docs (SPEC, ARCHITECTURE, DECISIONS, ENVIRONMENT, DEPENDENCIES, INFRASTRUCTURE, RUNBOOK).
43
  - 2026-04-28 β€” Installed `hf` CLI + the `hf-cli` skill globally for Claude Code.
44
  - 2026-04-28 β€” Scaffolded FastAPI app, Dockerfile, and HF Space README frontmatter.