Spaces:
Sleeping
Sync _project docs to deployed reality
Browse files- ARCHITECTURE: root route, parse() signature change, response shape, bucket
mount, 5-min sleep, public-Space sharp edge.
- CURRENT_STATE: deployed Space verified live; remaining task is
ANTHROPIC_API_KEY + first real PDF.
- CHANGELOG: dated v0 deploy entry; consolidated Unreleased into shipped.
- RUNBOOK: secret-setting recipe via huggingface_hub; dev-mode debug step;
hf spaces info command.
- INFRASTRUCTURE: public visibility, dev-mode off, deploy date.
- ENVIRONMENT: marks API_TOKEN as set, ANTHROPIC_API_KEY as pending.
- DECISIONS: new entry for public-Space + bearer-token auth model.
- ROADMAP: filled in (was placeholder).
- TODO: deploy items moved to Done; ANTHROPIC_API_KEY surfaced as next.
- SPEC: bucket replaces persistent-storage tier; sleep 15β5 min.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- _project/ARCHITECTURE.md +47 -29
- _project/CHANGELOG.md +14 -7
- _project/CURRENT_STATE.md +22 -10
- _project/DECISIONS.md +24 -0
- _project/ENVIRONMENT.md +3 -3
- _project/INFRASTRUCTURE.md +3 -1
- _project/ROADMAP.md +11 -11
- _project/RUNBOOK.md +50 -14
- _project/SPEC.md +4 -4
- _project/TODO.md +12 -6
|
@@ -17,25 +17,30 @@
|
|
| 17 |
- **Frontend:** none. JSON API only.
|
| 18 |
- **Build / package manager:** `uv` for dependency resolution; `pip` inside the
|
| 19 |
Docker image (smaller layer than installing `uv` into the runtime).
|
| 20 |
-
- **Hosting:** Hugging Face Space, **
|
| 21 |
-
|
|
|
|
| 22 |
|
| 23 |
## Components
|
| 24 |
|
| 25 |
-
- **`app/main.py`** β FastAPI app. Exposes `GET /
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
## Data flow
|
| 41 |
|
|
@@ -45,18 +50,26 @@
|
|
| 45 |
file: the PDF
|
| 46 |
parse_mode: "fast" (default) | "quality"
|
| 47 |
|
| 48 |
-
2. FastAPI handler validates the bearer token + form fields.
|
|
|
|
| 49 |
|
| 50 |
-
3. Handler
|
| 51 |
|
| 52 |
4. parser.parse runs marker-pdf on the GPU. On "quality" mode it also calls out
|
| 53 |
to Claude (Sonnet 4.6 by default) via marker's `ClaudeService` for layout,
|
| 54 |
table, and equation cleanup. See _project/ENVIRONMENT.md for env vars.
|
| 55 |
|
| 56 |
-
5. Handler returns JSON:
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
```
|
| 61 |
|
| 62 |
## Key directories
|
|
@@ -64,12 +77,13 @@
|
|
| 64 |
| Path | Purpose |
|
| 65 |
|---|---|
|
| 66 |
| `app/` | FastAPI app code (`main.py`, `parser.py`). |
|
| 67 |
-
| `/data/` (runtime, on the Space) | HF
|
| 68 |
|
| 69 |
## External touchpoints
|
| 70 |
|
| 71 |
-
- **Hugging Face Hub** β marker pulls its layout / OCR models from the Hub on
|
| 72 |
-
cold start; cached to `/data/hf` thereafter. No auth
|
|
|
|
| 73 |
- **Anthropic API (Claude)** β only called when `parse_mode=quality`. Marker
|
| 74 |
uses its built-in `ClaudeService`; we pin `claude-sonnet-4-6`. API key
|
| 75 |
stored as a Space secret. See [DEPENDENCIES.md](DEPENDENCIES.md).
|
|
@@ -79,10 +93,14 @@
|
|
| 79 |
- **Cold start is slow.** First request after sleep waits on container boot +
|
| 80 |
model load from `/data` (~30β60s). This is by design for cost; don't try to
|
| 81 |
paper over it with a keepalive.
|
| 82 |
-
- **Model cache MUST land on `/data`.**
|
| 83 |
-
start re-downloads ~5 GB of weights
|
| 84 |
-
|
| 85 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 86 |
- **Package name vs import name.** `pip install marker-pdf`, but `import marker`.
|
| 87 |
Easy to get wrong.
|
| 88 |
- **Single-process, GPU-bound.** Don't run multiple Uvicorn workers β they'd each
|
|
|
|
| 17 |
- **Frontend:** none. JSON API only.
|
| 18 |
- **Build / package manager:** `uv` for dependency resolution; `pip` inside the
|
| 19 |
Docker image (smaller layer than installing `uv` into the runtime).
|
| 20 |
+
- **Hosting:** Hugging Face Space (`hirokamitai/trenchlesspm`, **public**),
|
| 21 |
+
Docker SDK, A10 small GPU, HF Bucket `hirokamitai/trenchlesspm-cache`
|
| 22 |
+
mounted at `/data` for the model cache.
|
| 23 |
|
| 24 |
## Components
|
| 25 |
|
| 26 |
+
- **`app/main.py`** β FastAPI app. Exposes `GET /` (endpoint blurb, public),
|
| 27 |
+
`GET /health` (liveness + gpu/cpu, public), and `POST /parse` (bearer-gated).
|
| 28 |
+
Bearer-token auth via `Authorization: Bearer <secret>`; secret comes from the
|
| 29 |
+
`API_TOKEN` Space secret. Streams uploads to a temp file (no RAM buffering).
|
| 30 |
+
- **`app/parser.py`** β wraps marker-pdf. Loads the marker model dict once at
|
| 31 |
+
import time (module-level singleton) so subsequent requests reuse weights on
|
| 32 |
+
GPU. Exposes `parse(pdf_path, mode) -> tuple[str, int]` returning
|
| 33 |
+
`(markdown, page_count)`. `mode="quality"` flips marker's LLM-enhancement
|
| 34 |
+
flag on and routes through `marker.services.claude.ClaudeService`
|
| 35 |
+
(defaults to `claude-sonnet-4-6`).
|
| 36 |
+
- **`Dockerfile`** β CUDA-enabled base image, installs system deps marker
|
| 37 |
+
needs, `pip install` of marker-pdf + FastAPI stack, sets `HF_HOME=/data/hf`
|
| 38 |
+
and `TORCH_HOME=/data/torch` so model weights persist on the bucket. CMD
|
| 39 |
+
runs Uvicorn with `--timeout-keep-alive 1800` so long parses don't drop
|
| 40 |
+
the connection. Exposes port 7860 (HF Space convention).
|
| 41 |
+
- **`README.md`** (Space root) β HF Space frontmatter (`sdk: docker`,
|
| 42 |
+
`app_port: 7860`, `sleep_time: 300`, `suggested_hardware: a10g-small`,
|
| 43 |
+
`suggested_storage: small`), plus a short usage blurb.
|
| 44 |
|
| 45 |
## Data flow
|
| 46 |
|
|
|
|
| 50 |
file: the PDF
|
| 51 |
parse_mode: "fast" (default) | "quality"
|
| 52 |
|
| 53 |
+
2. FastAPI handler validates the bearer token + form fields. Upload streams
|
| 54 |
+
in 1 MiB chunks to a NamedTemporaryFile so 100+ MB PDFs don't OOM.
|
| 55 |
|
| 56 |
+
3. Handler times parser.parse(tmp_path, mode) with time.perf_counter().
|
| 57 |
|
| 58 |
4. parser.parse runs marker-pdf on the GPU. On "quality" mode it also calls out
|
| 59 |
to Claude (Sonnet 4.6 by default) via marker's `ClaudeService` for layout,
|
| 60 |
table, and equation cleanup. See _project/ENVIRONMENT.md for env vars.
|
| 61 |
|
| 62 |
+
5. Handler returns JSON:
|
| 63 |
+
{
|
| 64 |
+
"markdown": "...",
|
| 65 |
+
"mode": "fast"|"quality",
|
| 66 |
+
"page_count": <int>,
|
| 67 |
+
"duration_ms": <int>
|
| 68 |
+
}
|
| 69 |
+
page_count comes from marker's rendered.metadata, falling back to pypdf.
|
| 70 |
+
|
| 71 |
+
6. Space autosleeps after 5 min of no requests. Next request triggers a cold
|
| 72 |
+
start; model weights load from /data (HF Bucket) instead of re-downloading.
|
| 73 |
```
|
| 74 |
|
| 75 |
## Key directories
|
|
|
|
| 77 |
| Path | Purpose |
|
| 78 |
|---|---|
|
| 79 |
| `app/` | FastAPI app code (`main.py`, `parser.py`). |
|
| 80 |
+
| `/data/` (runtime, on the Space) | Mount point for HF Bucket `hirokamitai/trenchlesspm-cache`. Holds `hf/` (HF Hub cache) and `torch/` (Torch hub cache). Not in this repo. |
|
| 81 |
|
| 82 |
## External touchpoints
|
| 83 |
|
| 84 |
+
- **Hugging Face Hub** β marker pulls its layout / OCR models from the Hub on
|
| 85 |
+
first cold start; cached to `/data/hf` (HF Bucket mount) thereafter. No auth
|
| 86 |
+
needed for public models.
|
| 87 |
- **Anthropic API (Claude)** β only called when `parse_mode=quality`. Marker
|
| 88 |
uses its built-in `ClaudeService`; we pin `claude-sonnet-4-6`. API key
|
| 89 |
stored as a Space secret. See [DEPENDENCIES.md](DEPENDENCIES.md).
|
|
|
|
| 93 |
- **Cold start is slow.** First request after sleep waits on container boot +
|
| 94 |
model load from `/data` (~30β60s). This is by design for cost; don't try to
|
| 95 |
paper over it with a keepalive.
|
| 96 |
+
- **Model cache MUST land on `/data`.** If the bucket isn't mounted, every
|
| 97 |
+
cold start re-downloads ~5 GB of weights into ephemeral container disk
|
| 98 |
+
(lost on the next sleep). The Dockerfile sets `HF_HOME=/data/hf` and
|
| 99 |
+
`TORCH_HOME=/data/torch` so marker writes through the bucket.
|
| 100 |
+
- **Public Space, bearer auth.** The Space URL is reachable without HF auth.
|
| 101 |
+
Real access control is the bearer-token check in `app/main.py` β `/parse`
|
| 102 |
+
401s without `Authorization: Bearer <API_TOKEN>`. `/` and `/health` are
|
| 103 |
+
intentionally open and leak nothing.
|
| 104 |
- **Package name vs import name.** `pip install marker-pdf`, but `import marker`.
|
| 105 |
Easy to get wrong.
|
| 106 |
- **Single-process, GPU-bound.** Don't run multiple Uvicorn workers β they'd each
|
|
@@ -8,12 +8,19 @@ Newest first. Format: `## YYYY-MM-DD β short title`, then bullets.
|
|
| 8 |
|
| 9 |
## Unreleased
|
| 10 |
|
| 11 |
-
-
|
| 12 |
-
- `/parse` response shape extended with `page_count` and `duration_ms`; uploads stream to a temp file (no longer buffered in RAM) so 100+ MB construction PDFs don't OOM the A10 small. Uvicorn keep-alive bumped to 30 min for long parses. Space autosleep lowered from 15 min β 5 min to halve idle billing. Added `pypdf` as a fallback page-count source when marker's metadata key shifts between versions. Caller for these changes: TrenchlessPM, replacing LlamaParse.
|
| 13 |
-
- Initial FastAPI app: `GET /health`, `POST /parse` (multipart upload, bearer-token auth, `parse_mode` toggle for marker's LLM-enhancement flag).
|
| 14 |
-
- `quality` mode wired to Claude (Sonnet 4.6 via marker's `ClaudeService`); reads `ANTHROPIC_API_KEY`, optional `CLAUDE_MODEL` override.
|
| 15 |
-
- Dockerfile: CUDA-enabled base, marker-pdf installed, `HF_HOME` and `TORCH_HOME` pinned to `/data` so model cache survives cold starts.
|
| 16 |
-
- Root `README.md` carries the HF Space frontmatter (Docker SDK, A10 small, 15-min autosleep, persistent storage).
|
| 17 |
-
- Project docs filled in (SPEC, ARCHITECTURE, DECISIONS, ENVIRONMENT, DEPENDENCIES, INFRASTRUCTURE, RUNBOOK).
|
| 18 |
|
| 19 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
|
| 9 |
## Unreleased
|
| 10 |
|
| 11 |
+
- (nothing yet)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
|
| 13 |
---
|
| 14 |
+
|
| 15 |
+
## 2026-04-28 β v0 deploy to HF Space
|
| 16 |
+
|
| 17 |
+
- First deploy of marker to `hirokamitai/trenchlesspm` (public, Docker SDK, A10 small, dev-mode off, autosleep 5 min). Push via `git push origin master:main`.
|
| 18 |
+
- HF Bucket `hirokamitai/trenchlesspm-cache` (private, RW) mounted at `/data` for marker's ~5 GB model cache. Replaces the older fixed-fee Persistent Storage tier with pay-per-byte billing. No code change β `HF_HOME`/`TORCH_HOME` already point at `/data`. See DECISIONS for rationale.
|
| 19 |
+
- `API_TOKEN` set as a Space secret (via `huggingface_hub.add_space_secret`). `ANTHROPIC_API_KEY` still pending β `parse_mode=quality` will 400 until set.
|
| 20 |
+
- Verified live: `GET /` 200 (endpoint blurb), `GET /health` 200 with `mode: "gpu"`, `POST /parse` without auth 401.
|
| 21 |
+
- FastAPI app shipped: `GET /` (blurb, public), `GET /health` (liveness, public), `POST /parse` (bearer-gated, multipart, `parse_mode` toggle).
|
| 22 |
+
- `parse_mode=quality` routes through Claude Sonnet 4.6 via marker's `ClaudeService`; reads `ANTHROPIC_API_KEY`, optional `CLAUDE_MODEL` override.
|
| 23 |
+
- `/parse` response shape: `{markdown, mode, page_count, duration_ms}`. Uploads stream to a temp file (no RAM buffering) so 100+ MB construction PDFs don't OOM the A10 small. Uvicorn `--timeout-keep-alive 1800` so long parses don't drop. Added `pypdf` as a fallback page-count source when marker's metadata key shifts between versions. Caller these changes were sized for: TrenchlessPM, replacing LlamaParse.
|
| 24 |
+
- Dockerfile: CUDA 12.4 base, Python 3.11, marker-pdf installed, `HF_HOME=/data/hf` and `TORCH_HOME=/data/torch` so the bucket holds the model cache across cold starts.
|
| 25 |
+
- Root `README.md` carries the HF Space frontmatter (Docker SDK, A10 small, 5-min autosleep, persistent storage suggested).
|
| 26 |
+
- Project docs filled in (SPEC, ARCHITECTURE, DECISIONS, ENVIRONMENT, DEPENDENCIES, INFRASTRUCTURE, RUNBOOK, CURRENT_STATE, TODO, ROADMAP, CHANGELOG).
|
|
@@ -8,25 +8,37 @@
|
|
| 8 |
|
| 9 |
## What's built and working
|
| 10 |
|
| 11 |
-
-
|
| 12 |
-
-
|
| 13 |
-
-
|
| 14 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
|
| 16 |
## What's deployed
|
| 17 |
|
| 18 |
-
- **Environment:** Hugging Face Space `hirokamitai/trenchlesspm`
|
| 19 |
-
- **
|
| 20 |
-
- **
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
## What's in progress
|
| 23 |
|
| 24 |
-
-
|
| 25 |
-
-
|
|
|
|
| 26 |
|
| 27 |
## What's known broken / flaky
|
| 28 |
|
| 29 |
-
- Nothing observed yet
|
| 30 |
|
| 31 |
## Half-finished or abandoned
|
| 32 |
|
|
|
|
| 8 |
|
| 9 |
## What's built and working
|
| 10 |
|
| 11 |
+
- FastAPI app with three routes:
|
| 12 |
+
- `GET /` β endpoint blurb (public, no auth).
|
| 13 |
+
- `GET /health` β liveness + reports `gpu`/`cpu` (public, no auth).
|
| 14 |
+
- `POST /parse` β multipart upload β `{markdown, mode, page_count, duration_ms}` (bearer-gated).
|
| 15 |
+
- Bearer-token auth via `Authorization: Bearer <API_TOKEN>`; constant-time compare.
|
| 16 |
+
- Streamed uploads to a temp file (1 MiB chunks) β handles 100+ MB PDFs without OOM.
|
| 17 |
+
- `parse_mode=fast` runs marker-pdf alone; `parse_mode=quality` adds Claude Sonnet 4.6 LLM-enhancement via marker's `ClaudeService`.
|
| 18 |
+
- Page count from marker's rendered metadata, falling back to `pypdf`; duration timed with `time.perf_counter()`.
|
| 19 |
+
- Dockerfile (CUDA 12.4 base, Python 3.11) with `HF_HOME` and `TORCH_HOME` pinned to `/data` so marker's ~5 GB of model weights persist via the HF Bucket mount.
|
| 20 |
+
- Uvicorn `--timeout-keep-alive 1800` so long parses don't drop.
|
| 21 |
|
| 22 |
## What's deployed
|
| 23 |
|
| 24 |
+
- **Environment:** Hugging Face Space `hirokamitai/trenchlesspm` β public, Docker SDK, A10 small, dev-mode off, autosleep 5 min.
|
| 25 |
+
- **Persistence:** HF Bucket `hirokamitai/trenchlesspm-cache` (private, RW) mounted at `/data`.
|
| 26 |
+
- **Secrets set:** `API_TOKEN` β. `ANTHROPIC_API_KEY` is **not yet set** β so `parse_mode=quality` will 400 until it is.
|
| 27 |
+
- **Verified live (2026-04-28):**
|
| 28 |
+
- `GET /` β 200, returns endpoint blurb.
|
| 29 |
+
- `GET /health` β 200, `{"status":"ok","mode":"gpu"}` (GPU detected).
|
| 30 |
+
- `POST /parse` without auth β 401 (auth enforced).
|
| 31 |
+
- **Version / commit:** see latest `origin/main`. End-to-end PDF smoke test still pending (next item below).
|
| 32 |
|
| 33 |
## What's in progress
|
| 34 |
|
| 35 |
+
- Set `ANTHROPIC_API_KEY` on the Space (only blocks `parse_mode=quality`).
|
| 36 |
+
- End-to-end smoke test: upload a real contract PDF, verify both modes return sensible markdown and that cold start after autosleep loads weights from the bucket (no 5 GB re-download).
|
| 37 |
+
- Wire TrenchlessPM (`/Users/michael/Documents/apps/web/TrenchlessPM`) to call `/parse`, replacing LlamaParse.
|
| 38 |
|
| 39 |
## What's known broken / flaky
|
| 40 |
|
| 41 |
+
- Nothing observed yet at runtime.
|
| 42 |
|
| 43 |
## Half-finished or abandoned
|
| 44 |
|
|
@@ -27,6 +27,30 @@ Write decisions where someone smart would reasonably pick differently.
|
|
| 27 |
|
| 28 |
## Entries
|
| 29 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
## 2026-04-28 β Claude Sonnet 4.6 for marker's LLM-enhancement pass
|
| 31 |
|
| 32 |
**Decision:** point marker's `quality` mode at `marker.services.claude.ClaudeService`
|
|
|
|
| 27 |
|
| 28 |
## Entries
|
| 29 |
|
| 30 |
+
## 2026-04-28 β Public Space, bearer-token auth (not HF private + HF token)
|
| 31 |
+
|
| 32 |
+
**Decision:** the Space is **public**; access control to `/parse` is the
|
| 33 |
+
single shared bearer token in `app/main.py` (`API_TOKEN`, set as a Space
|
| 34 |
+
secret). `/` and `/health` are intentionally open and leak nothing.
|
| 35 |
+
**Context:** TrenchlessPM and any other internal caller needs to invoke
|
| 36 |
+
`/parse` from non-interactive code. HF Spaces gate private Spaces behind
|
| 37 |
+
the caller's HF user token, on top of any app-level auth.
|
| 38 |
+
**Alternatives considered:** keep the Space private and require callers
|
| 39 |
+
to send `Authorization: Bearer <HF_USER_TOKEN>` *plus* our app-level
|
| 40 |
+
`API_TOKEN`. Workable, but doubles the number of secrets each integrator
|
| 41 |
+
manages, and HF user tokens are coarse-grained (they grant access to the
|
| 42 |
+
whole HF account, not just this Space) β leaking one is a much bigger
|
| 43 |
+
incident than leaking the marker bearer.
|
| 44 |
+
**Tradeoffs:** the Space URL is publicly discoverable. Anyone can hit it,
|
| 45 |
+
and unauthenticated `/parse` calls will burn a tiny bit of edge bandwidth
|
| 46 |
+
returning 401. Acceptable β a 256-bit shared secret is the actual
|
| 47 |
+
enforcement boundary, and HF's edge handles abuse.
|
| 48 |
+
**Revisit if:** (a) we need fine-grained per-caller revocation (right now
|
| 49 |
+
rotating the single token forces every caller to update), (b) compliance
|
| 50 |
+
ever requires that the URL itself not be discoverable, or (c) we add
|
| 51 |
+
endpoints that are unsafe to expose at all (in which case route-level
|
| 52 |
+
auth is the answer, not Space-level).
|
| 53 |
+
|
| 54 |
## 2026-04-28 β Claude Sonnet 4.6 for marker's LLM-enhancement pass
|
| 55 |
|
| 56 |
**Decision:** point marker's `quality` mode at `marker.services.claude.ClaudeService`
|
|
@@ -12,7 +12,7 @@ do **not** live in this file β only pointers to them.
|
|
| 12 |
|
| 13 |
| Name | Purpose | Where the real value lives | Used by |
|
| 14 |
|---|---|---|---|
|
| 15 |
-
| `API_TOKEN` | Bearer token clients send in `Authorization: Bearer β¦` to call `/parse`. Single shared secret, no per-user tokens. | 1Password > marker > API_TOKEN | `app/main.py` |
|
| 16 |
|
| 17 |
## Optional
|
| 18 |
|
|
@@ -20,7 +20,7 @@ do **not** live in this file β only pointers to them.
|
|
| 20 |
|---|---|---|---|
|
| 21 |
| `HF_HOME` | Where the Hugging Face client caches model weights. Pinned to persistent storage so cold starts don't re-download ~5 GB. | `/data/hf` (set in Dockerfile) | n/a β set in image |
|
| 22 |
| `TORCH_HOME` | Where Torch hub caches checkpoints. Same reason as `HF_HOME`. | `/data/torch` (set in Dockerfile) | n/a β set in image |
|
| 23 |
-
| `ANTHROPIC_API_KEY` | Used by marker's LLM-enhancement path (Claude) when `parse_mode=quality`. Not needed if you never call quality mode. | 1Password > marker > ANTHROPIC_API_KEY | `app/parser.py` |
|
| 24 |
| `CLAUDE_MODEL` | Claude model ID used for quality mode. Override only if you want a different speed/cost/quality tradeoff. | `claude-sonnet-4-6` | n/a |
|
| 25 |
| `LOG_LEVEL` | Uvicorn log level. | `info` | n/a |
|
| 26 |
| `PORT` | Port FastAPI binds to. HF Spaces convention is 7860. | `7860` | n/a |
|
|
@@ -28,7 +28,7 @@ do **not** live in this file β only pointers to them.
|
|
| 28 |
## Where env vars are loaded
|
| 29 |
|
| 30 |
- **Local:** `.env` file at project root, loaded by `python-dotenv` from `app/main.py` on startup. Only used for development; production reads from the Space's secret store directly.
|
| 31 |
-
- **Production (HF Space):** set in the Space's **Settings β Variables and secrets** panel. `API_TOKEN` and `ANTHROPIC_API_KEY` go in **Secrets**; `HF_HOME` / `TORCH_HOME` are baked into the Dockerfile and don't need to be set here.
|
| 32 |
|
| 33 |
## Rotation notes
|
| 34 |
|
|
|
|
| 12 |
|
| 13 |
| Name | Purpose | Where the real value lives | Used by |
|
| 14 |
|---|---|---|---|
|
| 15 |
+
| `API_TOKEN` | Bearer token clients send in `Authorization: Bearer β¦` to call `/parse`. Single shared secret, no per-user tokens. **Set on the Space as of 2026-04-28.** | 1Password > marker > API_TOKEN | `app/main.py` |
|
| 16 |
|
| 17 |
## Optional
|
| 18 |
|
|
|
|
| 20 |
|---|---|---|---|
|
| 21 |
| `HF_HOME` | Where the Hugging Face client caches model weights. Pinned to persistent storage so cold starts don't re-download ~5 GB. | `/data/hf` (set in Dockerfile) | n/a β set in image |
|
| 22 |
| `TORCH_HOME` | Where Torch hub caches checkpoints. Same reason as `HF_HOME`. | `/data/torch` (set in Dockerfile) | n/a β set in image |
|
| 23 |
+
| `ANTHROPIC_API_KEY` | Used by marker's LLM-enhancement path (Claude) when `parse_mode=quality`. Not needed if you never call quality mode. **Not yet set on the Space as of 2026-04-28** β `parse_mode=quality` will 400 until it is. | 1Password > marker > ANTHROPIC_API_KEY | `app/parser.py` |
|
| 24 |
| `CLAUDE_MODEL` | Claude model ID used for quality mode. Override only if you want a different speed/cost/quality tradeoff. | `claude-sonnet-4-6` | n/a |
|
| 25 |
| `LOG_LEVEL` | Uvicorn log level. | `info` | n/a |
|
| 26 |
| `PORT` | Port FastAPI binds to. HF Spaces convention is 7860. | `7860` | n/a |
|
|
|
|
| 28 |
## Where env vars are loaded
|
| 29 |
|
| 30 |
- **Local:** `.env` file at project root, loaded by `python-dotenv` from `app/main.py` on startup. Only used for development; production reads from the Space's secret store directly.
|
| 31 |
+
- **Production (HF Space):** set in the Space's **Settings β Variables and secrets** panel, or programmatically via `huggingface_hub.HfApi().add_space_secret(...)`. `API_TOKEN` and `ANTHROPIC_API_KEY` go in **Secrets**; `HF_HOME` / `TORCH_HOME` are baked into the Dockerfile and don't need to be set here. See [RUNBOOK.md](RUNBOOK.md) for the API-based recipe (avoids putting the secret on the bash command line / in shell history).
|
| 32 |
|
| 33 |
## Rotation notes
|
| 34 |
|
|
@@ -19,14 +19,16 @@ and the `hf` CLI / git push to the Space repo.
|
|
| 19 |
- **Provider:** Hugging Face Spaces
|
| 20 |
- **Host name / label:** `hirokamitai/trenchlesspm`
|
| 21 |
- **URL:** `https://hirokamitai-trenchlesspm.hf.space`
|
|
|
|
| 22 |
- **Region:** US (HF default for GPU Spaces)
|
| 23 |
- **Size / tier:** A10 small (1Γ NVIDIA A10G, 24 GB VRAM)
|
| 24 |
- **SDK:** Docker
|
|
|
|
| 25 |
- **Persistent storage:** HF Bucket `hirokamitai/trenchlesspm-cache` (private, read+write) mounted at `/data`. Holds the marker model cache (`/data/hf`, `/data/torch`) so cold starts skip the ~5 GB model download.
|
| 26 |
- **Sleep behavior:** autosleep after 5 min idle (`sleep_time: 300` in README frontmatter)
|
| 27 |
- **Monthly cost:** A10 small billed by the second when awake (~$1.05/hr) + bucket storage (pay-per-byte, ~5 GB of marker weights). Target total <$5β10/mo at expected usage.
|
| 28 |
- **Paid with:** card on file at huggingface.co/settings/billing
|
| 29 |
-
- **
|
| 30 |
|
| 31 |
## How to access
|
| 32 |
|
|
|
|
| 19 |
- **Provider:** Hugging Face Spaces
|
| 20 |
- **Host name / label:** `hirokamitai/trenchlesspm`
|
| 21 |
- **URL:** `https://hirokamitai-trenchlesspm.hf.space`
|
| 22 |
+
- **Visibility:** **public** (real access control is the bearer-token check in `app/main.py`; see [DECISIONS.md](DECISIONS.md))
|
| 23 |
- **Region:** US (HF default for GPU Spaces)
|
| 24 |
- **Size / tier:** A10 small (1Γ NVIDIA A10G, 24 GB VRAM)
|
| 25 |
- **SDK:** Docker
|
| 26 |
+
- **Dev mode:** off (must stay off β when on, HF routes the public URL to a dev IDE and every endpoint 404s)
|
| 27 |
- **Persistent storage:** HF Bucket `hirokamitai/trenchlesspm-cache` (private, read+write) mounted at `/data`. Holds the marker model cache (`/data/hf`, `/data/torch`) so cold starts skip the ~5 GB model download.
|
| 28 |
- **Sleep behavior:** autosleep after 5 min idle (`sleep_time: 300` in README frontmatter)
|
| 29 |
- **Monthly cost:** A10 small billed by the second when awake (~$1.05/hr) + bucket storage (pay-per-byte, ~5 GB of marker weights). Target total <$5β10/mo at expected usage.
|
| 30 |
- **Paid with:** card on file at huggingface.co/settings/billing
|
| 31 |
+
- **First deploy:** 2026-04-28
|
| 32 |
|
| 33 |
## How to access
|
| 34 |
|
|
@@ -13,23 +13,23 @@ For why specific choices were made, see [DECISIONS.md](DECISIONS.md).
|
|
| 13 |
|
| 14 |
## Now (this month)
|
| 15 |
|
| 16 |
-
- **
|
| 17 |
-
-
|
|
|
|
| 18 |
|
| 19 |
## Next (1β3 months)
|
| 20 |
|
| 21 |
-
- **
|
| 22 |
-
-
|
| 23 |
|
| 24 |
## Later (3β6+ months)
|
| 25 |
|
| 26 |
-
- **
|
| 27 |
-
-
|
| 28 |
|
| 29 |
## Considered, not doing
|
| 30 |
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
-
|
| 35 |
-
-
|
|
|
|
| 13 |
|
| 14 |
## Now (this month)
|
| 15 |
|
| 16 |
+
- **Cut TrenchlessPM over from LlamaParse to marker** β finish the integration on the TrenchlessPM side and run the first real contract through `/parse`. Success: parses match or beat LlamaParse Premium quality at <$5/mo bill.
|
| 17 |
+
- **Verify cold-start cache hit** β confirm that after autosleep, the first wake reads weights from the bucket (no 5 GB redownload in the Logs tab). Success: cold-start latency settles around 30β60 s.
|
| 18 |
+
- **Set `ANTHROPIC_API_KEY`** so `parse_mode=quality` is usable. Success: a real contract round-trips through Sonnet 4.6 cleanup.
|
| 19 |
|
| 20 |
## Next (1β3 months)
|
| 21 |
|
| 22 |
+
- **Hash-keyed result caching** β skip the GPU when TrenchlessPM retries the same PDF. Probably an in-memory LRU or a small SQLite at `/data/cache.db`.
|
| 23 |
+
- **Operability** β wire the Space into the sapplab monitor dashboard via `.monitor.yml`; add a `429`-aware retry on the TrenchlessPM client side.
|
| 24 |
|
| 25 |
## Later (3β6+ months)
|
| 26 |
|
| 27 |
+
- **Multi-PDF batch endpoint** β only if a real workflow asks for it; sync request model is fine for v1.
|
| 28 |
+
- **Audit / replay** β externalize input+output to an R2/S3 bucket if a downstream consumer ever needs to dispute "what did marker return for this PDF?"
|
| 29 |
|
| 30 |
## Considered, not doing
|
| 31 |
|
| 32 |
+
- **Mounting an external S3/R2/GCS bucket for inputs/outputs** β see [DECISIONS.md](DECISIONS.md) "No external bucket"; speculative for v1.
|
| 33 |
+
- **Gradio UI on top of `/parse`** β JSON API only is a goal, not a v1 omission; see [SPEC.md](SPEC.md).
|
| 34 |
+
- **Multiple Uvicorn workers** β would OOM the GPU; concurrency comes from Space scaling, not in-process workers.
|
| 35 |
+
- **Local LLM for `quality` mode** β A10's VRAM is for marker; the LLM call stays external.
|
|
|
|
@@ -34,13 +34,16 @@ uvicorn app.main:app --host 0.0.0.0 --port 7860 --reload
|
|
| 34 |
Smoke test:
|
| 35 |
|
| 36 |
```bash
|
|
|
|
|
|
|
|
|
|
| 37 |
curl -s http://localhost:7860/health
|
| 38 |
# => {"status":"ok","mode":"cpu"}
|
| 39 |
|
| 40 |
curl -s -X POST http://localhost:7860/parse \
|
| 41 |
-H "Authorization: Bearer $API_TOKEN" \
|
| 42 |
-F "file=@sample.pdf" \
|
| 43 |
-
-F "parse_mode=fast" | jq
|
| 44 |
```
|
| 45 |
|
| 46 |
### Run tests
|
|
@@ -61,12 +64,28 @@ The Space repo is wired up as this repo's `origin` remote β local branch is
|
|
| 61 |
`master`, the Space's default branch is `main`, so all pushes are
|
| 62 |
`master:main`.
|
| 63 |
|
| 64 |
-
### Set Space secrets
|
|
|
|
|
|
|
| 65 |
|
| 66 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 67 |
|
| 68 |
-
|
| 69 |
-
|
|
|
|
|
|
|
| 70 |
|
| 71 |
### Deploy a change
|
| 72 |
|
|
@@ -103,10 +122,26 @@ curl -fsS https://hirokamitai-trenchlesspm.hf.space/health
|
|
| 103 |
If it's been idle >5 min the first request triggers a cold start (~30β60 s
|
| 104 |
wait). That's by design.
|
| 105 |
|
| 106 |
-
###
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 107 |
|
| 108 |
-
|
| 109 |
-
**Restart this Space** (just restarts the container).
|
| 110 |
|
| 111 |
### Tail logs
|
| 112 |
|
|
@@ -124,9 +159,10 @@ container boots and marker loads model weights from `/data`.
|
|
| 124 |
|
| 125 |
When something's broken, try these in order:
|
| 126 |
|
| 127 |
-
1. **Health check.** `curl https://hirokamitai-trenchlesspm.hf.space/health`. If this fails, the container is down
|
| 128 |
-
2. **
|
| 129 |
-
3. **
|
| 130 |
-
4. **
|
| 131 |
-
5. **
|
| 132 |
-
6. **
|
|
|
|
|
|
| 34 |
Smoke test:
|
| 35 |
|
| 36 |
```bash
|
| 37 |
+
curl -s http://localhost:7860/
|
| 38 |
+
# => {"service":"marker","endpoints":{...}}
|
| 39 |
+
|
| 40 |
curl -s http://localhost:7860/health
|
| 41 |
# => {"status":"ok","mode":"cpu"}
|
| 42 |
|
| 43 |
curl -s -X POST http://localhost:7860/parse \
|
| 44 |
-H "Authorization: Bearer $API_TOKEN" \
|
| 45 |
-F "file=@sample.pdf" \
|
| 46 |
+
-F "parse_mode=fast" | jq '{mode, page_count, duration_ms, head: (.markdown[:200])}'
|
| 47 |
```
|
| 48 |
|
| 49 |
### Run tests
|
|
|
|
| 64 |
`master`, the Space's default branch is `main`, so all pushes are
|
| 65 |
`master:main`.
|
| 66 |
|
| 67 |
+
### Set Space secrets
|
| 68 |
+
|
| 69 |
+
Either via the web UI (https://huggingface.co/spaces/hirokamitai/trenchlesspm/settings β Variables and secrets) or via the `huggingface_hub` Python API. **Never paste a secret on the bash command line directly** β pass it via env var so it doesn't land in shell history. Example:
|
| 70 |
|
| 71 |
+
```bash
|
| 72 |
+
HF_PY=/opt/homebrew/Cellar/hf/*/libexec/bin/python # bundled with `brew install hf`
|
| 73 |
+
SECRET_VALUE='<paste here>' $HF_PY -c '
|
| 74 |
+
import os
|
| 75 |
+
from huggingface_hub import HfApi
|
| 76 |
+
HfApi().add_space_secret(
|
| 77 |
+
repo_id="hirokamitai/trenchlesspm",
|
| 78 |
+
key="ANTHROPIC_API_KEY", # or API_TOKEN
|
| 79 |
+
value=os.environ["SECRET_VALUE"],
|
| 80 |
+
)
|
| 81 |
+
'
|
| 82 |
+
unset SECRET_VALUE
|
| 83 |
+
```
|
| 84 |
|
| 85 |
+
Required:
|
| 86 |
+
|
| 87 |
+
- `API_TOKEN` β bearer token callers must send. **Already set** as of 2026-04-28.
|
| 88 |
+
- `ANTHROPIC_API_KEY` β only needed for `parse_mode=quality`. **Pending** as of 2026-04-28.
|
| 89 |
|
| 90 |
### Deploy a change
|
| 91 |
|
|
|
|
| 122 |
If it's been idle >5 min the first request triggers a cold start (~30β60 s
|
| 123 |
wait). That's by design.
|
| 124 |
|
| 125 |
+
### Check Space metadata (stage, dev mode, hardware, sha)
|
| 126 |
+
|
| 127 |
+
```bash
|
| 128 |
+
hf spaces info hirokamitai/trenchlesspm
|
| 129 |
+
```
|
| 130 |
+
|
| 131 |
+
Look at `runtime.stage` (`RUNNING` / `BUILDING` / `STOPPED` / `SLEEPING`) and
|
| 132 |
+
`runtime.raw.devMode`. **Dev mode must be `false` for the public URL to route
|
| 133 |
+
to FastAPI** β when it's `true`, HF routes the URL to a dev IDE and every
|
| 134 |
+
endpoint 404s (you'll see HF's HTML 404 page, not our JSON).
|
| 135 |
+
|
| 136 |
+
### Restart / rebuild
|
| 137 |
+
|
| 138 |
+
```bash
|
| 139 |
+
# Toggle dev mode off (also forces a restart):
|
| 140 |
+
hf spaces dev-mode hirokamitai/trenchlesspm --stop
|
| 141 |
+
```
|
| 142 |
|
| 143 |
+
Web UI alternatives: **Settings β Factory rebuild** (rebuilds image) or
|
| 144 |
+
**Restart this Space** (just restarts the container).
|
| 145 |
|
| 146 |
### Tail logs
|
| 147 |
|
|
|
|
| 159 |
|
| 160 |
When something's broken, try these in order:
|
| 161 |
|
| 162 |
+
1. **Health check.** `curl https://hirokamitai-trenchlesspm.hf.space/health`. If this fails, the container is down, sleeping, or dev-mode is on.
|
| 163 |
+
2. **HTML 404 (not our JSON 404) on every endpoint?** `hf spaces info hirokamitai/trenchlesspm` β check `runtime.raw.devMode`. If `true`, run `hf spaces dev-mode hirokamitai/trenchlesspm --stop` (or toggle in the UI). Same symptom can also mean the Space is private and the caller has no HF auth β the Space should be public.
|
| 164 |
+
3. **Tail the Space logs** in the web UI's Logs tab. Look for Python tracebacks, OOMs ("CUDA out of memory"), or marker model-load errors.
|
| 165 |
+
4. **401 from `/parse`?** Verify the caller is sending `Authorization: Bearer <API_TOKEN>` and that the token matches the `API_TOKEN` Space secret. Re-set the secret if in doubt.
|
| 166 |
+
5. **Quality mode failing but fast mode works?** Almost certainly `ANTHROPIC_API_KEY` is missing/expired or you're rate-limited. Check the Space secrets and the Anthropic console for usage/quota.
|
| 167 |
+
6. **Cold starts re-download 5 GB every time?** The HF Bucket (`hirokamitai/trenchlesspm-cache`) isn't mounted, or `HF_HOME`/`TORCH_HOME` aren't pointing at `/data`. Verify the bucket is attached at `/data` in the Space's Storage Buckets settings and the Dockerfile's `ENV` lines are intact.
|
| 168 |
+
7. **Cross-reference [DEPENDENCIES.md](DEPENDENCIES.md)** for status pages of HF and Anthropic if multiple things are broken at once.
|
|
@@ -56,7 +56,7 @@ Not public, not multi-tenant.
|
|
| 56 |
|
| 57 |
- Budget: pay-per-second GPU billing on A10 small. Target <$5/mo at expected volume.
|
| 58 |
- Time: weekend build. Don't over-engineer.
|
| 59 |
-
- Platform: Hugging Face Space, Docker SDK, A10 small GPU,
|
| 60 |
-
|
| 61 |
-
- Other: model cache must live on `/data` (HF
|
| 62 |
-
container FS. Space configured to sleep after
|
|
|
|
| 56 |
|
| 57 |
- Budget: pay-per-second GPU billing on A10 small. Target <$5/mo at expected volume.
|
| 58 |
- Time: weekend build. Don't over-engineer.
|
| 59 |
+
- Platform: Hugging Face Space, Docker SDK, A10 small GPU, HF Bucket
|
| 60 |
+
(`hirokamitai/trenchlesspm-cache`) mounted at `/data`.
|
| 61 |
+
- Other: model cache must live on `/data` (the HF Bucket mount), not the
|
| 62 |
+
ephemeral container FS. Space configured to sleep after 5 min idle.
|
|
@@ -9,10 +9,10 @@ For "what's broken," also see CURRENT_STATE.md's known-issues section.
|
|
| 9 |
|
| 10 |
## Next up (doing soon, in order)
|
| 11 |
|
| 12 |
-
1. Set `
|
| 13 |
-
2.
|
| 14 |
-
3.
|
| 15 |
-
4.
|
| 16 |
|
| 17 |
## Backlog (not prioritized)
|
| 18 |
|
|
@@ -23,9 +23,9 @@ For "what's broken," also see CURRENT_STATE.md's known-issues section.
|
|
| 23 |
|
| 24 |
## Ideas / maybe
|
| 25 |
|
| 26 |
-
-
|
| 27 |
- Multi-PDF batch endpoint, if a real workflow shows up that wants it.
|
| 28 |
-
- Swap the
|
| 29 |
|
| 30 |
---
|
| 31 |
|
|
@@ -33,6 +33,12 @@ For "what's broken," also see CURRENT_STATE.md's known-issues section.
|
|
| 33 |
|
| 34 |
Keep the last ~10. Older items belong in [CHANGELOG.md](CHANGELOG.md).
|
| 35 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
- 2026-04-28 β Filled in `_project/` docs (SPEC, ARCHITECTURE, DECISIONS, ENVIRONMENT, DEPENDENCIES, INFRASTRUCTURE, RUNBOOK).
|
| 37 |
- 2026-04-28 β Installed `hf` CLI + the `hf-cli` skill globally for Claude Code.
|
| 38 |
- 2026-04-28 β Scaffolded FastAPI app, Dockerfile, and HF Space README frontmatter.
|
|
|
|
| 9 |
|
| 10 |
## Next up (doing soon, in order)
|
| 11 |
|
| 12 |
+
1. Set `ANTHROPIC_API_KEY` on the Space (https://huggingface.co/spaces/hirokamitai/trenchlesspm/settings β Secrets). Required for `parse_mode=quality`.
|
| 13 |
+
2. End-to-end smoke test: upload a real contract PDF and verify `{markdown, mode, page_count, duration_ms}` for both `fast` and `quality`.
|
| 14 |
+
3. After the first parse, sleep the Space (or wait 5 min) and confirm cold start loads weights from the bucket β no 5 GB re-download in the Logs tab.
|
| 15 |
+
4. Wire TrenchlessPM to call `/parse`, replacing LlamaParse. Document the integration on the TrenchlessPM side.
|
| 16 |
|
| 17 |
## Backlog (not prioritized)
|
| 18 |
|
|
|
|
| 23 |
|
| 24 |
## Ideas / maybe
|
| 25 |
|
| 26 |
+
- External audit-trail bucket (deferred β see [DECISIONS.md](DECISIONS.md) "No external bucket"). Revisit if a downstream caller ever needs replay.
|
| 27 |
- Multi-PDF batch endpoint, if a real workflow shows up that wants it.
|
| 28 |
+
- Swap the Claude model used for `quality` mode (e.g. bump to Opus 4.7, or try Haiku 4.5 for cheaper) if Sonnet 4.6 output disappoints on contracts. Set via `CLAUDE_MODEL` env var, no code change.
|
| 29 |
|
| 30 |
---
|
| 31 |
|
|
|
|
| 33 |
|
| 34 |
Keep the last ~10. Older items belong in [CHANGELOG.md](CHANGELOG.md).
|
| 35 |
|
| 36 |
+
- 2026-04-28 β Deployed to `hirokamitai/trenchlesspm`. Public, dev-mode off, A10 small running. `/`, `/health`, `/parse`-401 verified live.
|
| 37 |
+
- 2026-04-28 β Switched persistence to HF Bucket `hirokamitai/trenchlesspm-cache` mounted at `/data`. Cheaper than the older Persistent Storage tier.
|
| 38 |
+
- 2026-04-28 β Set `API_TOKEN` as a Space secret via `huggingface_hub.add_space_secret`.
|
| 39 |
+
- 2026-04-28 β Added `GET /` route so the Space URL doesn't 404 in a browser.
|
| 40 |
+
- 2026-04-28 β Wired `parse_mode=quality` to Claude Sonnet 4.6 via marker's `ClaudeService` (was Gemini default).
|
| 41 |
+
- 2026-04-28 β Streamed uploads to disk, added `page_count` + `duration_ms` to the `/parse` response, raised Uvicorn keep-alive to 30 min, lowered autosleep to 5 min.
|
| 42 |
- 2026-04-28 β Filled in `_project/` docs (SPEC, ARCHITECTURE, DECISIONS, ENVIRONMENT, DEPENDENCIES, INFRASTRUCTURE, RUNBOOK).
|
| 43 |
- 2026-04-28 β Installed `hf` CLI + the `hf-cli` skill globally for Claude Code.
|
| 44 |
- 2026-04-28 β Scaffolded FastAPI app, Dockerfile, and HF Space README frontmatter.
|