FIELD_NOTES.md

# Field Notes — building the iMessage → Calendar agent

What I set out to build, where reality bent the plan, and what I'd do next. This is
the "what I learned" companion to the product docs ([README](./README.md)) and the
design doc ([PLAN](./PLAN.md)).

## The goal in one line

Turn the calendar logistics buried in a chat thread — *"picture day moved to
Thursday 9am", "soccer is Tuesday now"* — into reviewed calendar events, from a
phone, with the data staying private.

## 1. "Read my iMessages" is impossible as literally asked — and that shaped everything

iOS exposes **no API** for iMessage/SMS *content*. There is no on-device path. The
only place the messages exist in a queryable form is a Mac, where they sync to
`~/Library/Messages/chat.db`. So the architecture forked early:

- A Mac-side collector ([`collector/collector.py`](./collector/collector.py)) reads
  `chat.db` (read-only, `mode=ro`) and POSTs new rows to the Space.
- "On my phone" was reinterpreted honestly as **used from** a phone browser — the
  Space is hosted, the UI is mobile-friendly, but the model runs in the Space.

The biggest adoption lesson came later: requiring a Mac collector + Full Disk
Access is a wall for a non-technical user. The fix was to make **paste-from-phone**
the hero path (the collector is now strictly optional) — no install, no DB, no
permissions. Most of that capability already existed in the Schedule tab; it was just
framed as secondary.

## 2. `attributedBody` is the iMessage parsing trap

Modern Messages often stores the body in `attributedBody` (an `NSAttributedString`
binary blob), **not** the `text` column. The collector reads `text` directly for
simplicity and **skips messages that only have `attributedBody`**
([`collector/collector.py:88-94`](./collector/collector.py)) — a deliberate, called-out
gap. The right move for production is to not hand-roll this: use `imessage-exporter`
(ReagentX) or `imessage_reader`. Noting the limitation in code beat pretending the
naive SQL was complete.

## 3. Relative dates are the real accuracy battleground

The hard part isn't "is there an event" — it's *when*. "Next Thursday", "the 14th",
"in two weeks" only resolve against a reference time. Two design responses:

- The system prompt pins **"Current datetime"** into every request and instructs the
  model to resolve relative dates from it ([`server/agent.py:21-34`](./server/agent.py)).
- **Conflict math is deterministic, not model-driven.** Overlap/adjacent/tight
  detection and alternative-time proposals live in
  [`calendar_out/freebusy.py`](./calendar_out/freebusy.py), because once you have ISO
  datetimes, interval math should never be left to an LLM. The model decides *what*;
  code decides *when-it-clashes*.

The stub extractor's naive "match a time → 1h event tomorrow"
([`server/agent.py:152-175`](./server/agent.py)) is intentionally dumb — it exists to
prove the pipeline, and its dumbness is a good reminder of exactly how much the
fine-tune has to get right.

## 4. Stub-first was the best architectural call

`USE_STUB_EXTRACTOR=1` swaps the model for a regex heuristic
([`server/agent.py:85,124`](./server/agent.py)), forced on in tests
([`tests/conftest.py`](./tests/conftest.py)). Payoffs:

- The whole app — paste → events → conflicts → `.ics` download → impact panel —
  **works end-to-end with no GPU**, so a demo (and CI) never depends on a model load.
- `llama_cpp` and the Google libs are **lazy-imported**, so `requirements-ci.txt` can
  exclude them and the test suite runs in seconds, offline.

Lesson: make the expensive dependency optional from day one and the cheap path
becomes your test harness, your demo, and your free tier all at once.

## 5. Reframing around one person changed the scope more than any feature

The project started as a four-track hackathon checklist. Rewriting it around a single
named person — a **busy parent** whose kid's events are buried in a class group chat —
forced three concrete changes: phone-paste as the default, a one-tap **Try a sample**
class-chat ([`ui/blocks.py`](./ui/blocks.py)), and a **"This week"** impact panel.

On measurement: `minutes_saved` ([`server/impact.py`](./server/impact.py)) is a
**configurable estimate, not a measurement** (default 8 min/event + 15 min/conflict).
Saying that plainly — in the UI, the README, and here — matters more than a
bigger-looking number. A capture is only counted when the parent *accepts* events by
exporting them, so the metric tracks value taken, not previews shown.

## 6. Fine-tuning economics: Modal credits + honest scope

QLoRA on a 31B needs an 80GB GPU. [`training/modal_train.py`](./training/modal_train.py)
wraps the existing `train_qlora.py` + `export_gguf.sh` to run on a serverless
A100/H100 and publish the GGUF to HF — roughly **$5–15 per run**, so ~$250 of credit
is 15–40 iterations. The "Well-Tuned" track went the distance: the eval-gated **E4B** fine-tune is
published and is what production serves —
[`build-small-hackathon/gemma-4-cal-gguf`](https://huggingface.co/build-small-hackathon/gemma-4-cal-gguf)
— after clearing the gate over six runs at zero quality cost vs. stock E4B. (Re-running the pipeline
still spends your own Modal credits; the turnkey path is there whenever you want to retrain.)

A small rule that paid off: training-data generation can use *any* offline tooling —
the "no cloud AI API" rule applies only to the **running app's inference**, not to
dataset prep.

## 7. Two models, not one — a 1B planner over the same tools

What shipped is two small local models, not one. The fine-tuned **gemma-cal E4B** does the
*reading* (thread → validated `ActionPlan`); a 1B **OpenBMB MiniCPM** does the *orchestrating*.
Clicking **Run the agents** hands the job to MiniCPM, which drives the Space's own MCP tools —
`extract_events → check_conflicts → make_ics` — as a visible multi-step agent
([`server/orchestrator.py`](./server/orchestrator.py)), consuming the *public* tool contract
instead of calling internals. Two things I'd underline: keep the planner **optional** (a
deterministic scripted plan is the fallback, so the agentic path never hard-depends on a second
model load), and don't let "agent" become a separate destination — the same **Run the agents**
action drives both the home workflow and the orchestrated trace, so it stays one engine, not a
second UI to keep in sync.

## 8. The Off-the-Grid tension

"No cloud AI APIs" and "serve a 31B" pull against each other: a Q4 31B GGUF is
~18–20GB and needs a GPU. Keeping inference **in the Space via `llama.cpp`** preserves
the privacy story but costs GPU. The honest compromise is the **E4B edge variant** for
the free tier, with the 31B as the headline. I deliberately did **not** offload
inference to a third-party endpoint, because "your own Modal GPU" and "a cloud AI API"
are easy to conflate and a purist judge would be right to dock it.

The same principle drove the trace-sharing design (below): the hosted Space holds **no
HF token** — it only offers a **local download**, and a separate local CLI does the
upload with your own auth.

## 9. What I'd do next

- **Durable trace/metrics store.** The activity bus is an 800-entry in-memory ring
  buffer ([`server/events.py`](./server/events.py)) — runs are lost on restart, so only
  recent runs are exportable. A small append-only store (the impact log already shows
  the pattern) would fix it.
- **Decode `attributedBody`** (or adopt `imessage-exporter`) so text-less messages stop
  being dropped.
- **A real eval set** from the expanded dataset — measure JSON validity + field
  accuracy, especially relative-date resolution and empty-list-on-chitchat.
- **Trace redaction as a tested invariant.** Today it's an allowlist over current emit
  sites ([`server/trace.py`](./server/trace.py)); a lint/test that fails when a new
  `emit(...)` puts free text on a non-`ingest` stage would keep it honest as the code
  grows.

## Publishing these notes

This file is linked from the README. It can also be pasted into the Space's README
(the Space card renders Markdown) or posted to the model/dataset repo's **Community**
tab on the Hub so others can learn from the build.