| # Dolphin Assistant for iPhone |
|
|
| Dolphin Assistant is a private, on-device iOS 18 assistant powered by the |
| stateful INT4 Dolphin 3.0 Core ML model. It can plan a short sequence of local |
| actions, ask for approval before sensitive work, execute registered tools, and |
| use the results to answer in the same conversation. Chats, typed knowledge, |
| context digests, session handoffs, memories, tasks, settings, and the activity |
| audit stay in the app's local container. |
|
|
| This is **bounded foreground autonomy**. A run follows an |
| observe → plan → approve → execute → record → reassess loop while the app is |
| active. iOS may suspend the process after it enters the background, so this is |
| not an indefinitely running background agent. |
|
|
| ## What it can do |
|
|
| The assistant has one canonical registry of local tools: |
|
|
| - read the current date and time; |
| - evaluate arithmetic with a constrained calculator; |
| - search or save durable legacy memories; |
| - capture, search, and inspect typed project knowledge with exact provenance; |
| - synthesize bounded project context or a deterministic evidence-backed project |
| review; |
| - create, list, inspect, restore, and deactivate typed session handoffs without |
| replaying actions or deleting the checkpoint; |
| - derive deterministic continuity insights; |
| - inspect a bounded code snippet with non-executing static heuristics; |
| - list, search, add, complete, and reopen local tasks; and |
| - read a bounded list of Calendar event titles and times for today, tomorrow, |
| or the next seven days after explicit opt-in, iOS full-access authorization, |
| and one-time approval for the exact read. |
|
|
| The Chat tab is the command surface. Activity groups each durable run with its |
| timeline, counters, receipts, failures, and cancellation state. Knowledge |
| provides Knowledge, Handoffs, and Tasks sections for manual capture, local |
| hybrid lexical search, deterministic synthesis, proactive insights, continuity |
| packages, legacy memories, and directly completable tasks. Settings uses an |
| explicit draft-and-save flow for run limits and the system prompt, and |
| separately owns model lifecycle, resource diagnostics, and protected |
| system-context consent. |
|
|
| The **Ask Dolphin**, **Continue Dolphin**, and **Capture Context** App Shortcuts |
| open the app with a sanitized request placed in the composer for review. They |
| never send the request, load the model, invoke a tool, restore a handoff, save |
| knowledge, or request permission on their own. |
|
|
| The default policy is intentionally conservative: |
|
|
| - local writes require approval; |
| - Calendar is disabled until the user enables it in Dolphin Settings and iOS |
| grants full access; each exact bounded read still requires approval once; |
| - external network tools are unavailable until the connected peer can be |
| validated before a request is sent; |
| - unknown tools and malformed arguments are rejected; |
| - exact normalized calls and bounded observations are retained as structured |
| Activity receipts rather than copied into conversational text; local-write |
| receipts are committed atomically with their changes, and partial side |
| effects remain visible after a stop or failure; |
| - approval is tied to one exact action and argument payload, expires after ten |
| minutes, and is consumed once; |
| - duplicate tool calls are rejected within a run; |
| - deterministic commands declare their complete ordered action list and must |
| produce a receipt for every planned call before the run can succeed; these |
| plans can run without loading the language model because they generate no |
| model turn; |
| - broad model-assisted routing exposes read tools only; local writes require an |
| explicit deterministic command and normal approval; |
| - tool output is length-bounded and treated as untrusted data; and |
| - each run is limited to three tool steps by default, with cancellation |
| requested at a 300-second deadline and a 256-token response budget. |
|
|
| Continuity deletions show their cascade before removing linked handoffs or |
| digests. The original request and receipts remain auditable until the separate |
| **Clear Chat & Run Activity** action removes messages, run records, run-scoped |
| Activity, and copied recent-chat excerpts from retained handoffs. |
|
|
| ## Prepare the model and project |
|
|
| The checked-in Xcode project is generated from `project.yml`. Prepare the |
| pinned model and regenerate the project with: |
|
|
| ```bash |
| cd examples/ios/DolphinCoreMLTestApp |
| zsh prepare-model.sh |
| ``` |
|
|
| The script downloads |
| `ales27pm/Dolphin3.0-CoreML@v2.0.0`, verifies all three package-file SHA-256 |
| hashes and the five tokenizer/config hashes, and runs XcodeGen when it is |
| installed. Install XcodeGen with `brew install xcodegen` if necessary, or |
| regenerate explicitly: |
|
|
| ```bash |
| xcodegen generate |
| open DolphinCoreMLTestApp.xcodeproj |
| ``` |
|
|
| The app target remains `DolphinCoreMLTestApp`, its bundle identifier remains |
| `com.ales27pm.DolphinCoreMLTestApp`, and the installed display name is |
| **Dolphin Assistant**. |
|
|
| ## Build and run |
|
|
| In Xcode: |
|
|
| 1. Select the `DolphinCoreMLTestApp` target. |
| 2. Choose your Apple Developer Team under Signing & Capabilities. |
| 3. Select an iPhone running iOS 18 or newer, or a suitable simulator for UI |
| work. |
| 4. Choose Product → Run. |
| 5. Explicit local commands can run before model load. In Settings, load the |
| model and wait for the ready state before starting a generated conversation. |
| 6. Optional: use **Grant and Enable Calendar** in Settings. The permission |
| prompt can originate only from this button, never from an agent run. |
|
|
| For a command-line compile check: |
|
|
| ```bash |
| xcodebuild \ |
| -project DolphinCoreMLTestApp.xcodeproj \ |
| -scheme DolphinCoreMLTestApp \ |
| -configuration Debug \ |
| -destination 'generic/platform=iOS Simulator' \ |
| build |
| ``` |
|
|
| Run the agent-logic and protocol tests on the macOS host without launching a |
| Simulator or compiling and embedding the 1.81 GB model package: |
|
|
| ```bash |
| zsh scripts/test-agent-logic.sh |
| ``` |
|
|
| The equivalent direct command is: |
|
|
| ```bash |
| swift test \ |
| --package-path . \ |
| --scratch-path DerivedDataHostTests |
| ``` |
|
|
| The unit tests cover canonical JSON/signature generation, strict parsing of the |
| model's `<tool_call>` envelope, exact argument normalization, the production |
| tool surface, whole-turn context trimming, structured receipt compatibility, |
| oversized-result rejection, deterministic read-observation grounding, durable |
| run-checkpoint migration and recovery, and mixed-result fail-closed behavior. |
| They verify natural-language multi-step planning, plan-limit refusal before any |
| execution, no-model deterministic execution, receipt correlation before trust, |
| and rejection or removal of hidden-reasoning output. |
| They also cover Calendar privacy minimization and authorization denial, intent |
| handoff sanitization and one-shot consumption, persistence schema |
| compatibility, resource preflight policy, typed knowledge bounds and hybrid |
| lexical ranking, deterministic synthesis and insights, handoff lifecycle, |
| schema-v5 continuity round trips, grounded continuity-tool answers, and bounded |
| non-executing code analysis. The host package excludes SwiftUI, app lifecycle, |
| Core ML inference, and App Intents metadata generation. |
| These tests exercise app logic, not Core ML inference. A successful compile or |
| unit-test run is not proof of model-backed behavior; verify an actual model run |
| and inspect Activity for the corresponding model and tool evidence. |
|
|
| ## Runtime design |
|
|
| Dolphin's native tool protocol is rendered directly into the prompt. Each |
| model turn starts with a fresh Core ML state. Prompt prefill is split into |
| chunks of at most 512 tokens while reusing that state within the turn, unlocking |
| the model's full 2,048-token state capacity for prompt plus generated tokens. |
| Greedy decoding favors stable tool-call JSON, and public conversation history is |
| trimmed to fit the context budget. Scratch tool exchanges are scoped to the |
| current run; private chain-of-thought is neither requested nor persisted. |
| User text, tool observations, and reconstructed tool-call arguments have all |
| tokenizer control-token sequences neutralized before they re-enter a prompt. |
| Explicit memory and task writes are routed deterministically through the same |
| approval and receipt path, while ordinary conversation receives no irrelevant |
| tool definitions. Legacy receipt prose from earlier builds is removed from |
| saved chat during workspace migration so it cannot contaminate later prompts. |
|
|
| Explicit multi-action requests joined with **then** become a bounded sequential |
| plan. The plan records tool names and required receipt identities—not private |
| chain-of-thought—and appears in Activity. Once every deterministic step has a |
| trusted receipt, Dolphin renders the answer directly from those receipts with |
| zero follow-up model turns. A fail-closed tool-only runtime lets this path work |
| without loading the 1.81 GB Core ML model; an accidental generation request is |
| rejected. Missing tools, denied actions, missing receipts, and plans over the |
| configured step limit fail closed. Complete hidden-reasoning blocks are removed |
| from model output, while malformed or reasoning-only output is rejected as an |
| empty response. |
|
|
| The schema-v5 workspace adds typed knowledge, deterministic digests, session |
| handoffs, and active-handoff selection. Saving a legacy memory also mirrors the |
| same content into typed knowledge when capacity permits. Search uses bounded, |
| deterministic BM25-like, phrase, coverage, and trigram signals with stable tie |
| breaking; it is hybrid lexical retrieval, not vector or embedding search. A |
| digest is derived after every fifth captured knowledge item, and restored |
| handoffs contribute context only—they cannot replay tools or approvals. |
|
|
| Stop requests are observed between Core ML predictions. Cancellation or failure |
| discards the current state, and overlapping runs are prevented. The app cancels |
| foreground work when it moves to the background rather than claiming work that |
| iOS may suspend. |
|
|
| Protected Calendar reads have four independent gates: explicit app opt-in, iOS |
| full-access authorization, approval for the normalized request, and an active |
| foreground scene before and after EventKit access. Only sanitized titles, |
| start/end times, and all-day flags cross the system-tool boundary; identifiers, |
| notes, attendees, URLs, and locations never do. |
|
|
| Each request also has a durable run record and checkpoint. The journal records |
| the current phase, active tool call, completed receipts, counters, and terminal |
| status. If iOS ends the process during a run, the next launch preserves that |
| evidence and marks the run interrupted instead of pretending it completed or |
| silently replaying an action. Local mutations and their corresponding receipts |
| remain one atomic persistence operation. |
|
|
| ## Limitations |
|
|
| - The package is about 1.81 GB. Its fixed FP16 KV caches use roughly 224 MiB |
| before model weights, activations, and runtime buffers. Use a recent, |
| high-memory iPhone with several gigabytes of free storage. |
| - The context window is 2,048 tokens total, including tool definitions, |
| conversation, tool observations, and generated output. Older history may be |
| omitted from a turn to stay within that limit. |
| - Generation is deliberately bounded and can be slow on older devices. A stop |
| cannot interrupt a Core ML prediction already in progress. |
| - Direct networking is intentionally unavailable in this build. URL syntax and |
| a preliminary DNS lookup are not enough to prevent DNS rebinding; a future |
| network tool must pin and validate the actual connected peer before sending. |
| - Tools operate only on the assistant's local store and explicitly approved |
| Calendar context. The app does not yet control Mail, read Contacts or |
| Location, execute arbitrary Shortcuts, or access other system services. |
| - Create State-inspired continuity is an independent, single-workspace local |
| implementation. It has no Create State account or compatibility layer, |
| embeddings, cloud sync, repository/media imports, team graph or RBAC, |
| background monitoring, or hosted MCP dependency. See |
| [CREATE_STATE_INTEGRATION.md](CREATE_STATE_INTEGRATION.md) for the immutable |
| public-source inventory and exact parity boundary. |
| - Local persistence is app-container storage, not a synced account or an |
| encrypted secrets vault. Deleting the app deletes that data unless it is |
| otherwise backed up by iOS. If the store cannot be opened or decoded, the |
| assistant fails closed instead of presenting volatile data as saved. |
| - Conversational responses and model-assisted read selection can be wrong. |
| Deterministic plans remain narrow host rules, and verified tool observations |
| remain the source of truth. The approval boundary reduces side effects; it |
| does not make model output factual. |
|
|