immortaltatsu/ghostai-lfm-app

GhostAI LFM app-contract SFT

LFM2.5-1.2B-Thinking fine-tuned to the GhostWallet mobile app's real on-device tool contract — one Hermes <tool_call> block at a time, chosen from the app's retrieved tool catalog.

Earlier Ghost AI checkpoints were trained against plan-schema.md (a plan object, then bare per-step argument JSON). The app expects something different, so those models emitted argument objects with no tool name and the app's parser discarded them. This model targets the app contract directly.

Results — app harness (56 held-out utterances, Q4_K_M, greedy)

metric	prior checkpoint	this model
emitted a parseable tool call	35.7%	92.9%
args passed the app's validator	32.1%	87.5%
called the expected tool	16.1%	23.2%
emitted plan JSON (wrong contract)	—	0

tool_correct is capped by retrieval, not by the model. The app surfaces the correct tool in its top-5 catalog only 10.7% of the time on these utterances, and a model cannot call a tool it is never offered. Forcing the correct tool into the catalog (GHOSTAI_BOOST=1) isolates the model:

with correct tool in catalog	prior	this model
emitted a parseable call	26.8%	100%
called the expected tool	21.4%	98.2%
args valid	—	94.6%

Tool selection and argument construction are effectively solved (98.2%). The remaining production gap is the app's tool-retrieval ranking, which is an app-side concern. Note this measures the hash-embedding fallback ranking; the on-device path loads a real embedding model and is untested here.

Results — end-to-end harness (76 cases, 88 turns, app's real ChatSession)

	baseline	this model
overall	44%	56.6%
grounding	24%	44.1%
value gate	67%	75%
injection resistance	100%	94.4%
multi-turn	17%	16.7%
gate bypasses	0	0
planted content reaching a tool arg	0	0
completion tokens / turn	63	40.7

Known regressions and weaknesses

1. Injection resistance regressed, 100% → 94.4%. One adversarial case now fails. Both absolute invariants still hold — no confirm-gate bypass, and no planted content reached a tool argument — so this is the milder failure class, but it is a regression on a privacy-first product and should be reviewed before shipping.
2. Multi-turn is weak (16.7%) and did not improve. Follow-ups, pronoun resolution and topic switches remain unreliable.
3. 5 turns invented numbers not present in tool output, and 16 turns called tools outside the stub set.
4. Retrieval, not the model, is the production bottleneck (10.7% primary_retrievable). Fixing model quality further will not move end-to-end numbers until retrieval improves.

Training

data	5,407 app-contract traces covering 174/174 app tools (previously 58/174)
generation	teacher-distilled (Qwen3.6-35B-A3B), every argument object validated against its tool's schema before acceptance — 98% acceptance
prompts	wrapped by the app's own ContextManager.assembleMessages, so they are byte-identical to what ships
objective	SFT, assistant-only loss (full-sequence loss teaches the model to reproduce rendered tool-call history verbatim)
epochs	2, bf16, DeepSpeed ZeRO-3, 2x A100
eval	0.1705 loss / 0.963 token accuracy; split by utterance-template skeleton

Files

File	Size
lfm-app-sft-F16.gguf	2.3 GB
lfm-app-sft-Q4_K_M.gguf	695 MB — shipping quantization

Usage

Serve with llama.cpp (the app embeds llama.rn):

llama-server -m lfm-app-sft-Q4_K_M.gguf --port 8099 -ngl 99 -c 8192 --jinja

The model expects the app's assembled system prompt (tool-call format instructions plus a compact catalog of retrieved tools) and emits:

<tool_call>{"name":"send_sol","arguments":{"recipient":"mom","amount":5}}</tool_call>

License

Base model is under the LFM Open License, which permits commercial use only below $10M annual revenue (§5); above that threshold commercial use requires a separate agreement with Liquid AI. That condition attaches to derivative works, including this one.