Post
155
🚀 opencoti-llamafile 0.10.5-c7 — a serving API agents can trust
New cut of the opencoti single-file inference engine, rebased onto llamafile 0.10.5 / llama.cpp (mozilla-ai) with 190 additive patches (series published in the repo). Same zero-dependency APE: one executable for Linux, Windows, macOS & BSD.
What's new vs c6:
PolyKV admission is now atomic and enforced by default. Every admit is a reservation against the pool's KV budget — the optimistic check-then-book race under concurrent agent spawns is gone (was 0/4/5 nondeterministic refusals on the same binary; now deterministic booked/refused accounting). Pool-less servers get the same math via --admission-poolless (a clean refusal in 0.38s instead of an 11s stall), sequences can be partially evicted instead of dropped, and SWA models size and admit on the same budget model (--swa-seq-budget: −40 GiB KV measured).
llamafile as an agent skill. opencoti now ships an opt-in plugin + embedded skill that teaches AI agents to drive the local engine: launch it, check capacity before spawning sub-agents, fork shared-prefix pools from a live session so N agents share one cached system prompt, token-exact.
Agentic reliability: Gemma-4 tool-call argument bleed fixed. An un-closed string argument no longer swallows the brace, thought channel and the next tool call — contained at map time, grammar untouched.
Vulkan ships for the first time. x86_64 + Windows Vulkan side-load DSOs (AMD / iGPU / RADV) alongside the CUDA ones.
Per-cut side-load isolation. The DSO cache dir is namespaced by the full engine version, so different cuts no longer collide on extraction.
Binaries (Linux, Windows, Windows-GPU, aarch64, universal), side-load DSOs and docs: ManniX-ITA/opencoti-llamafile
New cut of the opencoti single-file inference engine, rebased onto llamafile 0.10.5 / llama.cpp (mozilla-ai) with 190 additive patches (series published in the repo). Same zero-dependency APE: one executable for Linux, Windows, macOS & BSD.
What's new vs c6:
PolyKV admission is now atomic and enforced by default. Every admit is a reservation against the pool's KV budget — the optimistic check-then-book race under concurrent agent spawns is gone (was 0/4/5 nondeterministic refusals on the same binary; now deterministic booked/refused accounting). Pool-less servers get the same math via --admission-poolless (a clean refusal in 0.38s instead of an 11s stall), sequences can be partially evicted instead of dropped, and SWA models size and admit on the same budget model (--swa-seq-budget: −40 GiB KV measured).
llamafile as an agent skill. opencoti now ships an opt-in plugin + embedded skill that teaches AI agents to drive the local engine: launch it, check capacity before spawning sub-agents, fork shared-prefix pools from a live session so N agents share one cached system prompt, token-exact.
Agentic reliability: Gemma-4 tool-call argument bleed fixed. An un-closed string argument no longer swallows the brace, thought channel and the next tool call — contained at map time, grammar untouched.
Vulkan ships for the first time. x86_64 + Windows Vulkan side-load DSOs (AMD / iGPU / RADV) alongside the CUDA ones.
Per-cut side-load isolation. The DSO cache dir is namespaced by the full engine version, so different cuts no longer collide on extraction.
Binaries (Linux, Windows, Windows-GPU, aarch64, universal), side-load DSOs and docs: ManniX-ITA/opencoti-llamafile