prune: remove superseded/non-functional artifacts

README.md

@@ -18,10 +18,10 @@ tags:
 
 # LFM2.5-8B-A1B — Core AI (.aimodel)
 
-`LiquidAI/LFM2.5-8B-A1B` as Core AI `.aimodel` bundles for Apple silicon, published by
-[visible-cx](https://huggingface.co/visible-cx). `gpu-pipelined/` is a byte mirror of the
-Core AI model zoo bundle; `mf64/` is a re-export that corrects two defects in it (see
-**Defects in the published bundle**).
+`LiquidAI/LFM2.5-8B-A1B` as a Core AI `.aimodel` bundle for Apple silicon, published by
+[visible-cx](https://huggingface.co/visible-cx). `mf64/` is a re-export that adds the
+S=64 chunked-prefill entrypoint and a terminated reasoning-block chat template; the
+defective upstream mirror it replaces has been removed from this repo.
 
 It is a **sparse mixture-of-experts** model: 8.3B total parameters, ~1.5B active per token.
 24 layers — 18 short-convolution mixers and 6 GQA attention layers; the first two layers are
@@ -35,70 +35,42 @@ weights read as a dense over-read.
 | | |
 |---|---|
 | Base checkpoint | `LiquidAI/LFM2.5-8B-A1B` |
-| Upstream bundle (`gpu-pipelined/`) | `mlboydaisuke/LFM2.5-8B-A1B-CoreAI` @ `1e494ef6dd5c499e18d5c1730ddf4bda76b9e039` |
 | Zoo recipe | `models/lfm2.5-8b-a1b-moe/recipe.toml` → `["lfm2.5-8b-a1b"]`, `status = "verified"` |
-| Recipe (`mf64/`) | `export_lfm2_moe_multifunction.py` — `metalize_moe_batched` installs `BatchedMetalSwitchGLU` |
+| Recipe | `export_lfm2_moe_multifunction.py` — `metalize_moe_batched` installs `BatchedMetalSwitchGLU` |
 | Producer fingerprint | `coreai-core 1.0.0b2` on every inner asset `metadata.json` |
-| Asset creation dates | `20260718T140139Z` (`gpu-pipelined/`), `20260817T231316Z` (`mf64/`) |
+| Asset creation date | `20260817T231316Z` |
 | Weight format | **`sym8`** — symmetric-linear **int8**, per-K-block-32 scale, read through the `gather_qmm` Metal kernel |
 | Vocab | 128,000 |
-| Export functions | `gpu-pipelined/`: `main` only. `mf64/`: `main` (S=1 decode) + `prefill` (S=64) |
+| Export functions | `main` (S=1 decode) + `prefill` (S=64) |
 
 `sym8`, not int4. An `int4km-gather` variant exists upstream and is deliberately not
 published: non-QAT int4 does not hold quality on this model (+12 output flips out of 41
 against the fp16 reference), where `sym8` is clean (+1 flip out of 41, the fp16 ceiling).
 
-In `mf64/`, non-expert weights are the shipped int8 per-block-32 recipe calibrated on the
-decode spec, so they are bit-identical to `gpu-pipelined/`. `BatchedMetalSwitchGLU` serves
-S=64 by sorting (token, expert) pairs and falls back to the q=1 gather path at S=1, so both
-entrypoints keep the gather kernel.
+Non-expert weights are the shipped int8 per-block-32 recipe calibrated on the decode spec.
+`BatchedMetalSwitchGLU` serves S=64 by sorting (token, expert) pairs and falls back to the
+q=1 gather path at S=1, so both entrypoints keep the gather kernel.
 
 ## Contents
 
 | Path | Bytes | Files | Manifest context | Functions |
 |---|---:|---|---|---|
-| `gpu-pipelined/lfm2_5_8b_a1b_decode_sym8_gather` | 9,447,532,211 | 7 | 8192 | main (decode only) |
 | `mf64/lfm2_5_8b_a1b_decode_sym8_gather_mf64` | 9,447,767,758 | 7 | 16384 | main + prefill |
 
-Each folder holds `<name>.aimodel/` (`main.mlirb` ≈ 9.43 GB, `main.hash`, asset
+The folder holds `<name>.aimodel/` (`main.mlirb` ≈ 9.43 GB, `main.hash`, asset
 `metadata.json`), a bundle-level `metadata.json`, and `tokenizer/` (`tokenizer.json`,
 `tokenizer_config.json`, `chat_template.jinja`).
 
-**Stop token:** `eos_token = "<|im_end|>"` in both folders.
+**Stop token:** `eos_token = "<|im_end|>"`.
 
-## Defects in the published bundle
+## Chat template
 
-Two properties of `gpu-pipelined/` materially affect how it behaves. Both are corrected in
-`mf64/`, and the correction is measured — see **Performance**.
-
-**1. No `prefill` function.** `gpu-pipelined/` declares
-`function_map: {"main": ["main"]}`. Prefill therefore runs one token per pass, at a flat
-**~21.5 ms per prompt token**, and time-to-first-token dominates everything else the bundle
-does:
-
-| prompt | prompt tokens | TTFT | decode | TTFT share of wall |
-|---|---|---|---|---|
-| guided sample | 765 | 17.1 s | 45.9 tok/s | 93 % |
-| free-form 3.4k | 3,249 | 70.1 s | 47.0 tok/s | 79 % |
-| free-form 8k | 7,673 | 165.1 s | 45.2 tok/s | 93 % |
-
-`mf64/`'s S=64 prefill entrypoint reduces this to **6.3 ms per prompt token**, a **3.4×**
-speed-up, measured below.
-
-**2. The chat template does not terminate the reasoning block.** `gpu-pipelined/`'s
-`chat_template.jinja` emits `{{- "<|im_start|>assistant\n" -}}` with no `<think></think>`.
-LFM2.5-8B-A1B is an always-thinking model, so free-form generation spends the whole budget
-inside an unterminated `<think>` block: a 900-token generation produced **0 characters** of
-response. Guided decoding is unaffected, because the grammar forces structured output from
-the first token. Terminating the block —
-`{{- "<|im_start|>assistant\n<think></think>\n" -}}`, the same form the LFM2.5-2.6B bundles
-use — turns that run into a 4,838-character answer.
-
-`mf64/` fixes both: it adds a static S=64 chunked-prefill entrypoint sharing one copy of
-the `sym8` weights, and it emits the terminated think block in the generation prompt. Both
-fixes are confirmed by measurement: free-form generation returns **4,561 characters** with
-the bundle exactly as published, and prompt-token counts are uniformly **+3** against
-`gpu-pipelined/` on identical inputs — the `<think></think>` tokens.
+LFM2.5-8B-A1B is an always-thinking model, so the generation prompt terminates the
+reasoning block — `{{- "<|im_start|>assistant\n<think></think>\n" -}}`, the same form the
+LFM2.5-2.6B bundles use. Free-form generation therefore returns text with the bundle
+exactly as published (**4,561 characters** on the 3.4k measurement below) rather than
+spending the whole budget inside an unterminated `<think>` block. Guided decoding is
+unaffected either way, because the grammar forces structured output from the first token.
 
 ## Requirements
 
@@ -106,7 +78,7 @@ the bundle exactly as published, and prompt-token counts are uniformly **+3** ag
   an iPhone even with the memory-limit entitlement.
 - **Engine contract: 2 inputs** (`input_ids`, `position_ids`) → logits. No static inputs,
   no per-step mask.
-- **Memory, measured:** max RSS **9.0–11.0 GB** across both folders, peak in-process
+- **Memory, measured:** max RSS **9.0–10.7 GB**, peak in-process
   footprint 0.16–0.39 GB. **Minimum practical machine memory: 16 GB**, with little else
   resident — the model runs on that tier but leaves almost no headroom. 24 GB or more is
   the comfortable tier.
@@ -121,21 +93,19 @@ Measured on a **16 GB Apple silicon Mac (M2 Pro, macOS 27 beta)**, sequential en
 `reset()` between samples. Load excluded from `s/row`; sample 1 excluded as a cache-warm
 outlier.
 
-### `mf64/` — preferred bundle
+Chat template unmodified.
 
-Chat template unmodified. Same harness and prompts as the superseded bundle (below); the comparison column restates its figures.
-
-| | `gpu-pipelined/` | `mf64/` |
-|---|---|---|
-| Guided JSON parse | 10/10 | **10/10** |
-| Enum-clean | 10/10 | **10/10** |
-| s/row (long samples) | 17.88 | **7.70** |
-| s/row (short samples) | 8.59 | **4.71** |
-| Decode, guided | 45.2–46.6 tok/s | **51.9–53.5 tok/s** |
-| TTFT, guided | 6.5–23.3 s | **1.8–13.8 s** |
-| Prefill | 21.5 ms/token | **6.3 ms/token** |
-| Peak footprint | 0.16 GB | 0.39 GB |
-| Max RSS | 10.61 GB | 9.04 GB |
+| | `mf64/` |
+|---|---|
+| Guided JSON parse | **10/10** |
+| Enum-clean | **10/10** |
+| s/row (long samples) | **7.70** |
+| s/row (short samples) | **4.71** |
+| Decode, guided | **51.9–53.5 tok/s** |
+| TTFT, guided | **1.8–13.8 s** |
+| Prefill | **6.3 ms/token** |
+| Peak footprint | 0.39 GB |
+| Max RSS | 9.04 GB |
 
 Unguided, 900-token cap:
 
@@ -144,54 +114,24 @@ Unguided, 900-token cap:
 | 3.4k | 3,249 | **22.34 s** | 50.97 tok/s | 821 | **38.4 s** | 4,561 | 10.73 GB |
 | 8k | 7,673 | **48.07 s** | 47.67 tok/s | 900 | **66.8 s** | 5,084 | 10.55 GB |
 
-**Needle-in-haystack: 3/3 at 8k and 3/3 at 15k.** 8k returns the same answer as
-`gpu-pipelined/` at 7,813 prompt tokens, with TTFT **49.60 s** rather than 168.09 s. 15k
-is newly reachable — 14,566 prompt tokens, TTFT **91.41 s**, 44.63 tok/s, peak footprint
-0.39 GB — because the manifest declares 16384 and the prefill entrypoint makes that depth
-practical. Per-prompt-token prefill cost is the same 6.3 ms at 8k and 15k, so it is linear
-in depth.
+**Needle-in-haystack: 3/3 at 8k and 3/3 at 15k.** Three distinctive facts planted at
+10/50/90 % of the filler, strict scoring. 8k: 7,813 prompt tokens, TTFT **49.60 s**. 15k:
+14,566 prompt tokens, TTFT **91.41 s**, 44.63 tok/s, peak footprint 0.39 GB — reachable
+because the manifest declares 16384 and the prefill entrypoint makes that depth practical.
+Per-prompt-token prefill cost is the same 6.3 ms at 8k and 15k, so it is linear in depth.
 
 **Cold-load caveat.** The first load of a freshly downloaded copy was **60.6 s**, since it
 includes the graph compile. Subsequent loads were **22.9–26.2 s**. Budget roughly 17 GB of
 on-disk compile cache in addition to the 9.4 GB bundle.
 
-### `gpu-pipelined/` — superseded
-
-| | |
-|---|---|
-| Cold load | 21.3 s |
-| Guided JSON parse | **10/10** |
-| Enum-clean | **10/10** |
-| s/row (long samples) | **17.88** |
-| s/row (short samples) | **8.59** |
-| Decode | **45.2–46.6 tok/s** |
-| TTFT | **6.5–23.3 s** |
-| Peak footprint | 0.16 GB |
-| Max RSS | 10.61 GB |
-
 Decode is healthy for the size — an MoE with ~1B active parameters decoding faster than a
-dense 2.6B. **The per-sample cost is prefill, not decode**: at 17.88 s/row it is roughly
-10× the LFM2.5-1.2B, entirely because of the missing prefill function.
-
-Unguided, free-form generation from a fixed prompt, 900-token cap, with the chat template
-patched to terminate the think block (unpatched, these runs return 0 characters).
-
-| depth | prompt tokens | TTFT | decode | generated | wall | peak footprint | max RSS |
-|---|---|---|---|---|---|---|---|
-| 3.4k | 3,249 | 70.05 s | 47.0 tok/s | 900 | 89.0 s | 0.21 GB | 10.58 GB |
-| 8k | 7,673 | 165.11 s | 45.2 tok/s | 519 | 176.8 s | 0.27 GB | 11.03 GB |
-
-**Needle-in-haystack recall at 8k: 3/3** (this superseded bundle). Three distinctive facts planted at 10/50/90% of
-the filler, strict scoring; returned at 7,813 prompt tokens in a 27-token answer
-(TTFT 168.09 s, 45.3 tok/s). 15k was not run on this superseded bundle (`mf64/` above passes 3/3 at 15k): its manifest ceiling is
-8192, and S=1 prefill at that depth would cost roughly five minutes.
+dense 2.6B.
 
 ## Status
 
 | Artifact | Status |
 |---|---|
-| `gpu-pipelined/lfm2_5_8b_a1b_decode_sym8_gather` | **SUPERSEDED by `mf64/`. Qualified for guided generation, with defects.** Measured: 10/10 guided parse and enum-clean, 45.2–46.6 tok/s decode, 17.88/8.59 s/row, 3/3 needle recall at 8k. As published it has no prefill function (TTFT dominates: ~21.5 ms/prompt token) and its chat template does not terminate the reasoning block, so **free-form generation returns empty output unless the template is corrected**. |
-| `mf64/lfm2_5_8b_a1b_decode_sym8_gather_mf64` | **QUALIFIED. Preferred bundle.** Measured: 10/10 guided parse and enum-clean, 51.9–53.5 tok/s decode, 7.70/4.71 s/row, 3/3 needle recall at both 8k and 15k, and non-empty free-form output with the chat template unmodified. Corrects both defects in `gpu-pipelined/`: adds the S=64 prefill entrypoint (prefill 21.5 → 6.3 ms/token) and terminates the reasoning block. Manifest context 16384. |
+| `mf64/lfm2_5_8b_a1b_decode_sym8_gather_mf64` | **QUALIFIED.** Measured: 10/10 guided parse and enum-clean, 51.9–53.5 tok/s decode, 7.70/4.71 s/row, 3/3 needle recall at both 8k and 15k, and non-empty free-form output with the chat template unmodified. Manifest context 16384. |
 
 ## License
 

gpu-pipelined/lfm2_5_8b_a1b_decode_sym8_gather/lfm2_5_8b_a1b_decode_sym8_gather.aimodel/main.hash

@@ -1 +0,0 @@
-j{AC.��y���T�
_%��B
!{�!?qBu

gpu-pipelined/lfm2_5_8b_a1b_decode_sym8_gather/lfm2_5_8b_a1b_decode_sym8_gather.aimodel/main.mlirb

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:6a7b0c41432ed31f9e79ab869b10065496015f2590894201217bc4213f714275
-size 9429620942

gpu-pipelined/lfm2_5_8b_a1b_decode_sym8_gather/lfm2_5_8b_a1b_decode_sym8_gather.aimodel/metadata.json

@@ -1,5 +0,0 @@
-{
-  "assetVersion" : "2.0",
-  "producer" : "coreai-core 1.0.0b2",
-  "creationDate" : "20260718T140139Z"
-}

gpu-pipelined/lfm2_5_8b_a1b_decode_sym8_gather/metadata.json

@@ -1,28 +0,0 @@
-{
-  "metadata_version": "0.2",
-  "kind": "llm",
-  "name": "lfm2_5_8b_a1b_decode_sym8_gather",
-  "assets": {
-    "main": "lfm2_5_8b_a1b_decode_sym8_gather.aimodel"
-  },
-  "language": {
-    "tokenizer": "LiquidAI/LFM2.5-8B-A1B",
-    "vocab_size": 128000,
-    "max_context_length": 8192,
-    "embedded_tokenizer": true,
-    "function_map": {
-      "main": [
-        "main"
-      ]
-    }
-  },
-  "source": {
-    "model_definition": "torch",
-    "hf_model_id": "LiquidAI/LFM2.5-8B-A1B"
-  },
-  "compression": null,
-  "compilation": {
-    "date": "2026-07-18T14:01:42.873532+00:00",
-    "targets": []
-  }
-}

gpu-pipelined/lfm2_5_8b_a1b_decode_sym8_gather/tokenizer/chat_template.jinja

@@ -1,115 +0,0 @@
-{{- bos_token -}}
-{%- set preserve_thinking = preserve_thinking | default(false) -%}
-
-{%- macro format_arg_value(arg_value) -%}
-    {%- if arg_value is string -%}
-        {{- "'" + arg_value + "'" -}}
-    {%- elif arg_value is mapping -%}
-        {{- arg_value | tojson -}}
-    {%- else -%}
-        {{- arg_value | string -}}
-    {%- endif -%}
-{%- endmacro -%}
-
-{%- macro parse_content(content) -%}
-    {%- if content is string -%}
-        {{- content -}}
-    {%- else -%}
-        {%- set _ns = namespace(result="") -%}
-        {%- for item in content -%}
-            {%- if item["type"] == "image" -%}
-                {%- set _ns.result = _ns.result + "<image>" -%}
-            {%- elif item["type"] == "text" -%}
-                {%- set _ns.result = _ns.result + item["text"] -%}
-            {%- else -%}
-                {%- set _ns.result = _ns.result + item | tojson -%}
-            {%- endif -%}
-        {%- endfor -%}
-        {{- _ns.result -}}
-    {%- endif -%}
-{%- endmacro -%}
-
-{%- macro render_tool_calls(tool_calls) -%}
-    {%- set tool_calls_ns = namespace(tool_calls=[]) -%}
-    {%- for tool_call in tool_calls -%}
-        {%- set func_name = tool_call["function"]["name"] -%}
-        {%- set func_args = tool_call["function"]["arguments"] -%}
-        {%- set args_ns = namespace(arg_strings=[]) -%}
-        {%- for arg_name, arg_value in func_args.items() -%}
-            {%- set args_ns.arg_strings = args_ns.arg_strings + [arg_name + "=" + format_arg_value(arg_value)] -%}
-        {%- endfor -%}
-        {%- set tool_calls_ns.tool_calls = tool_calls_ns.tool_calls + [func_name + "(" + (args_ns.arg_strings | join(", ")) + ")"] -%}
-    {%- endfor -%}
-    {{- "<|tool_call_start|>[" + (tool_calls_ns.tool_calls | join(", ")) + "]<|tool_call_end|>" -}}
-{%- endmacro -%}
-
-{%- set ns = namespace(system_prompt="", last_user_index=-1) -%}
-{%- if messages[0]["role"] == "system" -%}
-    {%- if messages[0].get("content") -%}
-        {%- set ns.system_prompt = parse_content(messages[0]["content"]) -%}
-    {%- endif -%}
-    {%- set messages = messages[1:] -%}
-{%- endif -%}
-{%- if tools -%}
-    {%- set ns.system_prompt = ns.system_prompt + ("\n" if ns.system_prompt else "") + "List of tools: [" -%}
-    {%- for tool in tools -%}
-        {%- if tool is not string -%}
-            {%- set tool = tool | tojson -%}
-        {%- endif -%}
-        {%- set ns.system_prompt = ns.system_prompt + tool -%}
-        {%- if not loop.last -%}
-            {%- set ns.system_prompt = ns.system_prompt + ", " -%}
-        {%- endif -%}
-    {%- endfor -%}
-    {%- set ns.system_prompt = ns.system_prompt + "]" -%}
-{%- endif -%}
-{%- if ns.system_prompt -%}
-    {{- "<|im_start|>system\n" + ns.system_prompt + "<|im_end|>\n" -}}
-{%- endif -%}
-{%- for message in messages -%}
-    {%- if message["role"] == "user" -%}
-        {%- set ns.last_user_index = loop.index0 -%}
-    {%- endif -%}
-{%- endfor -%}
-{%- for message in messages -%}
-    {{- "<|im_start|>" + message.role + "\n" -}}
-    {%- if message.role == "assistant" -%}
-        {%- generation -%}
-        {%- if message.thinking is defined and (preserve_thinking or loop.index0 > ns.last_user_index) -%}
-            {{- "<think>" + message.thinking + "</think>" -}}
-        {%- endif -%}
-        {%- set _cfm_tag = "CONTINUE_FINAL_MESSAGE_TAG " -%}
-        {%- set _has_cfm = false -%}
-        {%- if message.content is defined -%}
-            {%- set content = parse_content(message.content) -%}
-            {%- if not (preserve_thinking or loop.index0 > ns.last_user_index) -%}
-                {%- if "</think>" in content -%}
-                    {%- set content = content.split("</think>")[-1] | trim -%}
-                {%- endif -%}
-            {%- endif -%}
-            {%- if message.tool_calls is defined and content.endswith(_cfm_tag) -%}
-                {%- set _has_cfm = true -%}
-                {%- set _trunc_len = (content | length) - (_cfm_tag | length) -%}
-                {{- content[:_trunc_len] -}}
-            {%- else -%}
-                {{- content -}}
-            {%- endif -%}
-        {%- endif -%}
-        {%- if message.tool_calls is defined -%}
-            {{- render_tool_calls(message.tool_calls) -}}
-        {%- endif -%}
-        {%- if _has_cfm -%}
-            {{- _cfm_tag -}}
-        {%- endif -%}
-        {{- "<|im_end|>\n" -}}
-        {%- endgeneration -%}
-    {%- else %}
-        {%- if message.get("content") -%}
-            {{- parse_content(message["content"]) -}}
-        {%- endif -%}
-        {{- "<|im_end|>\n" -}}
-    {%- endif %}
-{%- endfor -%}
-{%- if add_generation_prompt -%}
-    {{- "<|im_start|>assistant\n" -}}
-{%- endif -%}

gpu-pipelined/lfm2_5_8b_a1b_decode_sym8_gather/tokenizer/tokenizer.json

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:695be7802a0e4b8a81048f0ff5ebb7fc811a0ba5a6be63dbb24deb5a81096f41
-size 17905598

gpu-pipelined/lfm2_5_8b_a1b_decode_sym8_gather/tokenizer/tokenizer_config.json

@@ -1,11 +0,0 @@
-{
-  "backend": "tokenizers",
-  "bos_token": "<|startoftext|>",
-  "clean_up_tokenization_spaces": false,
-  "eos_token": "<|im_end|>",
-  "is_local": true,
-  "model_max_length": 1000000000000000019884624838656,
-  "pad_token": "<|pad|>",
-  "tokenizer_class": "TokenizersBackend",
-  "use_default_system_prompt": false
-}