crystal-ball: flashnext C1 two-floor guesser

A small side file for a fork of llama.cpp that streams the Qwen3.8-Flash-Next experts from SSD. For each floor n (2 to 47) it reads the router input from floor n-2 and scores all 512 experts of floor n, so the fork can start fetching them a floor early. It changes which experts are fetched early, never which are used: output is unchanged.

  • Shape: one linear layer per floor, 2560 -> 512 plus bias; f16 weights, f32 bias; 92 tensors, 120,686,368 bytes.
  • Confidence: min(1, 10 * softmax(z)); the fork fetches experts above a cut-off (we use 0.2, LLAMA_MOE_STREAM_LOOKAHEAD_CONF=0.2).
  • Use it with: the fork's party/trained-lookahead branch (engine b4745dbc3 or later), with LLAMA_MOE_STREAM_LOOKAHEAD_ALL=1 ..._DEPTH2=4 ..._GUESSER=<this file> ..._CONF=0.2 ..._KEEP=1. Only valid for Qwen3.8-Flash-Next (n_embd 2560, 512 experts); the engine refuses a shape mismatch.
  • Quality: two-floor top-10 recall (mean of floors 2-47) 67.2% on held-out generic papers, 71-72% on held-out chat/code papers. On a Mac mini M5 Pro 64GB with the 28 GiB cache and the MTP draft: writing +4.2% (code, high effort) and +8.2% (chat, medium effort) against the same engine without it; the gain is +6-11% on papers up to long and about +1% past 100K tokens of context.
  • sha256: c22af709d65b99bb2bc64aaa5b8b2d47c04c86367b0f9ba0fe452ab47ec053c7

Trained on the router inputs recorded while the model wrote replies to benchmark and coding/chat prompts.

Downloads last month
1
GGUF
Model size
60.3M params
Architecture
Hardware compatibility
Log In to add your hardware

We're not able to determine the quantization variants.

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support