crystal-ball: flashnext C1 two-floor guesser
A small side file for a fork of llama.cpp that streams the Qwen3.8-Flash-Next experts from SSD. For each floor n (2 to 47) it reads the router input from floor n-2 and scores all 512 experts of floor n, so the fork can start fetching them a floor early. It changes which experts are fetched early, never which are used: output is unchanged.
- Shape: one linear layer per floor, 2560 -> 512 plus bias; f16 weights, f32 bias; 92 tensors, 120,686,368 bytes.
- Confidence:
min(1, 10 * softmax(z)); the fork fetches experts above a cut-off (we use 0.2,LLAMA_MOE_STREAM_LOOKAHEAD_CONF=0.2). - Use it with: the fork's
party/trained-lookaheadbranch (engine b4745dbc3 or later), withLLAMA_MOE_STREAM_LOOKAHEAD_ALL=1 ..._DEPTH2=4 ..._GUESSER=<this file> ..._CONF=0.2 ..._KEEP=1. Only valid for Qwen3.8-Flash-Next (n_embd 2560, 512 experts); the engine refuses a shape mismatch. - Quality: two-floor top-10 recall (mean of floors 2-47) 67.2% on held-out generic papers, 71-72% on held-out chat/code papers. On a Mac mini M5 Pro 64GB with the 28 GiB cache and the MTP draft: writing +4.2% (code, high effort) and +8.2% (chat, medium effort) against the same engine without it; the gain is +6-11% on papers up to long and about +1% past 100K tokens of context.
- sha256:
c22af709d65b99bb2bc64aaa5b8b2d47c04c86367b0f9ba0fe452ab47ec053c7
Trained on the router inputs recorded while the model wrote replies to benchmark and coding/chat prompts.
- Downloads last month
- 1
Hardware compatibility
Log In to add your hardware
We're not able to determine the quantization variants.
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support