Qwen3.8-27B-Coder390 FP8 with V15 episodic memory

This package runs Coder390 with its matched DFlash2 draft in the native NInfer V15 server. The original FP8 text weights, vision tower, MTP head and DFlash2 component are packaged together. The V15 configuration, tokenizer and accompanying code enable a persistent episodic store. The image runs on a single Blackwell GPU with sm_120a support.

The 30.3 GB pack is byte-identical to the upstream Coder390 FP8 artifact. Its SHA-256 is ebacda594ff7bdce5ab13eac88ea1b15dff0df0fffb4691c1b333a88a4baceb3. The tokenizer, processor settings and chat template were extracted from that same artifact.

Run

hf download LTM-Lab/Qwen3.8-27B-Coder390-FP8-DFlash2-v15 --local-dir coder390-v15
docker run --rm --gpus all --stop-timeout 150 -p 19421:19421 \
  -v "$PWD/coder390-v15:/model:ro" -v ninfer-coder390-v15-memory:/memory \
  lovewin10/ninfer-v15:5a5c2c1-cu13-v15 --model-id coder390-v15 \
  --vision --spec dflash2 --draft-tokens 7 --kv-dtype fp8 \
  --max-context 262144 --kv-capacity auto --max-concurrency 8

Send chat requests to /v1/chat/completions using model ID coder390-v15. The volume holds the FORMAT 6 SQLite store and segment files. To use MTP, pass --spec mtp --draft-tokens 3. Build instructions for the included source patch are in the patch README.

Measurements

One RTX PRO 6000 (96 GB), one request at a time, FP8 KV, three fixed prompts, 400 output tokens per prompt. Mean decode speed excludes time to first token. The measurements used native build 77d3033.

Backend Mean decode speed Mean first-token latency Draft acceptance
DFlash2, 7 draft tokens 106.6 tok/s 0.39 s 20.77%
MTP, 3 draft tokens 98.8 tok/s 0.45 s 45.97%

With the updated reader and this model's own post-sleep store, Coder390 answered 9/11 questions correctly with DFlash2, up from 8/11 with the previous reader. The recovered answer was a two-hop city lookup. Neither build produced a final answer on two items within the 800-token budget. This comparison used the same frozen store and questions in both runs. The option recall_bridge_context is enabled in this package. It restores an eligible partner from a confirmed, unambiguous link within the remaining context budget.

License and credits

The upstream publisher lists Apache-2.0 for the model. See the original model card for the training and model lineage. The NInfer patch includes its Apache-2.0 license.

Downloads last month
2
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for LTM-Lab/Qwen3.8-27B-Coder390-FP8-DFlash2-v15