Post
83
Created a docker image for optimized serving of Qwen3.8-Flash-Next on a single DGX Spark. Find it here:
https://hub.docker.com/r/krisbailey/qwen3.8-flash-next-nvfp4-dgx-spark
The fastest I got it was 114 tok/s using 8 concurrent calls at the same time, and a little over 30 tok/s for single stream generation. It uses my "shortlist mtp" modification that only allows mtp to use 16,000 tokens instead of the full dictionary, but that speeds things up even though it causes slightly lower draft acceptance percentages.
https://hub.docker.com/r/krisbailey/qwen3.8-flash-next-nvfp4-dgx-spark
The fastest I got it was 114 tok/s using 8 concurrent calls at the same time, and a little over 30 tok/s for single stream generation. It uses my "shortlist mtp" modification that only allows mtp to use 16,000 tokens instead of the full dictionary, but that speeds things up even though it causes slightly lower draft acceptance percentages.