roboalchemist/Seed-Coder-8B-Base-MLX-mixed-3-4

Seed-Coder-8B-Base MLX mixed 3/4-bit

An Apple Silicon conversion of ByteDance-Seed/Seed-Coder-8B-Base for fast, local fill-in-the-middle code completion with MLX-LM.

The original model and this conversion are MIT licensed. ByteDance Seed developed and trained the model; roboalchemist produced and benchmarked this quantized derivative. No fine-tuning or weight merging was performed.

Quantization

Converted from the official BF16 weights with MLX-LM 0.31.1:

mlx_lm.convert \
  --hf-path ByteDance-Seed/Seed-Coder-8B-Base \
  --mlx-path Seed-Coder-8B-Base-MLX-mixed-3-4 \
  --quantize \
  --q-group-size 64 \
  --quant-predicate mixed_3_4

The resulting model averages 3.699 bits per weight. The mixed recipe retains four bits for selected sensitive projections and uses three bits elsewhere. The weights occupy about 3.6 GB.

Fill-in-the-middle

Seed-Coder uses suffix-prefix-middle ordering:

<[fim-suffix]>CODE_AFTER_CURSOR<[fim-prefix]>CODE_BEFORE_CURSOR<[fim-middle]>

Example with MLX-LM:

from mlx_lm import generate, load

model, tokenizer = load("roboalchemist/Seed-Coder-8B-Base-MLX-mixed-3-4")
prefix = "function label(user) {\n  return user."
suffix = "\n}\n"
prompt = (
    f"<[fim-suffix]>{suffix}"
    f"<[fim-prefix]>{prefix}"
    "<[fim-middle]>"
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=64))

For VS Code, Reticle can serialize this format and connect to a loopback MLX-LM server.

Local benchmark

Measured on a 128 GB M3 Max MacBook Pro with deterministic sampling, one warmup, and the median of three 133-token TypeScript FIM requests:

Metric	Result
Decode throughput	93.17 tokens/s
Uncached first token	284 ms
Complete 33-token insertion	648 ms
Peak model/inference memory	4.12 GB

With MLX-LM prompt caching, incremental identifier completions reused 34–35 of 36–37 prompt tokens and completed in 134–146 ms end to end. Eight additional TypeScript, Python, Go, Rust, and JavaScript FIM probes produced valid insertions.

These are local measurements, not a general quality or speed guarantee.

Limitations

• Quantization can change model output and reduce quality relative to BF16.
• This checkpoint has no EAGLE, Medusa, or MTP draft head.
• It runs on the Apple GPU through MLX; it is not an Apple Neural Engine conversion.
• mlx_lm.server is intended for trusted loopback use and provides only basic security checks. Do not expose it directly to a network.
• Code suggestions can be incorrect or insecure. Review generated code before accepting it.

See the upstream model card for training details, evaluation results, and the original model's intended use.