Akahsizrr/fuse-1-Lite-vLLM

Fuse3 vLLM Plugin

vLLM plugin for serving Fuse3 (fuse-1 Lite) models — LFM2 host + Qwen3.6 coding experts.

Installation

pip install -e .

Usage

Once installed, vLLM automatically discovers the plugin via Python entry points.

# Serve fuse-1 Lite with vLLM
vllm serve Akahsizrr/fuse-1-Lite \
  --mamba-cache-mode align \
  --max-model-len 4096

# Or with the Python API
from vllm import LLM
llm = LLM(
    model="Akahsizrr/fuse-1-Lite",
    mamba_cache_mode="align",
    max_model_len=4096,
)

How It Works

The plugin registers Fuse3ForCausalLM with vLLM's ModelRegistry. The model extends vLLM's native LFM2 implementation:

• Host layers: Reuses vLLM's Lfm2AttentionDecoderLayer and Lfm2ShortConvDecoderLayer
• Augmented layers: Wraps the host layer with a router + experts + scale
• Weight loading: Uses AutoWeightsLoader with a WeightsMapper that handles the host_layer. prefix and conv weight renaming

The plugin does NOT require --trust-remote-code — the model is registered natively.