seekr-org/span-02-bf16

Span-02 BF16

BF16 conversion of Respan's Span-02, a multimodal decision model based on Google's Gemma 4 26B-A4B. The original model and training are by Respan / Harvey Li; the base model is by Google. This repository packages a weight conversion by seekr-org, with no additional training.

The native NVFP4 experts and row-scaled FP8 weights are dequantized into standard BF16 safetensors for vLLM. Original BF16 vision encoder/projection, normalization weights, and router scales are preserved. This conversion cannot recover precision lost in the original quantization.

Property	Value
Source	respanai/span-02
Pinned source revision	63cbdd774391f1b7d9bc3829179efe996d0191ce
Architecture	Gemma4ForConditionalGeneration
Weight format	BF16, 32 standard safetensors shards
Weight tensor bytes	51,611,872,412 (about 51.6 GB / 48.1 GiB)
Inputs validated	Text/JSON and PNG/JPEG images
Validation hardware	One AMD MI300X

Serving

Use a vLLM build with Gemma 4 support. For /v1/systemone, the build must also include the SystemOne endpoint and the SpanStrategy integration used to validate this conversion. Downloading these weights does not install that server integration. The config's decision_config.prompt = "span" selects Span's native decision prompt and letter readout in that integration. Ordinary Gemma 4 chat serving alone does not implement the typed decision API.

vllm serve seekr-org/span-02-bf16 \
  --served-model-name span-02 \
  --dtype bfloat16 \
  --max-model-len 32768 \
  --enable-prefix-caching \
  --limit-mm-per-prompt '{"image":4,"audio":0,"video":0}' \
  --mm-processor-kwargs '{"max_soft_tokens":560}'

On the validated ROCm host, eager execution with TRITON_ATTN and TORCH_SDPA for multimodal encoder attention was used. Attention backend selection depends on the vLLM version and hardware. Serving needs memory beyond the weights for the vision encoder, activations, and KV cache.

With the Span SystemOne integration, make a text request:

curl http://localhost:8000/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "span-02",
    "state": {"ticket": "My package arrived damaged. I want a refund."},
    "questions": {
      "refund": {
        "type": "noul",
        "instructions": "Does the customer ask for a refund?"
      },
      "team": {
        "type": "choice",
        "instructions": "Which team should handle this ticket?",
        "criteria": {"returns": "Damaged items and refunds", "shipping": "Delivery status"}
      }
    }
  }'

For image requests, add "images": ["data:image/png;base64,<image bytes>"] or JPEG data URLs at the top level. Up to four images were validated.

Behavior and limitations

The validated integration uses one next-token read per question, with A/B/C option labels for choice, noul, and score. It supports up to 26 choices and 10 score levels, plus generated number answers. Images precede the state/question text in the prompt.

Automatic thinking is enabled by default for option questions. When the maximum initial option probability is below 0.8, the model generates a thought and reads the labels again. The final probabilities average the initial and post-thought distributions with equal weight.

Set the top-level reasoning object:

{"mode": "auto", "budget_ms": 700, "max_tokens": 256, "trace": true}

• mode: auto (default), on (attempt a thought for every question), or off (direct answer).
• budget_ms: thinking deadline from strategy processing, including rendering and initial reads; default 700 ms in auto mode and unlimited in on mode. 0 disables the clock. Final readout and number generation can extend total request time beyond this thinking budget.
• max_tokens: per-question thought cap, default 4096, range 1–4096; also limited by available context. At a cap or deadline, the thought is closed before readout. If less than 60 ms remain before thinking starts, the direct answer is returned.
• trace: include generated thought text under answers[id].reasoning.trace. usage.reasoning_tokens counts thought tokens within usage.output_tokens.

Use reasoning for these controls and keep chat_template_kwargs.enable_thinking off.

Number questions have no criteria, generate up to 32 answer tokens, and return number, text, and confidence. A nonnumeric/nonfinite answer returns number: null. Confidence is exp(mean generated-answer-token log probability). Number questions answer directly in auto mode and attempt a thought only in on mode, because they have no option-confidence gate.

{
  "model": "span-02",
  "state": {"a": 3, "b": 4},
  "questions": {"sum": {"type": "number", "instructions": "What is a + b?"}},
  "reasoning": {"mode": "on", "budget_ms": 0, "max_tokens": 256, "trace": true}
}

Label diagnostics apply only to option questions. For blended reads, label_mass averages the initial and final masses; argmax_is_label describes the final read.

• The native speculative decoder and GPU-specific deterministic thinking cost model are not implemented. This integration uses vLLM greedy decoding with explicit token/time limits.
• Confidence and usage follow the vLLM SystemOne definitions, which differ from the native Span server.
• Execution uses BF16 operations, not the native engine's FP8/NVFP4 activation quantization. Native-engine benchmark and accuracy parity have not been established.
• A 32,768-token context was configured for validation. Long-context capacity and serving throughput were not benchmarked.

Validation

Synthetic smoke tests on the full converted model produced:

Inputs	Correct decisions	Successful requests
Text	42/42	18/18
Images and text	33/33	11/11

Image cases cover color, count, relative position, and two-image ordering. Separate checks verified JPEG, four-image requests, malformed-image errors, number generation from text/images, and thinking gates/budgets. These 75 decisions are smoke coverage, not a reproduction of the Decision Index or the original model's reported benchmark results.

The integration includes strategy and question-type tests, checkpoint conversion tests, and real-server regression tests for direct readout, automatic/forced thought blending, number generation/confidence, and expired thinking budgets. Regression tests exercise real Gemma weights and the public endpoints. The full model also returned correct numeric answers from text and images and generated a bounded thought for an uncertain choice.

Reproducing the conversion

convert_span_checkpoint.py is included. Run it inside a Python environment with PyTorch, Hugging Face Hub, safetensors, and regex installed:

.venv/bin/python convert_span_checkpoint.py \
  respanai/span-02 ./span-02-bf16 \
  --revision 63cbdd774391f1b7d9bc3829179efe996d0191ce

CPU conversion is the default. --device cuda:0 uses a GPU; PyTorch on ROCm also uses the cuda:0 device name. The output directory must not already exist. The converter handles the source checkpoint's nonstandard metadata, unaligned tensor offsets, NVFP4 packing/scales, and FP8 row scales.

See conversion.json for provenance and SHA256SUMS for uploaded-file checksums.

License and attribution

The original model is distributed under the Gemma 4 / Apache 2.0 license. The license text and upstream attribution notices are preserved in LICENSE and NOTICE, with the conversion changes noted. The original Rust/CUDA serving engine and speculative draft model are not included in this repository.

Source model: respanai/span-02. Base model: google/gemma-4-26B-A4B-it.