WIlfLin/JEV-Qwen3.8-Flash-Next-Linear-Runtime

JEV Qwen3.8 Flash Next 180B — Linear · vLLM · Multimodal

Unofficial Jev-style closed-choice adaptation. Built from Qwen3.8 Flash Next, quantized by primitive-ai. This repository contains an 86 x 2560 original BF16 output head, runtime plugin and reproduction scripts. Backbone weights remain upstream and are fetched by python download_weights.py (about 184 GB). No fine-tuning, RL, temperature fitting or benchmark-driven prompt tuning was performed.

The plugin removes the full vocabulary projection and computes only 86 label logits. It scatters these into token-ID space to retain the existing vLLM one-token scoring API. There is no full-vocabulary GEMM. The default vLLM launch preserves the vision encoder and accepts up to eight images. Input: shared material, questions, and 2–86 options, each containing text, images, or both. Output: chosen index and conditional option probabilities; these are not calibrated correctness estimates. Context: 4096 tokens including prompt, options and one output token.

The choice API reuses one backend HTTP client for the application lifetime and closes it at shutdown. Native logits, the prompt and label mapping are unchanged. Repeated identical inputs can produce different probabilities or predictions with this inference stack; evaluate repeated runs rather than treating one result as definitive.

Default release: direct Linear, vLLM, multimodal

Run serve_backend.sh and serve_multimodal.py for the current release. The backend uses the original 86 x 2560 BF16 label head from head/, with no yes/no rows. The API scores all option labels in one inference request per question and normalizes those label logits. There is no candidate reranking. Material registration performs one separate cache-priming request.

The bundled multimodal API and Web UI support pictures in material, questions, and options, plus multiple questions per material ID. The launch enables image input, prefix caching, and cache usage reporting. The original text-only serve_choice.py remains available on port 8239.

shared_material/, shared_head/, and serve_shared_backend.sh are an older, explicitly opt-in Yes/No reranker experiment retained for reproducibility. They are not the default Linear multimodal release. Its measurements and older text benchmark results must not be presented as multimodal Linear results.

Intended use and limitations

This runtime is primarily intended for complex, high-difficulty closed-choice decisions, such as weighing conflicting evidence, applying multiple constraints, or distinguishing closely competing options. Performance on simple and standard tasks remains an area for improvement; this model should not be treated as a uniformly strong default across difficulty levels. This positioning reflects limited local public-set evaluation, not established superiority on every hard task or an official full-suite result. Validate performance on your own decision rubric and task distribution.

Run

Evaluated on RTX PRO 6000 Blackwell 96 GB, with PLE offloaded to CPU and more than 100 GiB available host RAM. Use the pinned vLLM container below; the plugin imports version-specific internals. From this repository directory:

python download_weights.py
bash serve_backend.sh

In a second terminal:

python -m pip install -r multimodal/requirements.txt
python serve_multimodal.py

Open http://127.0.0.1:8244/jev/. To validate real image requests:

python multimodal/validate.py --output multimodal-smoke.json

See API examples, limits, and validation. For the legacy text-only choice API, install transformers as well and run python serve_choice.py on port 8239 against the same backend.

Head provenance and projection checks are included. Historical raw runs remain in evaluations/ for traceability; no official benchmark score or ranking is claimed for this release. The backbone was verified against all 96 upstream safetensors SHA256 hashes before evaluation. Model/head weights use the included Qwen Community License 1.0. Runtime adaptation is provided with the same license. JevBench is by its original authors; this repository does not claim to be official TypeSafe Jev.