Instructions to use LTM-Lab/Qwen3.5-0.8B-v15 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use LTM-Lab/Qwen3.5-0.8B-v15 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="LTM-Lab/Qwen3.5-0.8B-v15", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForImageTextToText processor = AutoProcessor.from_pretrained("LTM-Lab/Qwen3.5-0.8B-v15", trust_remote_code=True) model = AutoModelForImageTextToText.from_pretrained("LTM-Lab/Qwen3.5-0.8B-v15", trust_remote_code=True, device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use LTM-Lab/Qwen3.5-0.8B-v15 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "LTM-Lab/Qwen3.5-0.8B-v15" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LTM-Lab/Qwen3.5-0.8B-v15", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/LTM-Lab/Qwen3.5-0.8B-v15
- SGLang
How to use LTM-Lab/Qwen3.5-0.8B-v15 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "LTM-Lab/Qwen3.5-0.8B-v15" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LTM-Lab/Qwen3.5-0.8B-v15", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "LTM-Lab/Qwen3.5-0.8B-v15" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LTM-Lab/Qwen3.5-0.8B-v15", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use LTM-Lab/Qwen3.5-0.8B-v15 with Docker Model Runner:
docker model run hf.co/LTM-Lab/Qwen3.5-0.8B-v15
Qwen3.5-0.8B-v15
็น้ซไธญๆๆ่ฆ๏ผ้ๆฏ Qwen/Qwen3.5-0.8B ็ V15 ่ฝๆ็ใๆฌ้่ๅๆจกๅ้ไฝๅ
็ธๅ๏ผๅค็ๆฏ่จๆถ็่จญๅฎ่ๅ
ฉๅ็จๅผๆชใ็จ Docker ้กๅ lovewin10/sglang-v15:nightly-dev-cu13-20260924-ffac53d7-v15๏ผไฟฎๆน้็ SGLang๏ผๆๅๆ๏ผๆจกๅๅธถๆ่ทจ่ซๆฑไฟๅญ็ๆ
็ฏ่จๆถ๏ผๅจไธๆฎตๅฐ่ฉฑ่ฃก่ชช้็ไบ๏ผไนๅพๅจๅฆไธๆฎตๅ
จๆฐ็ๅฐ่ฉฑ่ฃกๅๅพๅฐใV15 ่ V14 ็ธๆฏๆฟๆไบไธ็ๆจกๅ๏ผ่จๆถไธๅๆฌๅไปปไฝๆฑ่ฅฟ๏ผๅฎๅ
จไธ้่ฆ่จ็ทด๏ผ๏ผๅคไบ็ก็ ๏ผไผบๆๅจ้็ฝฎๆ๏ผ็จ่ขซๆๅ็ๆจกๅ่ชๅทฑๆด็่จๆถๅบซ๏ผ้ๆญๆฐ็่จๆถใๅไฝต้่ค็ใ่ซๆจกๅๅคๆทๆฐ็ไบๅฏฆๆฏๅฆๅไปฃไบ่็ใๅจ่ฌๅไธๅไบบไบ็ฉ็็ญ่จไน้ๆญๆฉ๏ผๅคข่ๅคๆท้ฝไธๅฏซๅ
ฅ่จๆถ๏ผไนไธๅช้คไปปไฝ่จๆถใCUDA graphใๅ็ถดๅฟซๅ่ๆจๆธฌ่งฃ็ขผ็
งๅธธ้่ใๅฏฆๆธฌ็ตๆ่ๅทฒ็ฅ็้ๅถๅฏซๅจไธ้ขใ
Qwen/Qwen3.5-0.8B converted to the V15 episodic memory architecture (Qwen3_5V15ForConditionalGeneration).
The weights are the original's, byte for byte. Nothing was trained, merged or quantized here, and the memory trains nothing either.
Added: the architecture name, an
auto_mapand the memory options (qwen_dense_v15_memory_config) inconfig.json, and two code files (configuration_qwen_dense_v15.py,modeling_qwen_dense_v15.py) that hold the memory.Served by the Docker image
lovewin10/sglang-v15:nightly-dev-cu13-20260924-ffac53d7-v15, a modified SGLang, the model has a memory that persists across requests: what was said in one chat is answered in a later, separate chat. The store is on disk and survives restarts.Against V14 (LTM-Lab/Qwen3.5-0.8B-v14): the memory's world model is gone (small predictors fitted to the store, the only training V14 did; they did not measurably help), and while the server is idle the memory sleeps (below).
The small model of the family: the one the memory was developed and benchmarked on. Use it to try the mechanism; what it recalls it uses less reliably than the larger models do, and as the judge of its own sleep it is weak (see Measured).
How the memory works
While a prompt is read, the keys and values its attention layers compute are cut into events of up to 128 tokens (the message after a system message always begins one, so a system prompt that every conversation repeats is stored once and never recalled into another) and written to a store on disk, each with a low-rank engram of the state it leaves in the model's recurrent layers. A later prompt is read once to see what it asks for; the stored events whose keys would take its attention are recalled by one vote for all layers and heads; their engrams are composed into the request's recurrent state and the prompt is read again with the recalled keys and values merged into the attention. Nothing is retrieved as text and nothing is added to the prompt. A question that asks for nothing in the store recalls little or nothing, but a stored event that merely resembles it can pass the vote.
Sleep. Once the server has had no user request for a while (10 minutes by default), the memory sleeps, using the served model through the server's own API:
- Replay. The events stored since the last sleep are read again as prompts with the memory read-only, those written by a prompt that recalled something first, then the newest first; what each one calls up is noted. The model is also asked, in two wordings, whether the note only asks something: a question stored without its answer answers nothing, and its look-alikes crowd the notes a later question needs. Two yeses take it out of recall (it stays on disk).
- Merging. An earlier event a replay calls up that is the same text told again is merged into the later one.
- Correction. For each earlier event a replay calls up, the most strongly called first, the model is asked in two wordings whether the later note changes what the earlier one says. Two yeses retire the earlier event: it is no longer recalled on its own, only together with the later one, right before it, because a correction often names only what changed ("the meeting moved to Thursday") and the earlier note says whose meeting it is. Any other answer changes nothing.
- Dreams. For some replayed events the model writes a question the note would answer; the question is read with the memory read-only, and the events of other conversations it calls up together with that note are proposed for a bridge.
- Bridges. The pairs the dreams proposed, the pairs a replay called up together and the pairs that share a rare word are put to the model: which particular person, thing or place do both notes name (not the speaker)? Where the name it gives is written in both notes, they get a bridge; a name it makes up, or none, bridges nothing. Once one of them is recalled, the other needs only 35% of the best vote (
linked_ratio) instead of half. - Downscaling. Every bridge that no prompt and no dream recalled both ends of since the last sleep is scaled down by 0.9: one unused for seven sleeps is no longer followed, and a lighter one is removed.
Nothing is trained. The dreams and the model's answers are never written to the store, and no event is deleted: a retired or merged one stays on disk (in the 4-bit format). A user's request stops the cycle at its next step, and the next idle stretch takes up what is left.
Serve with SGLang (Docker)
Needs any GPU; the Transformers class also runs on CPU, Docker with the NVIDIA container runtime, and a writable directory for the memory store.
hf download LTM-Lab/Qwen3.5-0.8B-v15 --local-dir <models>/Qwen3.5-0.8B-v15
docker run --rm -it --gpus all --ipc=host -p 8000:8000 `
-v "<models>:/model:ro" `
-v "<memory store>:/memory" `
lovewin10/sglang-v15:nightly-dev-cu13-20260924-ffac53d7-v15 `
sglang serve --model-path /model/Qwen3.5-0.8B-v15 --trust-remote-code `
--host 0.0.0.0 --port 8000 --mem-fraction-static 0.78 --served-model-name qwen-0.8b-v15 `
--chat-template /model/Qwen3.5-0.8B-v15/chat_template.jinja `
--reasoning-parser qwen3-thinking --tool-call-parser qwen3_coder `
--attention-backend flashinfer --linear-attn-backend triton --mamba-ssm-dtype float32 `
--speculative-algorithm EAGLE --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 `
--enable-linear-replayssm-spec --mamba-radix-cache-strategy extra_buffer_lazy `
--context-length 262144 --max-prefill-tokens 32768 --max-running-requests 8 `
--kv-cache-dtype fp8_e4m3 --max-mamba-cache-size 16
(PowerShell line continuations; in a POSIX shell end the lines with \.) The server speaks the OpenAI API on port 8000. The image is lmsysorg/sglang:nightly-dev-cu13-20260924-ffac53d7 with its SGLang replaced; a launch command written for that image runs unchanged apart from the /memory mount.
/memoryholds what the prompts contained: the token ids and the attention keys and values of every prompt the server has read. Treat it like chat logs; delete the directory to forget everything.- Every request reads and writes the same store. There is no separation between users: run one server with its own
/memoryfor each user or tenant whose memories must stay apart. - Generated text is not written. A reply becomes memory when a later prompt carries it, as chat front-ends do when they resend the conversation.
- A V15 directory opens a store of its own: a store written by a V14 image is not read.
-e SGLANG_V15_MEMORY=0serves the directory without memory, as the original model.-e SGLANG_V15_SLEEP_AFTER=<seconds>(default 600) is how long the server waits without a user request before it sleeps;-e SGLANG_V15_SLEEP=0turns the sleep off (the store is then only written and recalled, as V14's was without its world model). The sleep's own requests go through the server with ids beginningV15_DREAM_; they write nothing to the store and share no prefix cache with user requests, and every cycle logs a lineV15 sleep: cycle โฆwith what it did.-e SGLANG_V15_TRACE=1logs every recall decision (the votes, the events recalled).-e SGLANG_V15_INDEX_GB(default 2) is the GPU memory for the index's rows. With more stored payloads than fit, a vote reads the rest back from the store (seconds a vote from a hard disk) and the log says so once: raise it, or keep the store on an SSD.-e SGLANG_V15_WRITE_QUEUE_GB(default 8) bounds the RAM of events waiting to be written; a request beyond it stores nothing more, and the log says so.- The index keeps a little per event in memory and the rest on disk, so the server's memory stays level as the store grows; every 10 minutes the log says what it holds.
- The memory needs
--attention-backend flashinfer(orfa3), and TP, PP and DP attention of 1.
Serve with ninfer
ninfer is a single-GPU C++/CUDA inference server for Qwen3.5-architecture models. It offers OpenAI-compatible Chat Completions and Responses, Anthropic Messages, /generate scoring, MTP and DFlash2 speculative decoding, CUDA graphs, prefix reuse, quantized KV caches, and image and video input. Its native V15 build runs the episodic store and background writer inside ninfer-serve. CUDA handles voting, event encoding and the recalled attention and recurrent state. Sleep submits requests to the same engine.
What to use. ninfer loads its own .ninfer packs. Convert the BF16 weights of LTM-Lab/Qwen3.5-0.8B-v15 to an FP8 pack once; an -FP8 repository uses Qwen's block-FP8 layout, which the ninfer converter does not import. The native engine reads the V15 options and turn markers from the companion config.json and tokenizer.json. For the 27B, a converted pack with vision, MTP and a DFlash2 draft, with those companion files, is ready to download: LTM-Lab/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v15-ninfer-FP8; its card has what was measured with it.
Download the 27B FP8 pack and start the native server in Docker. The volume keeps the episodic store when the container exits.
hf download LTM-Lab/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v15-ninfer-FP8 --local-dir v15-ninfer
docker run --rm --gpus all --stop-timeout 150 -p 19421:19421 \
-v "$PWD/v15-ninfer:/model:ro" -v ninfer-v15-memory:/memory \
lovewin10/ninfer-v15:5a5c2c1-cu13-v15 --model-id qwen3.8-27b-v15
The image defaults to FP8 KV, MTP with three draft tokens, vision, a 262144-token context, automatic KV capacity and eight lanes. Pass server flags after the image name to override those defaults.
Build from source
Source builds use CUDA 13.0, CMake 3.28 or newer, Ninja, and development packages for FFmpeg (libavcodec, libavformat, libavutil, libswscale), libcurl and SQLite 3. The weight converter uses Python, PyTorch and Transformers. Run the server on a Blackwell GPU (sm_120a).
Build. The V15 support is a patch in this repository's ninfer-v15-patch/ folder, applied to upstream ninfer at commit 5a5c2c1:
hf download LTM-Lab/Qwen3.5-0.8B-v15 \
--include "ninfer-v15-patch/*" --local-dir .
git clone https://github.com/MerkyorLynn/ninfer-rollout ninfer && cd ninfer
git checkout 5a5c2c1
git apply ../ninfer-v15-patch/ninfer_v15.patch
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_CUDA_ARCHITECTURES=120a -DNINFER_BUILD_APPS=ON
cmake --build build --target ninfer-serve
Convert (once, from the ninfer checkout; add --device cpu when the GPU is busy):
hf download LTM-Lab/Qwen3.5-0.8B-v15 --local-dir v15-bf16
python3 -m tools.convert --model v15-bf16 \
--recipe tools/convert/recipes/qwen_v15_fp8.py \
--components text,vision,mtp --proposal \
--name v15-fp8 --out v15_fp8.ninfer
Serve the converted pack with its companion files and a persistent store.
./build/apps/ninfer-serve v15_fp8.ninfer --v15-memory ./v15-memory --v15-model-dir v15-bf16 \
--host 0.0.0.0 --port 19421 --model-id v15 \
--vision --spec mtp --draft-tokens 3 --kv-dtype fp8 \
--max-context 262144 --kv-capacity auto --max-concurrency 8
Server options
--v15-memory DIRselects the memory home and--v15-model-dir DIRsupplies the companion files. The pack's artifact ID determines the default store namespace. Use--v15-namespace,--v15-key-spaceor--v15-storage-pathfor explicit store selection.--kv-capacity autosizes the engine's KV cache with V15's planned GPU allocations included. Set the memory index and event KV budgets with--v15-index-giband--v15-kv-cache-gib.--v15-raw-cache-gibsets the host cache of stored event bytes.--v15-sleep-after SECONDSsets the idle time before consolidation (default 600).--v15-no-sleepdisables consolidation.--v15-tracelogs votes.- The engine keeps its KV dtypes (
bf16,int8,fp8,nvfp4,k8v4), MTP and DFlash2, CUDA graphs, prefix reuse, pause/resume, vision and rollout scoring. Omit both V15 directory flags for memory-free serving, or setMEMORY=0when using the image. V15_AUX_HOST_GIBsets the host memory for the memory's own extra saves. The default is four state images per lane. These saves never take space from the cache's own.- Per request:
"memory": falsein the request body serves that request without memory. TheX-Request-Idheader (orridin the body) names the request in the logs. - Votes while generating follow the SGLang V15 integration: the first comes after 32 generated tokens, then the interval doubles each time (64, 128, 256), up to 512.
- Send SIGTERM or Ctrl+C to
ninfer-servefor an orderly shutdown. The server drains the memory writer during shutdown. The Docker invocation allows 150 seconds for this.
Native V15 recall
The native server can use one additional note as attention context when sleep has confirmed a link to a directly recalled note. It uses the remaining memory-token budget. Equal-strength links to different eligible notes are left unresolved. The 27B packages enable this with recall_bridge_context in the model's memory config.
On a frozen 27B store, the new reader followed a confirmed link from an employer to its city. It answered 12 of 12 correction and two-hop questions, up from 11 with the previous reader. On a separate set of ten questions, the scores were 8 and 7. Both comparisons used the final visible answer, one request at a time. The Coder390 pack and its matched DFlash2 draft are available separately.
Results with the earlier Python memory service
Measured with the 27B conversion (LTM-Lab/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v15) on one RTX PRO 6000, with the FP8 pack, FP8 KV, MTP with 3 draft tokens using the Python-service build at dc51f5e (262144-token context, 8 lanes; auto gave 1,648,128 KV tokens): facts recalled 6/6, unrelated questions answered 4/4, eight chats telling and asking at once 8/8, a fact inside a 9,442-token prompt found, an image named, and the server alive after an aborted request. One 320-token reply took 4.8 to 8.5 s, first token included (38 to 66 tok/s); eight 200-token replies at once came to 98 tok/s in all. The same server without memory, same flags, replies at 104 tok/s and serves eight at once at 492 tok/s in all. With memory, while the service votes for one request the engine waits for its answer, and every lane waits with it; that wait is what eight requests at once lose.
Use with Transformers
The two code files are remote code for Transformers; the SGLang image runs the same code for every memory decision, its sleep included.
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
path = 'LTM-Lab/Qwen3.5-0.8B-v15'
model = AutoModelForImageTextToText.from_pretrained(path, trust_remote_code=True, dtype=torch.bfloat16).to('cuda').eval()
processor = AutoProcessor.from_pretrained(path)
def ask(text):
messages = [{'role': 'user', 'content': [{'type': 'text', 'text': text}]}]
prompt = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=False, enable_thinking=False)
ids = processor.tokenizer(prompt, return_tensors='pt', add_special_tokens=False).input_ids.to(model.device)
out = model.generate(input_ids=ids, max_new_tokens=40, do_sample=False)
return processor.tokenizer.decode(out[0, ids.shape[1]:], skip_special_tokens=True)
ask('Please remember: my locker code is 4817-2953.')
ask('My locker code changed: it is 7310-0584 now.')
model.sleep() # one sleep cycle, the model judging its own store
ask('What is my locker code?') # a separate call: recalled from the store
Every forward reads and writes the store on disk (under the user's state directory, named after the weights; the memory option storage_path sets another place). model.sleep() runs one sleep cycle with the model itself as the judge and returns its report (events replayed, merged and retired, dreams, bridges). model(...).memory_trace lists what each row wrote and recalled, model.open_memory_bank() returns the store (memories(), forget(name)), and model.memory_enabled = False gives the original model. The benchmark below was run this way.
Measured
The memory benchmark of the project (examples/memory_benchmark.py) on these weights with the Transformers class, 2026-11-16: bf16, greedy decoding, 40 new tokens per answer, one RTX PRO 6000 Blackwell. Each section writes earlier sessions into an empty store and then asks in a new session; with a sleep, model.sleep() runs once between the sessions and the questions, the model judging its own store. A hit is a reply that contains the reference answer. Three to ten questions per section, so a difference of one or two means nothing; a rerun of the same code moves a section by one or two.
| Section | Memory, no sleep | Memory, one sleep | Everything in one context, no memory (upper bound) | Question only |
|---|---|---|---|---|
| single: ten facts, a store each | 9 / 10 | 9 / 10 | 8 / 10 | 0 |
| noise: ten facts among 48 other sessions | 7 / 10 (the event with the answer recalled for 10) | 7 / 10 (recalled for 10) | 8 / 10 | 0 |
| crowd: ten facts among 248 other sessions | 6 / 10 (recalled for 8) | 6 / 10 (recalled for 8) | ||
| update: a fact corrected in a later session, asked for its present value | 1 / 6 (1 more answered the old value) | 2 / 6 (1 more answered the old value) | 6 / 6 | |
| hop: two facts from two sessions needed for one answer | 5 / 6 | 6 / 6 | 6 / 6 | |
| long: a fact in the middle of a long document | 3 / 3 | 3 / 3 | 3 / 3 | |
| unrelated questions (the noise store open) | 7 / 8, no event recalled | 7 / 8 | 7 / 8 without memory |
As the judge of its own sleep this model is weak: in the noise section, which holds no correction at all, it retired 13 notes as corrected by others. The answers did not suffer (a retired note is still recalled, before the note taken for its correction), but the larger models judge far better: on the 27B FP8 conversion every retirement, question and bridge of a measured sleep was right (its card).
Served by the SGLang image the directory behaves as the larger models' directories do; the image's own checks were run on the 27B FP8 conversion (its card).
Limits
- It recalls what a question asks for in similar words. The vote compares the question's attention queries with stored keys. A question that shares little with the note that answers it can miss it; the larger the model, the less this happens.
- Two-step questions are the weak case. "In which rack is the backup server?" needs the note that names the server and then the note about that server. The second is recalled if its vote reaches half of the first one's (
recall_ratio), or 35% of it (linked_ratio) once a sleep has bridged the two, and a bridge exists only where the model, asked during a sleep which person, thing or place both notes name, gave a name written in both. Two notes that refer to one person in different words ("my sister" and "Clara") are not bridged. - A corrected fact is retired only by a sleep. Until the server has slept, the old and the new note are both in the store, and when both are recalled the model usually, not always, answers the newer one. A sleep retires the old one only when the model says twice that the later note changes it; after that the old note is recalled only together with the new one, before it, so a reply can still mix them up. Recalled events carry no time stamp.
- The sleep uses the GPU while the server is idle. A cycle replays up to 64 events (and asks, of each one with a question mark, whether it only asks), judges up to 128 pairs (two questions each), dreams up to 16 questions and asks up to 64 pairs which name they share, all as requests to the server itself. A request that arrives during a sleep is served beside the sleep's request in flight and stops the cycle at its next step.
- The first token waits for the recall. A prompt that recalls something is read two or three times. Decoding is somewhat slower while recalled events are attended.
- A request waits until what the server read before it is stored, so that it can recall it: unnoticeable in chat, up to some ten seconds right after a document of several thousand tokens.
- No separation between users, and the store keeps what prompts contained (see above).
- Disk. About 6 KB per prompt token, plus a state engram per event. Events unused for seven days are rewritten in a 4-bit format. The memory deletes no event, retired and merged ones included: delete the store to forget.
- Recalling is not answering: among 48 other sessions the note with the answer was recalled for ten of ten questions and seven were answered; among 248 it was recalled for eight and six were answered.
- A corrected fact is the unsteady case on this model: one or two of six, with or without a sleep. Its sleep judges corrections badly (see Measured), so it retires notes that were not corrected and leaves real corrections in place.
- The recall thresholds (
recall_ratio,recall_floor,linked_ratioin the memory options) were set on Qwen3.5-0.8B and not re-tuned for this model. - The trained weights are untouched, so everything the original card says about the model itself still holds, including its own limitations.
Credits and license
- The model is Qwen/Qwen3.5-0.8B; its card follows below, unchanged, and its license applies to this repository (see the metadata above and the original card).
- The serving engine is SGLang (Apache-2.0); the image replaces the SGLang of
lmsysorg/sglang:nightly-dev-cu13-20260924-ffac53d7and is not an official SGLang image. - Conversion, memory code and image: LTM-Lab / lovewin10.
Original model card (Qwen/Qwen3.5-0.8B)
Qwen3.5-0.8B
This repository contains model weights and configuration files for the post-trained model in the Hugging Face Transformers format.
These artifacts are compatible with Hugging Face Transformers, vLLM, SGLang, KTransformers, etc.
In light of its parameter scale, the intended use cases are prototyping, task-specific fine-tuning, and other research or development purposes.
Over recent months, we have intensified our focus on developing foundation models that deliver exceptional utility and performance. Qwen3.5 represents a significant leap forward, integrating breakthroughs in multimodal learning, architectural efficiency, reinforcement learning scale, and global accessibility to empower developers and enterprises with unprecedented capability and efficiency.
Qwen3.5 Highlights
Qwen3.5 features the following enhancement:
Unified Vision-Language Foundation: Early fusion training on multimodal tokens achieves cross-generational parity with Qwen3 and outperforms Qwen3-VL models across reasoning, coding, agents, and visual understanding benchmarks.
Efficient Hybrid Architecture: Gated Delta Networks combined with sparse Mixture-of-Experts deliver high-throughput inference with minimal latency and cost overhead.
Scalable RL Generalization: Reinforcement learning scaled across million-agent environments with progressively complex task distributions for robust real-world adaptability.
Global Linguistic Coverage: Expanded support to 201 languages and dialects, enabling inclusive, worldwide deployment with nuanced cultural and regional understanding.
Next-Generation Training Infrastructure: Near-100% multimodal training efficiency compared to text-only training and asynchronous RL frameworks supporting massive-scale agent scaffolds and environment orchestration.
For more details, please refer to our blog post Qwen3.5.
Model Overview
- Type: Causal Language Model with Vision Encoder
- Training Stage: Pre-training & Post-training
- Language Model
- Number of Parameters: 0.8B
- Hidden Dimension: 1024
- Token Embedding: 248320 (Padded)
- Number of Layers: 24
- Hidden Layout: 6 ร (3 ร (Gated DeltaNet โ FFN) โ 1 ร (Gated Attention โ FFN))
- Gated DeltaNet:
- Number of Linear Attention Heads: 16 for V and 16 for QK
- Head Dimension: 128
- Gated Attention:
- Number of Attention Heads: 8 for Q and 2 for KV
- Head Dimension: 256
- Rotary Position Embedding Dimension: 64
- Feed Forward Network:
- Intermediate Dimension: 3584
- LM Output: 248320 (Tied to token embedding)
- MTP: trained with multi-steps
- Context Length: 262,144 natively
Benchmark Results
Language
| Qwen3-4B-2507 | Qwen3-1.7B | Qwen3.5-2B | Qwen3.5-0.8B | |
|---|---|---|---|---|
| Non-Thinking Mode | ||||
| MMLU-Pro | 69.6 | 40.2 | 55.3 | 29.7 |
| MMLU-Redux | 84.2 | 64.4 | 69.2 | 48.5 |
| C-Eval | 80.2 | 61.0 | 65.2 | 46.4 |
| SuperGPQA | 42.8 | 21.0 | 30.4 | 16.9 |
| IFEval | 83.4 | 68.2 | 61.2 | 52.1 |
| MMMLU | 64.9 | 46.7 | 56.9 | 34.1 |
| Knowledge & STEM (Thinking) | ||||
| MMLU-Pro | 74.0 | 56.5 | 66.5 | 42.3 |
| MMLU-Redux | 86.1 | 73.9 | 79.6 | 59.5 |
| C-Eval | 82.2 | 68.1 | 73.2 | 50.5 |
| SuperGPQA | 47.8 | 31.2 | 37.5 | 21.3 |
| GPQA | 65.8 | 40.1 | 51.6 | 11.9 |
| Instruction Following (Thinking๏ผ | ||||
| IFEval | 87.4 | 72.5 | 78.6 | 44.0 |
| IFBench | 50.4 | 26.7 | 41.3 | 21.0 |
| MultiChallenge | 41.7 | 27.2 | 33.7 | 18.9 |
| Long Context (Thinking๏ผ | ||||
| AA-LCR | 32.0 | 6.7 | 25.6 | 4.7 |
| LongBench v2 | 42.8 | 26.5 | 38.7 | 26.1 |
| Reasoning (Thinking๏ผ | ||||
| HMMT Feb 25 | 57.5 | 10.2 | 22.9 | -- |
| HMMT Nov 25 | 69.6 | 8.9 | 19.6 | -- |
| General Agent (Thinking๏ผ | ||||
| BFCL-V4 | 39.9 | -- | 43.6 | 25.3 |
| TAU2-Bench | 43.2 | -- | 48.8 | 11.6 |
| Multilingualism (Thinking๏ผ | ||||
| MMMLU | 70.8 | 57.0 | 63.1 | 44.3 |
| MMLU-ProX | 62.4 | 49.4 | 52.3 | 34.6 |
| NOVA-63 | 47.1 | 40.3 | 46.4 | 42.4 |
| INCLUDE | 64.4 | 51.8 | 55.4 | 40.6 |
| Global PIQA | 73.5 | 63.1 | 69.3 | 59.4 |
| PolyMATH | 46.2 | 25.2 | 26.1 | 8.2 |
| WMT24++ | 58.9 | 39.3 | 45.8 | 27.2 |
| MAXIFE | 72.1 | 50.7 | 60.6 | 39.2 |
* TAU2-Bench: we follow the official setup except for the airline domain, where all models are evaluated by applying the fixes proposed in the Claude Opus 4.5 system card.
* MMLU-ProX: we report the averaged accuracy on 29 languages.
* WMT24++: a harder subset of WMT24 after difficulty labeling and rebalancing; we report the averaged scores on 55 languages using XCOMET-XXL.
* MAXIFE: we report the accuracy on English + multilingual original prompts (totally 23 settings).
* Experimental settings: top_p=0.95, top_k=20, presence_penalty=1.5, and temperature=1.0 were used.
* Empty cells (--) indicate scores not yet available or not applicable.
Vision Language
| Qwen3-VL-4B | Qwen3-VL-2B | Qwen3.5-2B | Qwen3.5-0.8B | |
|---|---|---|---|---|
| STEM and Puzzle | ||||
| MMMU | 70.8 | 61.4 | 64.2/64.2 | 49/47.4 |
| MMMU-Pro | 57.0 | 42.5 | 50.3/47.7 | 31.2/31.4 |
| Mathvista(mini) | 79.5 | 73.6 | 76.7/73.9 | 62.2/58.6 |
| DynaMath | 74.4 | 66.7 | 73.6/69.6 | 49.9/46.5 |
| ZEROBench | 0.0 | 0.0 | 1.0/0.0 | 0.0/0.0 |
| ZEROBench_sub | 18.9 | 13.2 | 17.1/18.6 | 12.9/11.4 |
| VlmsAreBlind | 68.6 | 50.0 | 75.8/74.3 | 59.4/57.3 |
| General VQA | ||||
| RealWorldQA | 73.2 | 69.5 | 74.5/71.2 | 63.4/61.6 |
| MMStar | 73.2 | 68.1 | 71.7/68.0 | 58.3/55.9 |
| MMBenchEN-DEV-v1.1 | 86.7 | 81.9 | 83.3/81.3 | 69.9/68.0 |
| SimpleVQA | 48.8 | 43.6 | 38.5/39.5 | 31.3/30.4 |
| HallusionBench | 64.1 | 54.9 | 58.0/51.3 | 53.1/46.7 |
| Text Recognition and Document Understanding | ||||
| MMLongBench-Doc | 44.4 | 33.8 | 45.4/38.8 | 33.6/28.1 |
| AI2D_TEST | 84.9 | 80.4 | 83.3/81.5 | 69.9/68.7 |
| CC-OCR | 73.8 | 68.3 | 72.9/75.8 | 63.2/66.7 |
| OmniDocBench1.5 | 80.0 | 65.9 | 79.8/80.9 | 61.0/70.6 |
| CharXiv(RQ) | 50.3 | 37.1 | 58.8/52.6 | 41.3/38.2 |
| OCRBench | 80.8 | 79.2 | 84.5/85.4 | 74.5/79.1 |
| Spatial Intelligence | ||||
| RefCOCO(avg) | 88.2 | 84.8 | 84.8/84.3 | 79.3/77.8 |
| CountBench | 89.4 | 84.1 | 91.4/86.8 | 77.0/68.6 |
| ODInW13 | 39.4 | 36.0 | 35.9/40.5 | 31.6/33.2 |
| ERQA | 47.3 | 41.8 | 43.8/33.0 | 34.5/23.8 |
| EmbSpatialBench | 80.7 | 75.9 | 77.9/66.4 | 68.6/54.6 |
| RefSpatialBench | 45.3 | 28.9 | 32.9/30.0 | 23.5/21.7 |
| Hypersim | 11.9 | 11.2 | 12.4/12.4 | 11.9/11.0 |
| SUNRGBD | 28.0 | 28.6 | 28.7/25.6 | 26.1/23.3 |
| Nuscene | 4.9 | 4.0 | 6.9/8.5 | 5.7/7.0 |
| Video Understanding | ||||
| VideoMME(w sub.) | 76.0 | 67.9 | 75.6/-- | 63.8/-- |
| VideoMME(w/o sub.) | 68.9 | 62.1 | 69.0/-- | 57.7/-- |
| VideoMMMU | 69.4 | 54.1 | 62.1/-- | 44.3/-- |
| MLVU | 75.7 | 69.2 | 76.2/-- | 65.6/-- |
| MVBench | 69.3 | 64.5 | 64.9/-- | 55.8/-- |
| LVBench | 53.5 | 47.6 | 57.1/-- | 45.1/-- |
| MMVU | 58.6 | 48.9 | 48.6/-- | 34.3/-- |
| Visual Agent | ||||
| ScreenSpot Pro | 59.5 | 48.5 | --/54.5 | --/46.5 |
| Medical VQA | ||||
| SLAKE | 65.9 | 61.1 | 74.4/67.5 | 62.6/59.5 |
| PMC-VQA | 48.4 | 42.4 | 48.8/54.0 | 40.4/45.5 |
| MedXpertQA-MM | 26.3 | 13.0 | 26.9/19.1 | 17.1/25.3 |
* Scores of Qwen3.5 models are reported as Thinking / Non-thinking.
* MathVision: our modelโs score is evaluated using a fixed prompt, e.g., โPlease reason step by step, and put your final answer within \boxed{}.โ For other models, we report the higher score between runs with and without the \boxed{} formatting.
* Experimental settings: For the Video benchmarks, we used top_p=0.95, top_k=20, presence_penalty=1.5, and temperature=1.0. All other benchmarks adopted the same sampling configuration but with temperature=0.6 under the thinking mode. Under the non-thinking mode, the sampling parameters were set to top_p=0.8, top_k=20, presence_penalty=1.5, and temperature=0.7.
* Empty cells (--) indicate scores not yet available or not applicable.
Quickstart
Qwen3.5 models support both non-thinking and thinking mode. Qwen3.5-0.8B operates in non-thinking mode by default. To enable thinking, refer to the examples here.
For streamlined integration, we recommend using Qwen3.5 via APIs. Below is a guide to use Qwen3.5 via OpenAI-compatible API.
Serving Qwen3.5
Qwen3.5 can be served via APIs with popular inference frameworks. In the following, we show example commands to launch OpenAI-Compatible API servers for Qwen3.5 models.
Inference efficiency and throughput vary significantly across frameworks. We recommend using the latest framework versions to ensure optimal performance and compatibility. For production workloads or high-throughput scenarios, dedicated serving engines such as SGLang, KTransformers or vLLM are strongly recommended.
The model has a default context length of 262,144 tokens. If you encounter out-of-memory (OOM) errors, consider reducing the context window.
SGLang
SGLang is a fast serving framework for large language models and vision language models. SGLang from the main branch of the open-source repository is required for Qwen3.5, which can be installed using the following command in a fresh environment:
uv pip install 'git+https://github.com/sgl-project/sglang.git#subdirectory=python&egg=sglang[all]'
See its documentation for more details.
The following will create API endpoints at http://localhost:8000/v1:
Standard Version: The following command can be used to create an API endpoint with maximum context length 262,144 tokens using tensor parallel on 8 GPUs.
python -m sglang.launch_server --model-path Qwen/Qwen3.5-0.8B --port 8000 --tp-size 1 --mem-fraction-static 0.8 --context-length 262144Tool Use: To support tool use, you can use the following command.
python -m sglang.launch_server --model-path Qwen/Qwen3.5-0.8B --port 8000 --tp-size 1 --mem-fraction-static 0.8 --context-length 262144 --tool-call-parser qwen3_coderMulti-Token Prediction (MTP): The following command is recommended for MTP:
python -m sglang.launch_server --model-path Qwen/Qwen3.5-0.8B --port 8000 --tp-size 1 --mem-fraction-static 0.8 --context-length 262144 --speculative-algo NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4
vLLM
vLLM is a high-throughput and memory-efficient inference and serving engine for LLMs. vLLM from the main branch of the open-source repository is required for Qwen3.5, which can be installed using the following command in a fresh environment:
uv pip install vllm --torch-backend=auto --extra-index-url https://wheels.vllm.ai/nightly
See its documentation for more details.
For detailed Qwen3.5 usage guide, see the vLLM Qwen3.5 recipe.
The following will create API endpoints at http://localhost:8000/v1:
Standard Version: The following command can be used to create an API endpoint with maximum context length 262,144 tokens using tensor parallel on 8 GPUs.
vllm serve Qwen/Qwen3.5-0.8B --port 8000 --tensor-parallel-size 1 --max-model-len 262144Tool Call: To support tool use, you can use the following command.
vllm serve Qwen/Qwen3.5-0.8B --port 8000 --tensor-parallel-size 1 --max-model-len 262144 --enable-auto-tool-choice --tool-call-parser qwen3_coderMulti-Token Prediction (MTP): The following command is recommended for MTP:
vllm serve Qwen/Qwen3.5-0.8B --port 8000 --tensor-parallel-size 1 --max-model-len 262144 --speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}'Text-Only: The following command skips the vision encoder and multimodal profiling to free up memory for additional KV cache:
vllm serve Qwen/Qwen3.5-0.8B --port 8000 --tensor-parallel-size 1 --max-model-len 262144 --language-model-only
KTransformers
KTransformers is a flexible framework for experiencing cutting-edge LLM inference optimizations with CPU-GPU heterogeneous computing. For running Qwen3.5 with KTransformers, see the KTransformers Deployment Guide.
Hugging Face Transformers
Hugging Face Transformers contains a lightweight server which can be used for quick testing and moderate load deployment.
The latest transformers is required for Qwen3.5:
pip install "transformers[serving] @ git+https://github.com/huggingface/transformers.git@main"
See its documentation for more details. Please also make sure torchvision and pillow are installed.
Then, run transformers serve to launch a server with API endpoints at http://localhost:8000/v1; it will place the model on accelerators if available:
transformers serve --force-model Qwen/Qwen3.5-0.8B --port 8000 --continuous-batching
Using Qwen3.5 via the Chat Completions API
The chat completions API is accessible via standard HTTP requests or OpenAI SDKs. Here, we show examples using the OpenAI Python SDK.
Before starting, make sure it is installed and the API key and the API base URL is configured, e.g.:
pip install -U openai
# Set the following accordingly
export OPENAI_BASE_URL="http://localhost:8000/v1"
export OPENAI_API_KEY="EMPTY"
We recommend using the following set of sampling parameters for generation
- Non-thinking mode for text tasks:
temperature=1.0, top_p=1.00, top_k=20, min_p=0.0, presence_penalty=2.0, repetition_penalty=1.0- Non-thinking mode for VL tasks:
temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0- Thinking mode for text tasks:
temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0- Thinking mode for VL or precise coding (e.g. WebDev) tasks :
temperature=0.6, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0Please note that the support for sampling parameters varies according to inference frameworks.
Text-Only Input
from openai import OpenAI
# Configured by environment variables
client = OpenAI()
messages = [
{"role": "user", "content": "Give me a short introduction to large language models."},
]
chat_response = client.chat.completions.create(
model="Qwen/Qwen3.5-0.8B",
messages=messages,
max_tokens=32768,
temperature=1.0,
top_p=1.0,
presence_penalty=2.0,
extra_body={
"top_k": 20,
},
)
print("Chat response:", chat_response)
Image Input
from openai import OpenAI
# Configured by environment variables
client = OpenAI()
messages = [
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": {
"url": "https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5/demo/RealWorld/RealWorld-04.png"
}
},
{
"type": "text",
"text": "Where is this?"
}
]
}
]
chat_response = client.chat.completions.create(
model="Qwen/Qwen3.5-0.8B",
messages=messages,
max_tokens=32768,
temperature=0.7,
top_p=0.8,
presence_penalty=1.5,
extra_body={
"top_k": 20,
},
)
print("Chat response:", chat_response)
Video Input
from openai import OpenAI
# Configured by environment variables
client = OpenAI()
messages = [
{
"role": "user",
"content": [
{
"type": "video_url",
"video_url": {
"url": "https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5/demo/video/N1cdUjctpG8.mp4"
}
},
{
"type": "text",
"text": "Summarize the video content."
}
]
}
]
# When vLLM is launched with `--media-io-kwargs '{"video": {"num_frames": -1}}'`,
# video frame sampling can be configured via `extra_body` (e.g., by setting `fps`).
# This feature is currently supported only in vLLM.
#
# By default, `fps=2` and `do_sample_frames=True`.
# With `do_sample_frames=True`, you can customize the `fps` value to set your desired video sampling rate.
chat_response = client.chat.completions.create(
model="Qwen/Qwen3.5-0.8B",
messages=messages,
max_tokens=32768,
temperature=0.7,
top_p=0.8,
presence_penalty=1.5,
extra_body={
"top_k": 20,
"mm_processor_kwargs": {"fps": 2, "do_sample_frames": True},
},
)
print("Chat response:", chat_response)
Thinking Mode
Qwen3.5 does not officially support the soft switch of Qwen3, i.e.,
/thinkand/nothink.
You can make the model think before response by configuring the API parameters. For example,
from openai import OpenAI
# Configured by environment variables
client = OpenAI()
messages = [
{"role": "user", "content": "Type \"I love Qwen3.5\" backwards"},
]
chat_response = client.chat.completions.create(
model="Qwen/Qwen3.5-0.8B",
messages=messages,
max_tokens=81920,
temperature=1.0,
top_p=0.95,
presence_penalty=1.5,
extra_body={
"top_k": 20,
"enable_thinking": True,
},
)
print("Chat response:", chat_response)
In thinking mode, we have observed that when using the recommended sampling parameters, Qwen3.5-0.8B is more prone to entering thinking loops compared to other Qwen3.5 models, which may prevent it from terminating generation properly. We recommend further tuning the sampling parameters specific to your use case and utilizing the API's streaming generation mode (if supported) to enable timely detection and interruption of such anomalous generation behaviors.
Agentic Usage
Qwen3.5 excels in tool calling capabilities.
Qwen-Agent
We recommend using Qwen-Agent to quickly build Agent applications with Qwen3.5.
To define the available tools, you can use the MCP configuration file, use the integrated tool of Qwen-Agent, or integrate other tools by yourself.
import os
from qwen_agent.agents import Assistant
# Define LLM
# Using OpenAI-compatible API endpoint. The API backend should disable response parsers.
llm_cfg = {
# Use your own model service compatible with OpenAI API by vLLM/SGLang:
'model': 'Qwen/Qwen3.5-0.8B',
'model_type': 'qwenvl_oai',
'model_server': 'http://localhost:8000/v1', # api_base
'api_key': 'EMPTY',
'generate_cfg': {
'use_raw_api': True,
# Pass the parameter of whether to enable thinking mode in this way
# 'extra_body': {
# 'chat_template_kwargs': {'enable_thinking': True}
# },
},
}
# Define Tools
tools = [
{'mcpServers': { # You can specify the MCP configuration file
"filesystem": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-filesystem", "/Users/xxxx/Desktop"]
}
}
}
]
# Define Agent
bot = Assistant(llm=llm_cfg, function_list=tools)
# Streaming generation
messages = [{'role': 'user', 'content': 'Help me organize my desktop.'}]
for responses in bot.run(messages=messages):
pass
print(responses)
# Streaming generation
messages = [{'role': 'user', 'content': 'Develop a dog website and save it on the desktop'}]
for responses in bot.run(messages=messages):
pass
print(responses)
Qwen Code
Qwen Code is an open-source AI agent for the terminal, optimized for Qwen models. It helps you understand large codebases, automate tedious work, and ship faster.
For more information, please refer to Qwen Code.
Best Practices
To achieve optimal performance, we recommend the following settings:
Sampling Parameters:
We suggest using the following sets of sampling parameters depending on the mode and task type:
- Non-thinking mode for text tasks:
temperature=1.0,top_p=1.00,top_k=20,min_p=0.0,presence_penalty=2.0,repetition_penalty=1.0 - Non-thinking mode for VL tasks:
temperature=0.7,top_p=0.80,top_k=20,min_p=0.0,presence_penalty=1.5,repetition_penalty=1.0 - Thinking mode for text tasks:
temperature=1.0,top_p=0.95,top_k=20,min_p=0.0,presence_penalty=1.5,repetition_penalty=1.0 - Thinking mode for VL or precise coding (e.g., WebDev) tasks:
temperature=0.6,top_p=0.95,top_k=20,min_p=0.0,presence_penalty=0.0,repetition_penalty=1.0
- Non-thinking mode for text tasks:
For supported frameworks, you can adjust the
presence_penaltyparameter between 0 and 2 to reduce endless repetitions. However, using a higher value may occasionally result in language mixing and a slight decrease in model performance.
Adequate Output Length: We recommend using an output length of 32,768 tokens for most queries. For benchmarking on highly complex problems, such as those found in math and programming competitions, we suggest setting the max output length to 81,920 tokens. This provides the model with sufficient space to generate detailed and comprehensive responses, thereby enhancing its overall performance.
Standardize Output Format: We recommend using prompts to standardize model outputs when benchmarking.
- Math Problems: Include "Please reason step by step, and put your final answer within \boxed{}." in the prompt.
- Multiple-Choice Questions: Add the following JSON structure to the prompt to standardize responses: "Please show your choice in the
answerfield with only the choice letter, e.g.,"answer": "C"."
No Thinking Content in History: In multi-turn conversations, the historical model output should only include the final output part and does not need to include the thinking content. It is implemented in the provided chat template in Jinja2. However, for frameworks that do not directly use the Jinja2 chat template, it is up to the developers to ensure that the best practice is followed.
Long Video Understanding: To optimize inference efficiency for plain text and images, the
sizeparameter in the releasedvideo_preprocessor_config.jsonis conservatively configured. It is recommended to set thelongest_edgeparameter in the video_preprocessor_config file to 469,762,048 (corresponding to 224k video tokens) to enable higher frame-rate sampling for hour-scale videos and thereby achieve superior performance. For example,{"longest_edge": 469762048, "shortest_edge": 4096}Alternatively, override the default values via engine startup parameters. For implementation details, refer to: vLLM / SGLang.
Citation
If you find our work helpful, feel free to give us a cite.
@misc{qwen3.5,
title = {{Qwen3.5}: Towards Native Multimodal Agents},
author = {{Qwen Team}},
month = {February},
year = {2026},
url = {https://qwen.ai/blog?id=qwen3.5}
}
- Downloads last month
- 102