Image-Text-to-Text
Transformers
Safetensors
qwen3_5
vision-language
tool-calling
agent
local-agents
edge
conversational
Instructions to use KartiOS/Karti-Small-VL-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use KartiOS/Karti-Small-VL-4B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="KartiOS/Karti-Small-VL-4B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("KartiOS/Karti-Small-VL-4B") model = AutoModelForMultimodalLM.from_pretrained("KartiOS/Karti-Small-VL-4B", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use KartiOS/Karti-Small-VL-4B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "KartiOS/Karti-Small-VL-4B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "KartiOS/Karti-Small-VL-4B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/KartiOS/Karti-Small-VL-4B
- SGLang
How to use KartiOS/Karti-Small-VL-4B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "KartiOS/Karti-Small-VL-4B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "KartiOS/Karti-Small-VL-4B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "KartiOS/Karti-Small-VL-4B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "KartiOS/Karti-Small-VL-4B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use KartiOS/Karti-Small-VL-4B with Docker Model Runner:
docker model run hf.co/KartiOS/Karti-Small-VL-4B
Commit History
-
record v1 measured throughput; point the v0 mirror note at the pre-v1 tag
Verified signature31a361c -
results figure; banner build-state corrected to v1 released
Verified signature9c8bbbc -
concise card: parameters, results, measured DGX Spark throughput
Verified signatured4448d6 -
card leads with the result; recipe.json scrubbed of internal infrastructure
Verified signaturee849947 -
remove internal governance record from the public repo
Verified signature21121ce -
v1: first trained checkpoint (merged BF16)
Verified signatureReplaces the v0 baseline, which was the UNMODIFIED Qwen3.5-4B and stays fetchable at the pre-v1 tag. invented-identifier 0.378 -> 0.023, panel-reading 0.595 -> 0.967, ScreenSpot 0.718 -> 0.907, vision_probe 0.727 -> 0.732 (no regression), VizWiz false-refusal 0.220 -> 0.227 (flat). Read the caveats in the card: much of the ScreenSpot gain is learning the normalised 0-1000 coordinate convention rather than better sight, and v1 does not read photographs better than v0 -- it merely stops over-refusing.0564e69
-
v0 has a vision baseline now: vision_probe 0.727 +/- 0.030
The card and recipe both said no vision benchmark existed. One does - a curated slice mined from MMStar, RealWorldQA and V*Bench, selected model-independently and scored in its own lane. v0 measured 0.727 +/- 0.030 over 220 items with a clean contamination canary, on the untrained base. The private camera task is referenced without its score: it runs only against local services and those frames stay on our hardware.0c32436 -
Correct the card and recipe: v0 weights ARE published here
README's Versions section and recipe.json still said the weights were not mirrored, contradicting the Weights section and publication.json after commit 8f16249 uploaded them. Both shard SHA-256 digests match Qwen/Qwen3.5-4B @ 851bf6e8, so the mirror claim is verified, not asserted.4afb0bd
-
Publish v0 as a runnable release
The weights are mirrored here now, so the repository is self-contained rather than a card pointing somewhere else. Adds a working load snippet and a serve line carrying the two settings this model needs and will not announce: the qwen3_xml tool parser, and thinking disabled. Records the weights approval alongside the original scaffolding one instead of overwriting it. The gate on TRAINED checkpoints stays closed.03210fd -
v0 weights: Qwen3.5-4B BF16 at pinned revision 851bf6e8
Verified signature8f16249 -
Pin and measure v0, the reference baseline
v0 is the starting line: the pinned base at BF16, measured on our own hardware so every later version has an exact thing to beat. Records throughput, the vision check, the tool-call parse and both canaries. The throughput is the notable result - at BF16 this 4B runs at about the same single-stream rate as the 27B NVFP4 on the same box, which is a memory-bandwidth finding and the case for quantizing the deployment lane.03884af -
Correct the base to Qwen3.5-4B
The chat-template gate was right but did not finish the argument: more than one base passed it. Among those, Qwen3.5-4B shares Qwen3_5ForConditionalGeneration with the resident local 27B vision model, so it needs no second serving path, and it has 4 KV heads against 8 for half the KV cache. The vision tower configuration is identical, so no sight is traded. Records the two carried differences: the non-JSON call serialization (which the control run must report rather than hide) and thinking-on-by-default with its required empty-content canary.9b981c1 -
Scaffolding: base pinned, identity scheme defined, no run executed
Verified signature9ec4367 -
initial commit
Verified signature6862973