| # SimpleTool-VLM |
|
|
| SimpleTool-VLM is a research release of a Qwen3-VL-4B model adapted for structured tool calls from images and text. It emits a function name and only the argument heads required by the supplied tool schema. The public release contains inference code and reproducibility notes; model weights are hosted separately on Hugging Face. |
|
|
| ## Highlights |
|
|
| - Direct and Adaptive modes. |
| - Function plus dynamically sized `arg1...argN` heads (`N` is the maximum property count in the supplied tools). |
| - Image input is passed as pixels; invoice values are not copied into hidden prompt fields. |
| - Raw branch outputs and legality are preserved for auditing. |
|
|
| ## Intended use |
|
|
| Research on real-time multimodal structured decisions, tool calling, and embodied-agent interfaces. This is not an OCR benchmark or a general invoice accounting system. Validate every action before execution. |
|
|
| ## Model variants |
|
|
| - BF16 merged checkpoint: `Cialtion/SimpleTool-VLM`. |
| - W4A16 checkpoint: `Cialtion/SimpleTool-VLM-W4A16` (when the upload is complete). |
| - FP8 checkpoint: `Cialtion/SimpleTool-VLM-FP8` (when the upload is complete). |
|
|
| The quantized variants use the same task adapter and tokenizer contract; backend support depends on the serving stack. |
|
|
| ## Quick start |
|
|
| The lightweight HTTP implementation is in `inference/`. It accepts one PNG, text messages, and OpenAI-style function tools. See `examples/invoice_request.json`. The service returns `prediction.raw`, `prediction.legal`, `events`, and server timings. |
|
|
| ## Limitations |
|
|
| This release does not claim strict adapter-to-merged numerical parity, arbitrary-image robustness, OCR superiority, or native Qwen chain-of-thought. Adaptive `content` is a short task-controlled field. The included invoice demo is an end-to-end protocol demonstration, not a financial approval system. |
|
|
| ## License and attribution |
|
|
| Code and release documentation are Apache-2.0. The checkpoint remains subject to the Qwen3-VL base-model license and any upstream terms. Users must review upstream terms before redistribution or commercial deployment. |
|
|
