Files changed (1) hide show
  1. README.md +53 -31
README.md CHANGED
@@ -16,17 +16,31 @@ pipeline_tag: image-text-to-text
16
  ---
17
 
18
  # LFM2-VL-450M — Torq build (Synaptics SL2619 NPU)
 
19
 
20
- Pre-compiled **Torq VMFB** build of LiquidAI's
21
- [LFM2-VL-450M](https://huggingface.co/LiquidAI/LFM2-VL-450M) vision-language model,
22
- ready to run on the Synaptics **SL2619** edge NPU. The SigLIP vision encoder and the
23
- LFM2 hybrid conv/attention text decoder both execute on the NPU in bf16; the token
 
 
 
 
 
 
 
 
 
 
 
24
  embeddings run on the host CPU.
25
 
26
  Image + prompt → caption / visual question answering. The image is encoded once and
27
  its KV cache is reused, so follow-up questions about the same image stay fast.
28
 
29
- ## Contents
 
 
30
 
31
  | File | Size | Role |
32
  |---|--:|---|
@@ -40,28 +54,7 @@ its KV cache is reused, so follow-up questions about the same image stay fast.
40
  | `cats-and-dogs-256.jpg` | — | sample 256-res image for the demo |
41
  | `onnx/` | ~2 GB | reference ONNX exports (vision encoder, merged decoder, embeddings) for non-Torq runtimes |
42
 
43
- ## Quick start
44
-
45
- Runs through the **LiquidAI-LFM2-VL-450M** demo in
46
- [synaptics-torq/torq-examples](https://github.com/synaptics-torq/torq-examples/tree/main/LiquidAI/LiquidAI-LFM2-VL-450M):
47
-
48
- ```sh
49
- # downloads this repo to models/Synaptics/LiquidAI-LFM2-VL-450M/
50
- python setup_demos.py LiquidAI-LFM2-VL-450M
51
-
52
- cd LiquidAI/LiquidAI-LFM2-VL-450M
53
- MODELS=../../models/Synaptics/LiquidAI-LFM2-VL-450M
54
- python src/infer.py \
55
- -m $MODELS/decoder_nolm.vmfb \
56
- --lm-head $MODELS/lm_head.vmfb \
57
- --vision $MODELS/vision_encoder_256.vmfb \
58
- --image-decoder $MODELS/decoder_image_2part_ \
59
- --image $MODELS/cats-and-dogs-256.jpg
60
- ```
61
-
62
- Then ask questions at the `Q:` prompt (e.g. *"What is the breed of the dog?"*).
63
-
64
- ## Model details
65
 
66
  - **Base model:** LiquidAI LFM2-VL-450M (SigLIP vision tower + LFM2 language model).
67
  - **Text decoder:** LFM2 — hidden size 1024, 16 layers, 16 attention heads, vocabulary 65 536, hybrid short-convolution + grouped-query attention.
@@ -70,14 +63,43 @@ Then ask questions at the `Q:` prompt (e.g. *"What is the breed of the dog?"*).
70
  - **Target:** Synaptics SL2619, compiled with the Torq compiler.
71
  - **On-device performance (SL2619, indicative):** vision encode ~2.4 s, image→KV prefill ~3.7 s, decode ~3.6–4.2 tok/s.
72
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
73
  ## License & attribution
74
 
75
- This repository is a redistribution of a model created by **Liquid AI, Inc.**,
76
- licensed under the **LFM Open License v1.0**. Copies of the license and the
77
- attribution notices are included alongside the model files:
78
 
79
  - **[LICENSE](https://huggingface.co/Synaptics/LiquidAI-LFM2-VL-450M/blob/main/LICENSE)** — a verbatim copy of the LFM Open License v1.0.
80
  - **[NOTICE](https://huggingface.co/Synaptics/LiquidAI-LFM2-VL-450M/blob/main/NOTICE)** — the copyright, patent, trademark, and attribution
81
  notices retained from the original Work (per Section 4(c) of the license).
82
 
83
- Original model: **[LFM2-VL-450M](https://huggingface.co/LiquidAI/LFM2-VL-450M)** · Copyright © Liquid AI, Inc.
 
 
 
 
 
 
 
 
16
  ---
17
 
18
  # LFM2-VL-450M — Torq build (Synaptics SL2619 NPU)
19
+ <img src="synaptics-logo-full-color.png" alt="Synaptics" width="600"/></img>
20
 
21
+ This repository provides compiled model files for LiquidAI's LFM2-VL-450M vision-language model, ready
22
+ to run on the **Synaptics SL2610-series Torq NPU**.
23
+ Give it an image and a natural-language question, and it answers questions about that image.
24
+
25
+ **Quick start guide**:
26
+ - [Buy a Machina kit](https://www.synaptics.com/products/embedded-processors/sl2610-product-line#devKit): Get an SL2600 Machina kit delivered to you
27
+ - [Torq Examples](https://synaptics-torq.github.io/torq-compiler/v/latest/): Use Torq-examples LiquidAI/LiquidAI-LFM2-VL-450M scripts to download and deploy on your Machina kit
28
+
29
+ <img src="astra_sl2600_machina.png" alt="SL2600 Machina kit" width="400"/></img>
30
+
31
+ ## Model Overview
32
+
33
+ LFM2‑VL is designed to process text and images with variable resolutions. Built on the LFM2 backbone, it is optimized for low-latency and edge AI applications.
34
+
35
+ LFM2-VL utilizes hybrid conv/attention text decoders that execute on the NPU in bf16; the token
36
  embeddings run on the host CPU.
37
 
38
  Image + prompt → caption / visual question answering. The image is encoded once and
39
  its KV cache is reused, so follow-up questions about the same image stay fast.
40
 
41
+ ## Model Features
42
+
43
+ ### Contents
44
 
45
  | File | Size | Role |
46
  |---|--:|---|
 
54
  | `cats-and-dogs-256.jpg` | — | sample 256-res image for the demo |
55
  | `onnx/` | ~2 GB | reference ONNX exports (vision encoder, merged decoder, embeddings) for non-Torq runtimes |
56
 
57
+ ## Model Details
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
58
 
59
  - **Base model:** LiquidAI LFM2-VL-450M (SigLIP vision tower + LFM2 language model).
60
  - **Text decoder:** LFM2 — hidden size 1024, 16 layers, 16 attention heads, vocabulary 65 536, hybrid short-convolution + grouped-query attention.
 
63
  - **Target:** Synaptics SL2619, compiled with the Torq compiler.
64
  - **On-device performance (SL2619, indicative):** vision encode ~2.4 s, image→KV prefill ~3.7 s, decode ~3.6–4.2 tok/s.
65
 
66
+ ## Tested Platforms
67
+
68
+ - [Synaptics Astra™ Machina SL2619 2GB](https://www.synaptics.com/products/embedded-processors/astra-machina-foundation-series)
69
+ ## Metrics
70
+
71
+ | Platform | Model / Stage | Environment | NPU Clock | TTFT | Infer / s |
72
+ | --------- | --------- | --------- | --------- | --------- | --------- |
73
+ | SL2619 2GB | LFM2-VL-450M | Torq v2.0.0 | 1 GHz | 2844 ms | 3.4 |
74
+
75
+
76
+ ## Deployment
77
+
78
+ The models have been tested with the following environment.
79
+
80
+ - Torq Compiler: *v2.0.0*
81
+ - Torq Runtime: *v2.0.0* included in Astra SDK release *scarthgap_6.12_v2.4.0*
82
+
83
+
84
+ ### Usage Tutorials / Example Apps
85
+
86
+ A usage example is provided in the [Torq Examples / LiquidAI-LFM2-VL-450M](https://github.com/synaptics-torq/torq-examples/tree/main/LiquidAI/LiquidAI-LFM2-VL-450M).
87
+
88
+ Check out the [README](https://github.com/synaptics-torq/torq-examples/blob/main/LiquidAI/LiquidAI-LFM2-VL-450M/README.md) for instructions.
89
+
90
  ## License & attribution
91
 
92
+ This repository is a redistribution of a model created by **Liquid AI, Inc.**, licensed under the **LFM Open License v1.0**. Copies of the license and the attribution notices are included alongside the model files:
 
 
93
 
94
  - **[LICENSE](https://huggingface.co/Synaptics/LiquidAI-LFM2-VL-450M/blob/main/LICENSE)** — a verbatim copy of the LFM Open License v1.0.
95
  - **[NOTICE](https://huggingface.co/Synaptics/LiquidAI-LFM2-VL-450M/blob/main/NOTICE)** — the copyright, patent, trademark, and attribution
96
  notices retained from the original Work (per Section 4(c) of the license).
97
 
98
+ Original model: **[LFM2.5-230M](https://huggingface.co/LiquidAI/LFM2-VL-450M)** · Copyright © Liquid AI, Inc.
99
+
100
+ ## Learn More
101
+
102
+ - [Synaptics AI Developer Zone](https://developer.synaptics.com?utm_source=hf): Get started with documentation, tutorials and resources for your Edge AI journey.
103
+ - [Torq Compiler Documentation](https://synaptics-torq.github.io/torq-compiler/v/latest/): Learn more about the Torq compiler based on MLIR and IREE.
104
+ - [Synaptics Astra SDK](https://synaptics-astra.github.io/doc/v/latest/): Learn more about the Yocto Project-based Linux software available for Astra SL processors.
105
+ - [Astra Support Portal](https://synacsm.atlassian.net/servicedesk/customer/portal/543?utm_source=hf): Connect with our engineering team and community.