Publish Liquid Crow model card and examples

README.md

@@ -1,19 +1,97 @@
 ---
+base_model: LiquidAI/LFM2.5-VL-450M-Extract
+library_name: transformers
+pipeline_tag: image-text-to-text
 license: apache-2.0
 tags:
   - vitrus
-  - vision
-  - world-description
+  - vision-language-model
+  - multimodal
+  - lora
+  - onnx
   - gguf
+  - world-model
 ---
 
-# Vitrus World Descriptor
+# Liquid Crow
 
-A compact vision model package for describing the visible world from a camera frame.
+[![Liquid Crow by Vitrus](media/liquid-crow-cover.png)](https://www.vitrus.com/news/micro-world-models-1)
 
-## Files
+**A small brain. Intelligent. Always observing.**
 
-- `vitrus-world-descriptor-q4_k_m.gguf` — language model
-- `vitrus-world-descriptor-vision-f16.gguf` — vision encoder and projector
+Liquid Crow is a compact visual brain designed to stay close to the world it watches: small enough to live beside physical agents, intelligent enough to turn a camera frame into useful context, and always observing so the system can understand what changes next.
 
-Both files are required for visual inference.
+[Live camera demo](https://www.vitrus.com/demos/micro-world-model) · [Research note](https://www.vitrus.com/news/micro-world-models-1) · [Vitrus](https://www.vitrus.com)
+
+## System
+
+```text
+camera frame → image processor → vision projector → compact VLM → world description
+                                        │
+                         LoRA · ONNX · ONNX Q4 · GGUF
+```
+
+This repository is the single public release point for Liquid Crow: the model adapter, deployment formats, processor configuration, integrity manifests, code examples, and media.
+
+## Example
+
+<video src="media/liquid-crow-demo.mp4" controls width="100%"></video>
+
+[Open the example video](https://huggingface.co/lucas-vitrus/liquid-crow/blob/main/media/liquid-crow-demo.mp4)
+
+## Repository layout
+
+| Path | Contents |
+|---|---|
+| `lora/` | PEFT LoRA adapter and processor files |
+| `onnx/model.onnx` | FP16 ONNX forward-logits graph |
+| `onnx/model-q4.onnx` | ONNX Runtime weight-only 4-bit graph |
+| `gguf/` | Q4_K_M language model and FP16 vision projector |
+| `transformers/` | tokenizer, processor, chat template, and model configuration |
+| `examples/` | LoRA, ONNX Runtime, and llama.cpp examples |
+| `media/` | cover image and example video |
+| `RELEASE_MANIFEST.json` | sizes and SHA-256 hashes for the primary artifacts |
+
+## Quick start: LoRA
+
+```bash
+pip install torch transformers peft pillow huggingface_hub
+python examples/load_lora.py path/to/image.jpg
+```
+
+The example downloads the Liquid Crow adapter from this repository and applies it to `LiquidAI/LFM2.5-VL-450M-Extract`.
+
+## Quick start: GGUF
+
+```bash
+hf download lucas-vitrus/liquid-crow \
+  gguf/LFM2.5-VL-450M-reasoning-tracing-step2000-Q4_K_M.gguf \
+  gguf/LFM2.5-VL-450M-reasoning-tracing-step2000-mmproj-f16.gguf \
+  --local-dir ./liquid-crow
+
+llama-server \
+  -m ./liquid-crow/gguf/LFM2.5-VL-450M-reasoning-tracing-step2000-Q4_K_M.gguf \
+  --mmproj ./liquid-crow/gguf/LFM2.5-VL-450M-reasoning-tracing-step2000-mmproj-f16.gguf \
+  -c 1024 --image-min-tokens 64 --image-max-tokens 256
+```
+
+## Quick start: ONNX Runtime
+
+```bash
+pip install onnxruntime transformers torch pillow huggingface_hub
+python examples/onnx_forward.py path/to/image.jpg --quantized
+```
+
+The ONNX files expose a multimodal forward-logits graph. Autoregressive generation remains the responsibility of the consuming runtime. The quantized graph requires ONNX Runtime support for `com.microsoft::MatMulNBits`.
+
+## Runtime notes
+
+- Both GGUF files are required for visual inference.
+- The LoRA adapter requires the base model.
+- `lora/` contains the final adapter; the ONNX and GGUF deployment exports use the selected step-2,000 checkpoint.
+- `onnx/model-q4.onnx` quantizes eligible matrix weights; other operations retain their exported precision.
+- See `SHA256SUMS` and `export_manifest.json` for artifact integrity and interface details.
+
+## License
+
+Apache-2.0. The base model remains subject to its upstream terms.

RELEASE_MANIFEST.json

@@ -0,0 +1,40 @@
+{
+  "repository": "lucas-vitrus/liquid-crow",
+  "artifacts": [
+    {
+      "path": "lora/adapter_model.safetensors",
+      "bytes": 28748008,
+      "sha256": "b240113d9abb27ebaeb6e6705e5481cc3882b3befed085fb56a71258dfe6c7f6"
+    },
+    {
+      "path": "onnx/model.onnx",
+      "bytes": 898430690,
+      "sha256": "d3a817fbbb916cf76abcca23be0742f09210e098500c904a78956a50758d4c14"
+    },
+    {
+      "path": "onnx/model-q4.onnx",
+      "bytes": 367162143,
+      "sha256": "0b8483f1f94dc1c3c0ef53b4b53248d867e6c7e29fb6dab6bc08c75654e8701e"
+    },
+    {
+      "path": "gguf/LFM2.5-VL-450M-reasoning-tracing-step2000-Q4_K_M.gguf",
+      "bytes": 229312864,
+      "sha256": "ac097f6bdefc0d04457ff4dafe980c45fc83ca84170d19791d94c844fd327e16"
+    },
+    {
+      "path": "gguf/LFM2.5-VL-450M-reasoning-tracing-step2000-mmproj-f16.gguf",
+      "bytes": 189125248,
+      "sha256": "7397ddeaa3dd43c27878c4b5270d3eb1dfbc21a93dc637f49aa2d9b9b11201f9"
+    },
+    {
+      "path": "media/liquid-crow-cover.png",
+      "bytes": 681437,
+      "sha256": "354d6d6cbb60b186dfde1fb83e7e80df6e165efe885215cf450f72741ecac659"
+    },
+    {
+      "path": "media/liquid-crow-demo.mp4",
+      "bytes": 2753701,
+      "sha256": "86dc40bc8ec2180e728ddca7a5c1dd97ba298b688c867bb648a72b0b240abb4d"
+    }
+  ]
+}

examples/load_lora.py

@@ -0,0 +1,67 @@
+#!/usr/bin/env python3
+"""Run one image-description turn with the published Vitrus LoRA adapter."""
+
+from __future__ import annotations
+
+import sys
+
+import torch
+from huggingface_hub import snapshot_download
+from peft import PeftModel
+from PIL import Image
+from transformers import AutoModelForImageTextToText, AutoProcessor
+
+
+REPO_ID = "lucas-vitrus/liquid-crow"
+BASE_MODEL = "LiquidAI/LFM2.5-VL-450M-Extract"
+PROMPT = "Describe the world you see in details."
+
+
+def main() -> None:
+    if len(sys.argv) != 2:
+        raise SystemExit("usage: python examples/load_lora.py path/to/image.jpg")
+
+    image = Image.open(sys.argv[1]).convert("RGB")
+    adapter_dir = snapshot_download(REPO_ID, allow_patterns=["lora/*"])
+    device = "cuda" if torch.cuda.is_available() else "cpu"
+    dtype = torch.float16 if device == "cuda" else torch.float32
+
+    processor = AutoProcessor.from_pretrained(
+        BASE_MODEL,
+        min_image_tokens=64,
+        max_image_tokens=256,
+        do_image_splitting=True,
+    )
+    base = AutoModelForImageTextToText.from_pretrained(
+        BASE_MODEL,
+        dtype=dtype,
+        low_cpu_mem_usage=True,
+    ).to(device)
+    model = PeftModel.from_pretrained(base, f"{adapter_dir}/lora").eval()
+
+    messages = [
+        {
+            "role": "user",
+            "content": [
+                {"type": "text", "text": PROMPT},
+                {"type": "image", "image": image},
+            ],
+        }
+    ]
+    inputs = processor.apply_chat_template(
+        messages,
+        add_generation_prompt=True,
+        tokenize=True,
+        return_tensors="pt",
+        return_dict=True,
+    ).to(device)
+
+    with torch.inference_mode():
+        output_ids = model.generate(**inputs, max_new_tokens=256, do_sample=False)
+
+    generated_ids = output_ids[:, inputs["input_ids"].shape[1] :]
+    print(processor.batch_decode(generated_ids, skip_special_tokens=True)[0].strip())
+
+
+if __name__ == "__main__":
+    main()

examples/onnx_forward.py

@@ -0,0 +1,73 @@
+#!/usr/bin/env python3
+"""Run a single multimodal forward pass with the published ONNX graph."""
+
+from __future__ import annotations
+
+import argparse
+from pathlib import Path
+
+import numpy as np
+import onnxruntime as ort
+from huggingface_hub import hf_hub_download
+from PIL import Image
+from transformers import AutoProcessor
+
+
+REPO_ID = "lucas-vitrus/liquid-crow"
+BASE_MODEL = "LiquidAI/LFM2.5-VL-450M-Extract"
+INPUT_NAMES = (
+    "input_ids",
+    "pixel_values",
+    "spatial_shapes",
+    "pixel_attention_mask",
+    "attention_mask",
+)
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("image", type=Path)
+    parser.add_argument("--quantized", action="store_true")
+    return parser.parse_args()
+
+
+def main() -> None:
+    args = parse_args()
+    filename = "onnx/model-q4.onnx" if args.quantized else "onnx/model.onnx"
+    model_path = hf_hub_download(REPO_ID, filename=filename)
+    image = Image.open(args.image).convert("RGB")
+
+    processor = AutoProcessor.from_pretrained(
+        BASE_MODEL,
+        min_image_tokens=64,
+        max_image_tokens=256,
+        do_image_splitting=True,
+    )
+    messages = [
+        {
+            "role": "user",
+            "content": [
+                {"type": "text", "text": "Describe the world you see in details."},
+                {"type": "image", "image": image},
+            ],
+        }
+    ]
+    inputs = processor.apply_chat_template(
+        messages,
+        add_generation_prompt=True,
+        tokenize=True,
+        return_tensors="np",
+        return_dict=True,
+    )
+    ort_inputs = {name: np.asarray(inputs[name]) for name in INPUT_NAMES}
+    session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])
+    logits = session.run(["logits"], ort_inputs)[0]
+    next_token = int(logits[0, -1].argmax())
+
+    print(f"logits shape: {logits.shape}")
+    print(f"greedy next token: {next_token} ({processor.tokenizer.decode([next_token])!r})")
+    print("Use these logits inside an autoregressive generation loop for full decoding.")
+
+
+if __name__ == "__main__":
+    main()

examples/requirements.txt

@@ -0,0 +1,7 @@
+huggingface_hub
+numpy
+onnxruntime
+peft
+pillow
+torch
+transformers

examples/run_gguf.sh

@@ -0,0 +1,16 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+MODEL_DIR="${1:-./liquid-crow}"
+
+hf download lucas-vitrus/liquid-crow \
+  gguf/LFM2.5-VL-450M-reasoning-tracing-step2000-Q4_K_M.gguf \
+  gguf/LFM2.5-VL-450M-reasoning-tracing-step2000-mmproj-f16.gguf \
+  --local-dir "$MODEL_DIR"
+
+llama-server \
+  -m "$MODEL_DIR/gguf/LFM2.5-VL-450M-reasoning-tracing-step2000-Q4_K_M.gguf" \
+  --mmproj "$MODEL_DIR/gguf/LFM2.5-VL-450M-reasoning-tracing-step2000-mmproj-f16.gguf" \
+  -c 1024 \
+  --image-min-tokens 64 \
+  --image-max-tokens 256