Update README.md

README.md

@@ -1,3 +1,148 @@
----
-license: apple-amlr
----
+---
+license: apple-amlr
+pipeline_tag: text-generation
+library_name: litert-lm
+tags:
+- ml-fastvlm
+- litert
+- litertlm
+base_model:
+- apple/FastVLM-0.5B
+---
+
+# litert-community/FastVLM-0.5B
+
+*Main Model Card*: [apple/FastVLM-0.5B](https://huggingface.co/apple/FastVLM-0.5B)
+
+This model card provides *FastVLM-0.5B converted for LiteRT* that are ready for deployment on Android and Desktop.
+
+FastVLM was introduced in [FastVLM: Efficient Vision Encoding for Vision Language Models](https://www.arxiv.org/abs/2412.13303). *(CVPR 2025)*, this model demonstrates improvement in time-to-first-token (TTFT) with performance and is suitable for edge device deployment.
+
+*Disclaimer*: This model converted for LiteRT is licensed under the [Apple Machine Learning Research Model License Agreement](https://huggingface.co/apple/deeplabv3-mobilevit-small/blob/main/LICENSE). The model is converted and quantized from PyTorch model weight into the LiteRT/Tensorflow-Lite format (no retraining or further customization).
+
+# How to Use
+
+## Android
+
+### 1. Add the dependency
+
+Make sure you have the necessary dependency in your `Gradle` file.
+
+```
+dependencies {
+    implementation("com.google.ai.edge.litertlm:litertlm:<LATEST_VERSION>")
+}
+```
+
+### 2. Inference with the LiteRT-LM API
+
+```kotlin
+import com.google.ai.edge.litertlm.*
+
+suspend fun main() {
+  Engine.setNativeMinLogSeverity(LogSeverity.ERROR) // hide log for TUI app
+  val engineConfig = EngineConfig(
+      modelPath = "/path/to/your/model.litertlm", // Replace with model path
+      backend = Backend.CPU, // Or Backend.GPU
+      visionBackend = Backend.GPU,
+  )
+
+  // See the Content class for other variants.
+  val multiModalMessage = Message.of(
+      Content.ImageFile("/path/to/image"),
+      Content.Text("Describe this image."),
+  )
+  Engine(engineConfig).use { engine ->
+    engine.initialize()
+
+    engine.createConversation().use { conversation ->
+      while (true) {
+        print("\n>>> ")
+        conversation.sendMessageAsync(Message.of(readln())).collect { print(it) }
+      }
+    }
+  }
+}
+```
+
+Try running this model on NPU by using this .litertlm file and setting your EngineConfig’s backend to NPU. To check if your phone’s NPU is supported see this [guide](https://ai.google.dev/edge/litert/next/npu).
+
+## Desktop 
+
+To build a Desktop application, C++ is the current recommendation. See the following code sample.
+
+```cpp
+// Create engine with proper multimodality backend.
+auto engine_settings = EngineSettings::CreateDefault(
+    model_assets,
+    /*backend=*/litert::lm::Backend::CPU,
+    /*vision_backend*/litert::lm::Backend::GPU,
+);
+
+// Send message to the LLM with image data.
+absl::StatusOr<Message> model_message = (*conversation)->SendMessage(
+    JsonMessage{
+        {"role", "user"},
+        {"content", { // Now content must be an array.
+          {{"type", "text"}, {"text", "Describe the following image: "}},
+          {{"type", "image"}, {"path", "/file/path/to/image.jpg"}}
+        }},
+    });
+CHECK_OK(model_message);
+
+// Print the model message.
+std::cout << *model_message << std::endl;
+```
+
+# Performance
+
+## Android
+
+Benchmarked on Xiaomi 17 Pro Max.
+
+
+<table border="1">
+  <tr>
+   <th style="text-align: left">Backend</th>
+   <th style="text-align: left">Quantization scheme</th>
+   <th style="text-align: left">Context length</th>
+   <th style="text-align: left">Prefill (tokens/sec)</th>
+   <th style="text-align: left">Decode (tokens/sec)</th>
+   <th style="text-align: left">Time-to-first-token (sec)</th>
+   <th style="text-align: left">Memory (RSS in MB)</th>
+   <th style="text-align: left">Model size (MB)</th>
+   <th style="text-align: left">Model File</th>
+  </tr>
+  <tr>
+<td><p style="text-align: left">GPU</p></td>
+<td><p style="text-align: left">dynamic_int8</p></td>
+<td><p style="text-align: right">1280</p></td>
+<td><p style="text-align: right">- tk/s</p></td>
+<td><p style="text-align: right">- tk/s</p></td>
+<td><p style="text-align: right">- s</p></td>
+<td><p style="text-align: right">- MB</p></td>
+<td><p style="text-align: right">- MB</p></td>
+<td><p style="text-align: left"><a style="text-decoration: none" href="https://huggingface.co/litert-community/FastVLM-0.5B/resolve/main/FastVLM-0.5B.litertlm">&#128279;</a></p></td>
+</tr>
+
+<tr>
+<td><p style="text-align: left">NPU</p></td>
+<td><p style="text-align: left">dynamic_int8</p></td>
+<td><p style="text-align: right">1280</p></td>
+<td><p style="text-align: right">- tk/s</p></td>
+<td><p style="text-align: right">- tk/s</p></td>
+<td><p style="text-align: right">- s</p></td>
+<td><p style="text-align: right">- MB</p></td>
+<td><p style="text-align: right">- MB</p></td>
+<td><p style="text-align: left"><a style="text-decoration: none" href="https://huggingface.co/litert-community/FastVLM-0.5B/resolve/main/FastVLM-0.5B.sm8850.litertlm">&#128279;</a></p></td>
+</tr>
+
+</table>
+
+
+
+Notes:
+* Model Size: measured by the size of the file on disk.
+* Benchmark is run with cache enabled and initialized. During the first run, the latency and memory usage may differ.
+
+