JOSIE-1.1-4B-Instruct_DYNAMIC WI4 AFP32 — LiteRT-LM
On-device .litertlm conversion of Goekdeniz-Guelmez/JOSIE-1.1-4B-Instruct for the LiteRT-LM runtime. Runs completely offline on Android, Pixel, and Google Tensor / Mali-based devices.
| Spec | Value |
|---|---|
| Base | Goekdeniz-Guelmez/JOSIE-1.1-4B-Instruct |
| Format | .litertlm (LiteRT-LM bundle) |
| Quantization | DYNAMIC WI4 AFP32 |
| Context | 4096 tokens |
| Size | 2.09 GB |
| Modalities | Text only (vision/audio weights stripped) |
| License | Apache 2.0 |
What This Model Is
This is a community-packaged variant of the model provided for on-device inference via the LiteRT-LM engine.
Key characteristics:
- Text-only — image and audio understanding weights were removed to keep the bundle small and optimized for text tasks.
- 4096 context window — configured for local inference memory efficiency.
- WI4 quantized — optimized for mobile storage with balanced precision.
- On-device — runs completely offline on supported Android / Pixel / Tensor devices.
How to Use
Google AI Edge Gallery (Pixel / Android)
- Install Google AI Edge Gallery or a compatible fork.
- Add model via HuggingFace URL:
Werve/JOSIE-1.1-4B-Instruct-litert-lm - Download the 2.09 GB bundle over WiFi.
- Chat offline — no data leaves your device.
LiteRT-LM CLI (macOS / Linux / Desktop)
pip install litert-lm
litert-lm run --from-huggingface-repo Werve/JOSIE-1.1-4B-Instruct-litert-lm \
JOSIE-1.1-4B-Instruct_DYNAMIC WI4 AFP32.litertlm \
--prompt "Your prompt here" \
--backend gpu
Technical Details
Export Parameters
- Quantization:
dynamic_wi4_afp32 - Context Length:
4096 - Export Library:
litert-torch0.9.1+
Weight Extraction
Only text-decoder safetensors were retained. Multimodal components (vision/audio) were stripped to reduce total size from the original checkpoint.
Credits & Attribution
- Original Model:
Goekdeniz-Guelmez/JOSIE-1.1-4B-Instruct - LiteRT-LM packaging: Werve
This is an unofficial community build. It is not published by Google or the official LiteRT team.
- Downloads last month
- 39
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for Werve/JOSIE-1.1-4B-Instruct-litert-lm
Base model
Qwen/Qwen3-4B-Instruct-2507 Finetuned
Goekdeniz-Guelmez/JOSIE-1.1-4B-Instruct