| --- |
| license: apache-2.0 |
| datasets: |
| - moca-embed/dclm_20b |
| - openbmb/UltraChat |
| language: |
| - en |
| library_name: transformers |
| --- |
| # Model Card for Model ID |
|
|
| <!-- Provide a quick summary of what the model is/does. --> |
|
|
|
|
| ## Model Details |
| - 245M parameters |
| - 4 Layers |
| - D_size 1280 |
| - 16 MoE |
| - 8 KV |
| - FP32 2.3GB - Onix export |
| |
| Trained on only 20B tokens of web text data. |
| |
| Fine-tuned on 80K of UltraChat, no LoRA or similar tricks. |
| |
| ### Model Description |
| |
| # Lulu Local Android Demo |
| |
| **Lulu Local** is an offline Android AI demo by **Open Machine**. |
| |
| This release runs a local Lulu language model directly on an Android phone using **ONNX Runtime CPU inference**. |
| |
| No cloud. |
| No server. |
| No GPU. |
| No NPU. |
| No internet required after install. |
| |
| Runs on the Samsung A25 5G. |
| |
| This is a raw early proof that a custom local model can run directly on consumer Android hardware. |
| |
| For the record this is a literally un-optimized model, with heavily python loop, pure ONNX export of 2.3GB FP32. This is currently running on the CPU, we haven't touched the NPU, Vulcan or anything else yet. |
| The current generation takes about three minutes (a full forward pass on 128CTX as I mentioned, it's unoptimized), and APK file is here with GitHub follows for Onix model and Android. Again No Custom Runtimes: Just standard ONNX format loaded straight into Android memory. |
| This is running on your Exynos—with the consideration that after we chatted for 10 minutes, the battery didn't move, and no heating occurred. |
| We completed everything in the last two days: training, benchmarks, fine-tuning, and Onix runtime, all for less than €1000. |
| |
| Why this is interesting |
| |
| Most mobile LLM demos rely on one or more of the following: |
| |
| heavily quantized models |
| GPU acceleration |
| NPU acceleration |
| server-side inference |
| vendor SDKs |
| cloud APIs |
| |
| This demo is intentionally simple and direct: |
| |
| Android app |
| + ONNX Runtime |
| + local tokenizer |
| + local ONNX model |
| + CPU only |
| |
| The current model is not small, not heavily optimized, and not using mobile accelerator tricks. |
| That is the point of the demo. |
| |
| Model architecture note |
| |
| The Android build uses a stateful single-token step ONNX export. |
| |
| The runtime loop is: |
| |
| token_id + position + cache tensors |
| → ONNX step model |
| → logits + updated cache tensors |
| → sample next token |
| → repeat |
|
|
| This replaced the earlier full-sequence ONNX path, which was much slower and used much more memory during generation. |
|
|
| Current ONNX interface: |
|
|
| Inputs: |
| - token_id: [1, 1] int64 |
| - pos: [1] int64 |
| - k_0, v_0 ... k_23, v_23 |
| |
| Outputs: |
| - logits: [1, 32000] float32 |
| - out_k_0, out_v_0 ... out_k_23, out_v_23 |
| |
| Cache shape per K/V tensor: |
| |
| [1, 16, 128, 80] |
| |
| Total runtime cache is about 31 MB. |
| |
| - **Developed by: The Open Machine** |
| - **Model type:** [The Open Machine Transformers Version] |
| - **Language(s) (NLP):** [English] |
| - **License:** [Apache 2.0 ] |
| |
| ### Model Sources [optional] |
| |
| <!-- Provide the basic links for the model. --> |
| |
| - **Repository:** [Wiull be provided in upcoming days] |
| - **Paper [optional]:** [Coming Soon] |
| - **Demo [optional]:** [More Information Needed] |
| |
| ## Uses |
| |
| Demo highlights |
| Fully offline Android assistant |
| Runs on mobile CPU only |
| Stateful single-token ONNX generation |
| Live token streaming UI |
| Battery / RAM / speed display |
| Cool / Turbo mode |
| Cool: 2 CPU threads |
| Turbo: 4 CPU threads |
| No GPU acceleration |
| No NPU acceleration |
| No network calls required for inference |
| |
| Tested device |
| |
| Early demo testing was done on a Samsung A25-class Android phone. |
| |
| Observed behavior: |
| |
| Model loads locally from app storage |
| Generation works fully offline |
| CPU-only generation is slow but usable for demo purposes |
| Example speed observed around 0.20 tok/s, depending on temperature, prompt length, and thread mode |
| |
| This is not yet optimized. |
| |
| Install |
| |
| Download the APK: |
| |
| LuluLocal-Android-CPU-fp32.apk |
| |
| On Android: |
| |
| Open the APK file. |
| Allow install from unknown sources if Android asks. |
| Install. |
| Open Lulu. |
| Wait for the model to load. |
| Ask a question. |
| |
| First load may take longer because the app prepares the local ONNX model. |
| |
| ### Direct Use |
| |
| <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. --> |
| |
| [Privacy |
| |
| Inference is local. |
| |
| The demo is designed so prompts are processed on-device. |
| No cloud inference is required. |
| |
| If you build or modify the app, review the source code and Android permissions yourself.] |
| |
| |
| |
| ### Out-of-Scope Use |
| |
| [Important warning |
| |
| This is an experimental local AI demo. |
| |
| The model may: |
| |
| hallucinate |
| answer incorrectly |
| repeat itself |
| generate incomplete text |
| be slow on low-end hardware |
| consume significant battery and RAM |
| |
| Do not use this for medical, legal, financial, emergency, or safety-critical decisions.] |
| |
| ## Bias, Risks, and Limitations |
| |
| <!-- This section is meant to convey both technical and sociotechnical limitations. --> |
| |
| [Current limitations |
| CPU only |
| fp32 ONNX model is large |
| no NPU backend yet |
| no GPU/Vulkan backend yet |
| no quantization yet |
| context length currently limited |
| APK size is large |
| generation quality is still experimental] |
| |
| |
| ## Model Card Authors [optional] |
| Credits |
| |
| Built by Open Machine. |
| |
| Lulu is an experimental local AI assistant project focused on running useful AI directly on personal devices. |
| |
| ## Model Card Contact |
| |
| Open Machine |
| info@theopenmachine.com |
| |
