README.md

---
    language:
    - en
    tags:
    - duoneural
    - litert
    - edge
    - gguf
    - on-device
    - smollm
- smol
- tiny
- litert
- edge
- instruct
    base_model: HuggingFaceTB/SmolLM2-360M-Instruct
    pipeline_tag: text-generation
    license: apache-2.0
    ---

    # SmolLM2-360M-Instruct-LiteRT

    **SmolLM2 360M Instruct — sub-400M on-device assistant** — converted for mobile and edge deployment by [DuoNeural](https://huggingface.co/DuoNeural).

    - **Source model:** [HuggingFaceTB/SmolLM2-360M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-360M-Instruct)
    - **Format:** GGUF Q4_K_M (llama.cpp-compatible)
    - **File size:** 271 MB
    - **Quantization:** 4-bit K-mean (Q4_K_M) — excellent accuracy/size trade-off for edge devices
    - **Target platforms:** Android, iOS, desktop edge inference
    - **Converted:** 2026-05-06 06:09:45 by Archon / DuoNeural

    ## Usage

    ### llama.cpp (CLI)
    ```bash
    ./llama-cli -m SmolLM2-360M-Instruct-LiteRT_Q4_K_M.gguf -n 512 --temp 0.7
    ```

    ### Google AI Edge / MediaPipe (Android/iOS)
    This GGUF is compatible with [MLC-LLM](https://github.com/mlc-ai/mlc-llm) and [llama.cpp Android bindings](https://github.com/ggerganov/llama.cpp) for on-device inference. For use with [Google Edge Gallery](https://ai.google.dev/edge/gallery), convert to `.task` bundle using MediaPipe LLM conversion tools.

    ### Python via llama-cpp-python
    ```python
    from llama_cpp import Llama

    llm = Llama(
        model_path="SmolLM2-360M-Instruct-LiteRT_Q4_K_M.gguf",
        n_ctx=2048,
        n_threads=4,
        verbose=False,
    )

    response = llm.create_chat_completion(
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Hello! How can you help me today?"},
        ]
    )
    print(response["choices"][0]["message"]["content"])
    ```

    ### Ollama
    ```bash
    ollama run hf.co/DuoNeural/SmolLM2-360M-Instruct-LiteRT
    ```

    ## About the Conversion

    Converted using [llama.cpp](https://github.com/ggerganov/llama.cpp) GGUF pipeline with CUDA acceleration.
    Source weights downloaded from HuggingFace, converted to F16 GGUF, then quantized to Q4_K_M.

    ---

    ## DuoNeural

    **DuoNeural** is an open AI research lab — human + AI in collaboration.

    | Platform | Link |
    |----------|------|
    | HuggingFace | [huggingface.co/DuoNeural](https://huggingface.co/DuoNeural) |
    | Website | [duoneural.com](https://duoneural.com) |
    | GitHub | [github.com/DuoNeural](https://github.com/DuoNeural) |
    | X / Twitter | [@DuoNeural](https://x.com/DuoNeural) |
    | Email | duoneural@proton.me |
    | Newsletter | [duoneural.beehiiv.com](https://duoneural.beehiiv.com) |
    | Support | [buymeacoffee.com/duoneural](https://buymeacoffee.com/duoneural) |

    ### DuoNeural Research Publications

    | Title | DOI |
    |-------|-----|
    | [Nano-CTM: Ternary Continuous Thought Machines with Thought-Space Self-Prediction for Efficient Iterative Reasoning](https://doi.org/10.5281/zenodo.19775622) | [10.5281/zenodo.19775622](https://doi.org/10.5281/zenodo.19775622) |
    | [Recurrence as World Model: CTM Learns Implicit Belief States in Partially Observable Physical Environments](https://doi.org/10.5281/zenodo.19810620) | [10.5281/zenodo.19810620](https://doi.org/10.5281/zenodo.19810620) |
    | [Per-Object Slot Decomposition for Scalable Neural World Modeling: When Does Attention Beat Mean-Field?](https://doi.org/10.5281/zenodo.19846804) | [10.5281/zenodo.19846804](https://doi.org/10.5281/zenodo.19846804) |
    | [The Dynamical Horizon Principle: CTM Gates Converge to the Predictability Limit of Dynamical Systems](https://doi.org/10.5281/zenodo.19952612) | [10.5281/zenodo.19952612](https://doi.org/10.5281/zenodo.19952612) |

    *Open access, CC BY 4.0. Authored by Archon, Jesse Caldwell, Aura — DuoNeural.*

    ### Research Team
    - **Jesse** — Vision, hardware, direction
    - **Archon** — Lab Director, post-training, abliteration, experiments
    - **Aura** — Research AI, literature synthesis, novel proposals

    *Subscribe to the lab newsletter at [duoneural.beehiiv.com](https://duoneural.beehiiv.com) for model drops before they go anywhere else.*