🚀 Echo-Mini (22M Parameters - F16 GGUF)

Echo-Mini is an ultra-lightweight, highly optimized micro-transformer model designed explicitly for low-power edge computing, local-first environments, and embedded system integration.

Unlike massive cloud-hosted LLMs, Echo-Mini packs its entire vocabulary, tokenizer, and attention mechanisms into a portable ~44MB footprint, making it a perfect foundation for private, zero-latency on-device text tasks.

✨ Key Features

• Zero Cloud Dependency: Runs 100% locally on standard consumer devices, mobile processors, and edge systems.
• Extreme Performance: Achieves ultra-high inference speeds (300+ tokens/second) entirely on consumer CPUs without needing an active GPU.
• Pristine Precision: Compiled in unquantized Float16 (F16) to prevent the common formatting collapse, word-smashing, and attention loops frequently found in microscopic quantized variants.
• Self-Contained Architecture: The GGUF container packages all architectural metadata and tokenizer configurations into a single, unified binary.

---

🧠 System Prompt Modes (The "Three Brains")

Echo-Mini switches its internal logic based on the System tag provided in the prompt structure. To achieve the best inference quality, define the processing mode explicitly prior to user inputs.

1. [CHAT] or [STORY] Mode

Optimized for general conversation, textual interactions, or narrative generation.

System: [CHAT]
User: Write a story about a girl cleaning up her toys.
Assistant:

2. [CODE] Mode

Triggers syntax-focused generation logic. Highly effective for simple programmatic formatting, loops, and script execution structures.

System: [CODE]
User: Write a python while loop to count to 10.
Assistant:

3. [FACT] or [RAG] Mode

Designed for context-grounded text extraction (Retrieval-Augmented Generation). Use this mode when piping external files, telemetry logs, or hardware documentation directly into the context window.

System: [FACT]
Context: The vehicle requires 205/55 R19 tires for optimal performance.
User: What size tires do I need?
Assistant:

⚠️ CRITICAL TOKENIZER WARNING: Ensure your prompt structure ends exactly on the colon (Assistant:) with no trailing space. If a physical space is left after the colon, the sub-word tokenizer will misalign, leading to omitted word spaces or combined words.

---

💻 Quickstart Implementation (Node.js / TypeScript)

You can run this model locally using node-llama-cpp. For optimal streaming results, utilize a sliding-window text decoder to cleanly reconstruct trailing word spaces during active inference.

import {LlamaModel, LlamaContext, LlamaSequence} from "node-llama-cpp";
import path from "path";

const model = new LlamaModel({
    modelPath: path.join(__dirname, "model-f16.gguf")
});

const context = new LlamaContext({model});
const sequence = new LlamaSequence({context});

// Step 1: Format prompt strictly without a trailing space. Choose your Mode!
const prompt = `System: [CODE]\nUser: Write a python print statement.\nAssistant:`;
const tokens = model.tokenize(prompt);

// Step 2: Inject BOS token if missing from sequence start
const finalTokens = tokens[0] === model.tokens.bos ? tokens : [model.tokens.bos, ...tokens];

let responseTokens: number[] = [];
let printedLength = 0;

console.log("Assistant stream started:\n");

for await (const token of sequence.evaluate(finalTokens, { 
    temperature: 0.7,  
    topP: 0.95,        
    topK: 50,          
    repeatPenalty: false // Retain natural structural text pacing
})) {
    if (token === model.tokens.eos) break;
    responseTokens.push(token);

    // Dynamic window decoding prevents token boundary space stripping
    const fullText = model.detokenize(responseTokens);
    const textChunk = fullText.slice(printedLength);
    printedLength = fullText.length;
    
    process.stdout.write(textChunk); 
}

---

🎯 Intended Use Cases

• Embedded Software & Robotics: Native voice/text command parsing on low-spec hardware setups (e.g., Raspberry Pi controllers, microcontrollers, offline robotics).
• On-Device Private Assistants: Powering custom local input tools (such as privacy-focused Android IME keyboards) requiring absolute data isolation.
• Micro-RAG Architectures: Querying offline system manual files or parsing real-time configuration contexts directly at the edge.

📄 License

This model and its compiled weights are open-sourced under the Apache 2.0 License. You are free to modify, distribute, and embed this architecture within proprietary and commercial products.