| --- |
| title: JackAILocal |
| emoji: 🔒 |
| colorFrom: green |
| colorTo: gray |
| sdk: gradio |
| sdk_version: 6.18.0 |
| python_version: '3.13' |
| app_file: app.py |
| pinned: true |
| license: other |
| short_description: Private AI that runs 100% offline — no cloud, ever. |
| hf_username: jackboy70 |
| hf_oauth: true |
| hf_oauth_scopes: |
| - inference-api |
| models: |
| - Qwen/Qwen3.5-4B |
| - Qwen/Qwen3.6-27B |
| - google/gemma-4-12B-it |
| tags: |
| - gradio |
| - build-small-hackathon |
| - track:backyard |
| - sponsor:openai |
| - sponsor:modal |
| - achievement:offgrid |
| - achievement:offbrand |
| - achievement:llama |
| - achievement:fieldnotes |
| - backyard ai |
| - backyard-ai |
| - off the grid |
| - off-the-grid |
| - off brand |
| - off-brand |
| - llama champion |
| - llama-champion |
| - llama.cpp |
| - tiny titan |
| - tiny-titan |
| - best agent |
| - best-agent |
| - best demo |
| - best-demo |
| - bonus quest champion |
| - bonus-quest-champion |
| - community choice |
| - community-choice |
| - modal |
| - best use of modal |
| - best-use-of-modal |
| - field notes |
| - field-notes |
| - zerogpu |
| - local |
| - offline |
| - privacy |
|
|
| --- |
| |
| # 🔒 JackAILocal — your AI, sealed in a box |
|
|
| **Private AI that runs 100% offline. No cloud. No account. No data leaving the machine — ever.** |
|
|
| JackAILocal turns any laptop, USB stick, external SSD, or LAN box into a complete private AI workspace: chat, voice, vision, and documents — all running on small open models on the device itself. This Space is the **one-click builder** that configures and ships that offline workspace for you. The AI you build never phones home. |
|
|
| > 🤗 **Hugging Face Build Small Hackathon** · Track: **Backyard AI** · Every model is ≤32B and the default runtime ships on a **4B** model. |
|
|
| > ✅ **Tested on Windows and the hosted Cloud Space only.** The macOS, Linux, USB and SSD targets are implemented but not yet independently verified. |
|
|
| --- |
|
|
| ## 🎬 See it work |
|
|
| | | | |
| |---|---| |
| | **▶️ Demo video** | `https://youtu.be/OON9hfPGqqk` — real laptop, **Wi-Fi physically off**, answering questions, transcribing voice, and reading an image with zero network. | |
| | **🐦 Social post** | `https://x.com/JacquesGariepy/status/2066340944329224593?s=20` | |
| | **📝 Build report (field notes)** | [`submission/FIELD_NOTES.md`](submission/FIELD_NOTES.md) | |
|
|
| --- |
|
|
| ## ⏱️ For judges: evaluate in 60 seconds |
|
|
| 1. **Open the Builder tab above.** Pick a use case (e.g. *"Private document assistant"*) and a target (e.g. *Windows ZIP*). |
| 2. Click **Sign in with Hugging Face** → **Prepare / publish**. The **AI configuration agent** (Gemma 4 12B, served on **Modal**) reviews your hardware, picks a compliant small model, and returns a validated build plan. |
| 3. Open the **audit JSON** — every decision shows the **full agent trace**: the exact prompt sent and the raw model output. Nothing is faked; when no model is configured the panel says so instead of inventing an answer. Watch the build logs. Once the build finishes, **the download link for your custom ZIP package will appear at the very bottom of the page**. Download it and unzip. it on your target machine. |
| 4. Watch the **demo video** to see the *output* of that build — a sealed AI running on a real machine with the network cut. |
|
|
| That is the whole pitch: this Space **configures and ships** the AI; the **video proves** it runs with no cloud. |
|
|
| --- |
|
|
| ## 🌳 The problem (Backyard AI) |
|
|
| Most "local AI" tools are a thin wrapper around an API key. The moment the Wi-Fi drops — or a clinic, law office, field site, or privacy-conscious household refuses to send data to someone else's server — they stop working. |
|
|
| People who actually need AI off the grid have no good option: |
|
|
| - a **nurse** in a rural clinic who can't upload patient notes to a cloud, |
| - a **shop owner** who wants a document assistant but not a subscription that reads their books, |
| - a **parent** who wants a homework helper that works on the cabin trip with no signal, |
| - a **regulated SMB** that legally cannot let data leave the building. |
|
|
| JackAILocal is built for them. You configure it here, ship it to a USB/SSD/installer/LAN box, and from that point on it is **completely self-contained**. The runtime path is simply: |
|
|
| ```text |
| WebUI → jackailocald (Rust) → local Ollama / llama.cpp model |
| ``` |
|
|
| There is **no cloud inference fallback**. A capability is shown as *unavailable* when its local binary or model is missing — it is never quietly replaced by a remote call. |
|
|
| --- |
|
|
| ## 🏆 Why this wins (rubric alignment) |
|
|
| | Badge / award | How JackAILocal earns it | |
| |---|---| |
| | 🌲 **Track: Backyard AI** | A real, polished tool that solves a real daily problem: private AI for people and SMBs who can't or won't use the cloud. | |
| | 🔌 **Off the Grid** | The shipped runtime does **100% local inference**. The demo video is recorded with the network physically disconnected. | |
| | 🐤 **Tiny Titan** (≤4B) | The default runtime chat model is **Qwen3.5 4B** — the everyday experience runs on a genuinely tiny model. | |
| | 🎨 **Off Brand** | A custom, themed builder console plus a fully hand-built offline **WebUI** (not default Gradio) shipped with the product. | |
| | 🤖 **Best Agent** | The configuration step is a real **multi-step decision engine**: it can `ASK_USER`, `ASK_HUMAN_REVIEW`, `REJECT`, or emit a build action — with a policy gate that has final authority — and every step exposes its agent trace. | |
| | 🦙 **llama.cpp** | Ships an optional `llama.cpp` OpenAI-compatible server path alongside Ollama for the runtime. | |
| | ☁️ **Best Use of Modal** | The hosted configuration agent (Gemma 4 12B IT) is served from a **Modal vLLM** endpoint. | |
| | 📓 **Field Notes** | A full build report is included — see [`submission/FIELD_NOTES.md`](submission/FIELD_NOTES.md). | |
| | 🌟 **Bonus Quest Champion** | One submission, stacking the most bonus criteria at once. | |
|
|
| --- |
|
|
| ## 🤖 The configuration agent (Best Agent) |
|
|
| The Builder does **not** treat the model as a chatbot. It treats it as an **internal decision engine** that must return schema-validated JSON. The UI sends the selected use case and target, the hardware profile, the available backend/model plan, client constraints, and any previous answers. The model must then return one of: |
|
|
| - `ASK_USER` — with concrete questions when critical info is missing, |
| - `ASK_HUMAN_REVIEW` — when a human operator should approve, |
| - `REJECT` — when the request is unsafe or non-compliant, |
| - a **build action** — the validated packaging plan. |
|
|
| A deterministic **policy gate** still has final authority: it blocks USB builds without a real target, backend mismatches, missing secrets, and unsafe manifest patches — even if the model says otherwise. The remote agent can only modify the **packaging manifest**; the shipped runtime stays local and offline once models are preloaded. |
|
|
| Crucially, **no fake decision is ever generated.** If no agent endpoint is configured, the panel reports the missing secrets explicitly rather than fabricating an "AI decision." |
|
|
| --- |
|
|
| ## 🧠 Small-model stack (everything ≤32B) |
|
|
| | Role | Model | Size | Where | |
| |---|---|---|---| |
| | Default runtime chat | `Qwen/Qwen3.5-4B` | 4B | On device (Ollama) | |
| | Power profile (24GB VRAM) | `Qwen/Qwen3.6-27B` | 27B | On device (Ollama) | |
| | Configuration agent | `google/gemma-4-12B-it` | 12B | Modal vLLM (build-time only) | |
| | Speech-to-text | `whisper.cpp` (ggml-base) | — | On device | |
| | Text-to-speech | Piper (`en_US-libritts_r-medium`, CC BY 4.0) | — | On device | |
| | Vision / OCR (SCOUT) | any installed vision Ollama model | ≤32B | On device | |
|
|
| No model above the 32B cap is ever offered. The default demo deliberately runs on a 4B model so it works on an ordinary laptop without a 24GB GPU. |
|
|
| --- |
|
|
| ## 🧩 What the shipped product includes |
|
|
| - Local chat through installed Ollama models, with an optional `llama.cpp` server path |
| - Persistent conversation threads (filter, rename, delete) |
| - **SCOUT** image analysis and text extraction via a local vision model |
| - Offline **speech-to-text** (whisper.cpp) and **text-to-speech** (Piper) |
| - Local documents and a bilingual **Field Manual** |
| - **AES-256-GCM** encrypted import/export packs for threads, documents, and safe settings |
| - Opt-in **Phone Access** on the local network with pairing token + QR code |
| - Hardware status, model availability, benchmark, and a privacy-safe support ZIP |
| - **English and French** WebUI |
| - Ed25519 offline licensing and signed, rollback-capable offline updates |
|
|
| --- |
|
|
| ## 🚀 Run locally |
|
|
| ```bash |
| pip install -r requirements.txt |
| python app.py |
| # open http://127.0.0.1:7860 |
| ``` |
|
|
| The hosted Space **only builds and publishes packages** (Windows / macOS / Linux-Docker ZIPs + a standalone PC analyzer). It never pretends to run a client runtime for you — when you ship the package and launch it on a real machine, *that* is where the offline AI lives. |
|
|
| Build targets (run locally on Windows/macOS): |
|
|
| ```cmd |
| BUILD-LOCAL.cmd BUILD-USB.cmd BUILD-SSD.cmd |
| ``` |
|
|
| Each builder validates the payload, installs the selected Ollama model(s), installs Voice assets, writes a SHA-256 manifest, and **refuses to report success** if required runtime or legal assets are missing. |
|
|
| --- |
|
|
| ## 🛡️ Security boundaries |
|
|
| - Loopback bind by default; pairing token required for any non-loopback access |
| - No shell tools exposed to the model |
| - No private update key in exported packages |
| - AES transfer-pack passphrases are never stored |
| - Support ZIP excludes conversations and documents |
| - SHA-256 package manifest + signed-manifest hooks |
|
|
| JackAILocal does not claim protection from an already-compromised host OS. |
|
|
| --- |
|
|
| ## ✅ Verification |
|
|
| ```powershell |
| cargo check |
| cargo test --no-run |
| python -m py_compile app.py saas/gradio/app.py |
| node --check webui/app-v15.js |
| python qa/static_audit.py |
| ``` |
|
|
| See [`submission/FIELD_NOTES.md`](submission/FIELD_NOTES.md) for the build story. The full source, the Rust runtime, and the package builders live in the [project's GitHub repository](https://github.com/JacquesGariepy/jackailocal). |
|
|
| --- |
|
|
| *Built for the 🤗 Build Small Hackathon. Small models. Big privacy. Off the grid.* |
|
|
| Post it - Social-media post : https://x.com/JacquesGariepy/status/2066340944329224593?s=20 |
|
|
| Video demo : https://youtu.be/OON9hfPGqqk |
|
|
| Gradio app : https://huggingface.co/spaces/build-small-hackathon/jackailocal |
