--- title: JackAILocal emoji: ๐Ÿ”’ colorFrom: green colorTo: gray sdk: gradio sdk_version: 6.18.0 python_version: '3.13' app_file: app.py pinned: true license: other short_description: Private AI that runs 100% offline โ€” no cloud, ever. hf_username: jackboy70 hf_oauth: true hf_oauth_scopes: - inference-api models: - Qwen/Qwen3.5-4B - Qwen/Qwen3.6-27B - google/gemma-4-12B-it tags: - gradio - build-small-hackathon - track:backyard - sponsor:openai - sponsor:modal - achievement:offgrid - achievement:offbrand - achievement:llama - achievement:fieldnotes - backyard ai - backyard-ai - off the grid - off-the-grid - off brand - off-brand - llama champion - llama-champion - llama.cpp - tiny titan - tiny-titan - best agent - best-agent - best demo - best-demo - bonus quest champion - bonus-quest-champion - community choice - community-choice - modal - best use of modal - best-use-of-modal - field notes - field-notes - zerogpu - local - offline - privacy --- # ๐Ÿ”’ JackAILocal โ€” your AI, sealed in a box **Private AI that runs 100% offline. No cloud. No account. No data leaving the machine โ€” ever.** JackAILocal turns any laptop, USB stick, external SSD, or LAN box into a complete private AI workspace: chat, voice, vision, and documents โ€” all running on small open models on the device itself. This Space is the **one-click builder** that configures and ships that offline workspace for you. The AI you build never phones home. > ๐Ÿค— **Hugging Face Build Small Hackathon** ยท Track: **Backyard AI** ยท Every model is โ‰ค32B and the default runtime ships on a **4B** model. > โœ… **Tested on Windows and the hosted Cloud Space only.** The macOS, Linux, USB and SSD targets are implemented but not yet independently verified. --- ## ๐ŸŽฌ See it work | | | |---|---| | **โ–ถ๏ธ Demo video** | `https://youtu.be/OON9hfPGqqk` โ€” real laptop, **Wi-Fi physically off**, answering questions, transcribing voice, and reading an image with zero network. | | **๐Ÿฆ Social post** | `https://x.com/JacquesGariepy/status/2066340944329224593?s=20` | | **๐Ÿ“ Build report (field notes)** | [`submission/FIELD_NOTES.md`](submission/FIELD_NOTES.md) | --- ## โฑ๏ธ For judges: evaluate in 60 seconds 1. **Open the Builder tab above.** Pick a use case (e.g. *"Private document assistant"*) and a target (e.g. *Windows ZIP*). 2. Click **Sign in with Hugging Face** โ†’ **Prepare / publish**. The **AI configuration agent** (Gemma 4 12B, served on **Modal**) reviews your hardware, picks a compliant small model, and returns a validated build plan. 3. Open the **audit JSON** โ€” every decision shows the **full agent trace**: the exact prompt sent and the raw model output. Nothing is faked; when no model is configured the panel says so instead of inventing an answer. Watch the build logs. Once the build finishes, **the download link for your custom ZIP package will appear at the very bottom of the page**. Download it and unzip. it on your target machine. 4. Watch the **demo video** to see the *output* of that build โ€” a sealed AI running on a real machine with the network cut. That is the whole pitch: this Space **configures and ships** the AI; the **video proves** it runs with no cloud. --- ## ๐ŸŒณ The problem (Backyard AI) Most "local AI" tools are a thin wrapper around an API key. The moment the Wi-Fi drops โ€” or a clinic, law office, field site, or privacy-conscious household refuses to send data to someone else's server โ€” they stop working. People who actually need AI off the grid have no good option: - a **nurse** in a rural clinic who can't upload patient notes to a cloud, - a **shop owner** who wants a document assistant but not a subscription that reads their books, - a **parent** who wants a homework helper that works on the cabin trip with no signal, - a **regulated SMB** that legally cannot let data leave the building. JackAILocal is built for them. You configure it here, ship it to a USB/SSD/installer/LAN box, and from that point on it is **completely self-contained**. The runtime path is simply: ```text WebUI โ†’ jackailocald (Rust) โ†’ local Ollama / llama.cpp model ``` There is **no cloud inference fallback**. A capability is shown as *unavailable* when its local binary or model is missing โ€” it is never quietly replaced by a remote call. --- ## ๐Ÿ† Why this wins (rubric alignment) | Badge / award | How JackAILocal earns it | |---|---| | ๐ŸŒฒ **Track: Backyard AI** | A real, polished tool that solves a real daily problem: private AI for people and SMBs who can't or won't use the cloud. | | ๐Ÿ”Œ **Off the Grid** | The shipped runtime does **100% local inference**. The demo video is recorded with the network physically disconnected. | | ๐Ÿค **Tiny Titan** (โ‰ค4B) | The default runtime chat model is **Qwen3.5 4B** โ€” the everyday experience runs on a genuinely tiny model. | | ๐ŸŽจ **Off Brand** | A custom, themed builder console plus a fully hand-built offline **WebUI** (not default Gradio) shipped with the product. | | ๐Ÿค– **Best Agent** | The configuration step is a real **multi-step decision engine**: it can `ASK_USER`, `ASK_HUMAN_REVIEW`, `REJECT`, or emit a build action โ€” with a policy gate that has final authority โ€” and every step exposes its agent trace. | | ๐Ÿฆ™ **llama.cpp** | Ships an optional `llama.cpp` OpenAI-compatible server path alongside Ollama for the runtime. | | โ˜๏ธ **Best Use of Modal** | The hosted configuration agent (Gemma 4 12B IT) is served from a **Modal vLLM** endpoint. | | ๐Ÿ““ **Field Notes** | A full build report is included โ€” see [`submission/FIELD_NOTES.md`](submission/FIELD_NOTES.md). | | ๐ŸŒŸ **Bonus Quest Champion** | One submission, stacking the most bonus criteria at once. | --- ## ๐Ÿค– The configuration agent (Best Agent) The Builder does **not** treat the model as a chatbot. It treats it as an **internal decision engine** that must return schema-validated JSON. The UI sends the selected use case and target, the hardware profile, the available backend/model plan, client constraints, and any previous answers. The model must then return one of: - `ASK_USER` โ€” with concrete questions when critical info is missing, - `ASK_HUMAN_REVIEW` โ€” when a human operator should approve, - `REJECT` โ€” when the request is unsafe or non-compliant, - a **build action** โ€” the validated packaging plan. A deterministic **policy gate** still has final authority: it blocks USB builds without a real target, backend mismatches, missing secrets, and unsafe manifest patches โ€” even if the model says otherwise. The remote agent can only modify the **packaging manifest**; the shipped runtime stays local and offline once models are preloaded. Crucially, **no fake decision is ever generated.** If no agent endpoint is configured, the panel reports the missing secrets explicitly rather than fabricating an "AI decision." --- ## ๐Ÿง  Small-model stack (everything โ‰ค32B) | Role | Model | Size | Where | |---|---|---|---| | Default runtime chat | `Qwen/Qwen3.5-4B` | 4B | On device (Ollama) | | Power profile (24GB VRAM) | `Qwen/Qwen3.6-27B` | 27B | On device (Ollama) | | Configuration agent | `google/gemma-4-12B-it` | 12B | Modal vLLM (build-time only) | | Speech-to-text | `whisper.cpp` (ggml-base) | โ€” | On device | | Text-to-speech | Piper (`en_US-libritts_r-medium`, CC BY 4.0) | โ€” | On device | | Vision / OCR (SCOUT) | any installed vision Ollama model | โ‰ค32B | On device | No model above the 32B cap is ever offered. The default demo deliberately runs on a 4B model so it works on an ordinary laptop without a 24GB GPU. --- ## ๐Ÿงฉ What the shipped product includes - Local chat through installed Ollama models, with an optional `llama.cpp` server path - Persistent conversation threads (filter, rename, delete) - **SCOUT** image analysis and text extraction via a local vision model - Offline **speech-to-text** (whisper.cpp) and **text-to-speech** (Piper) - Local documents and a bilingual **Field Manual** - **AES-256-GCM** encrypted import/export packs for threads, documents, and safe settings - Opt-in **Phone Access** on the local network with pairing token + QR code - Hardware status, model availability, benchmark, and a privacy-safe support ZIP - **English and French** WebUI - Ed25519 offline licensing and signed, rollback-capable offline updates --- ## ๐Ÿš€ Run locally ```bash pip install -r requirements.txt python app.py # open http://127.0.0.1:7860 ``` The hosted Space **only builds and publishes packages** (Windows / macOS / Linux-Docker ZIPs + a standalone PC analyzer). It never pretends to run a client runtime for you โ€” when you ship the package and launch it on a real machine, *that* is where the offline AI lives. Build targets (run locally on Windows/macOS): ```cmd BUILD-LOCAL.cmd BUILD-USB.cmd BUILD-SSD.cmd ``` Each builder validates the payload, installs the selected Ollama model(s), installs Voice assets, writes a SHA-256 manifest, and **refuses to report success** if required runtime or legal assets are missing. --- ## ๐Ÿ›ก๏ธ Security boundaries - Loopback bind by default; pairing token required for any non-loopback access - No shell tools exposed to the model - No private update key in exported packages - AES transfer-pack passphrases are never stored - Support ZIP excludes conversations and documents - SHA-256 package manifest + signed-manifest hooks JackAILocal does not claim protection from an already-compromised host OS. --- ## โœ… Verification ```powershell cargo check cargo test --no-run python -m py_compile app.py saas/gradio/app.py node --check webui/app-v15.js python qa/static_audit.py ``` See [`submission/FIELD_NOTES.md`](submission/FIELD_NOTES.md) for the build story. The full source, the Rust runtime, and the package builders live in the [project's GitHub repository](https://github.com/JacquesGariepy/jackailocal). --- *Built for the ๐Ÿค— Build Small Hackathon. Small models. Big privacy. Off the grid.* Post it - Social-media post : https://x.com/JacquesGariepy/status/2066340944329224593?s=20 Video demo : https://youtu.be/OON9hfPGqqk Gradio app : https://huggingface.co/spaces/build-small-hackathon/jackailocal