README.md

metadata

title: JackAILocal
emoji: 🔒
colorFrom: green
colorTo: gray
sdk: gradio
sdk_version: 6.18.0
python_version: '3.13'
app_file: app.py
pinned: true
license: other
short_description: Private AI that runs 100% offline — no cloud, ever.
hf_username: jackboy70
hf_oauth: true
hf_oauth_scopes:
  - inference-api
models:
  - Qwen/Qwen3.5-4B
  - Qwen/Qwen3.6-27B
  - google/gemma-4-12B-it
tags:
  - gradio
  - build-small-hackathon
  - track:backyard
  - sponsor:openai
  - sponsor:modal
  - achievement:offgrid
  - achievement:offbrand
  - achievement:llama
  - achievement:fieldnotes
  - backyard ai
  - backyard-ai
  - off the grid
  - off-the-grid
  - off brand
  - off-brand
  - llama champion
  - llama-champion
  - llama.cpp
  - tiny titan
  - tiny-titan
  - best agent
  - best-agent
  - best demo
  - best-demo
  - bonus quest champion
  - bonus-quest-champion
  - community choice
  - community-choice
  - modal
  - best use of modal
  - best-use-of-modal
  - field notes
  - field-notes
  - zerogpu
  - local
  - offline
  - privacy

🔒 JackAILocal — your AI, sealed in a box

Private AI that runs 100% offline. No cloud. No account. No data leaving the machine — ever.

JackAILocal turns any laptop, USB stick, external SSD, or LAN box into a complete private AI workspace: chat, voice, vision, and documents — all running on small open models on the device itself. This Space is the one-click builder that configures and ships that offline workspace for you. The AI you build never phones home.

🤗 Hugging Face Build Small Hackathon · Track: Backyard AI · Every model is ≤32B and the default runtime ships on a 4B model.

✅ Tested on Windows and the hosted Cloud Space only. The macOS, Linux, USB and SSD targets are implemented but not yet independently verified.

---

🎬 See it work

▶️ Demo video	https://youtu.be/OON9hfPGqqk — real laptop, Wi-Fi physically off, answering questions, transcribing voice, and reading an image with zero network.
🐦 Social post	https://x.com/JacquesGariepy/status/2066340944329224593?s=20
📝 Build report (field notes)	submission/FIELD_NOTES.md

---

⏱️ For judges: evaluate in 60 seconds

1. Open the Builder tab above. Pick a use case (e.g. "Private document assistant") and a target (e.g. Windows ZIP).
2. Click Sign in with Hugging Face → Prepare / publish. The AI configuration agent (Gemma 4 12B, served on Modal) reviews your hardware, picks a compliant small model, and returns a validated build plan.
3. Open the audit JSON — every decision shows the full agent trace: the exact prompt sent and the raw model output. Nothing is faked; when no model is configured the panel says so instead of inventing an answer. Watch the build logs. Once the build finishes, the download link for your custom ZIP package will appear at the very bottom of the page. Download it and unzip. it on your target machine.
4. Watch the demo video to see the output of that build — a sealed AI running on a real machine with the network cut.

That is the whole pitch: this Space configures and ships the AI; the video proves it runs with no cloud.

---

🌳 The problem (Backyard AI)

Most "local AI" tools are a thin wrapper around an API key. The moment the Wi-Fi drops — or a clinic, law office, field site, or privacy-conscious household refuses to send data to someone else's server — they stop working.

People who actually need AI off the grid have no good option:

• a nurse in a rural clinic who can't upload patient notes to a cloud,
• a shop owner who wants a document assistant but not a subscription that reads their books,
• a parent who wants a homework helper that works on the cabin trip with no signal,
• a regulated SMB that legally cannot let data leave the building.

JackAILocal is built for them. You configure it here, ship it to a USB/SSD/installer/LAN box, and from that point on it is completely self-contained. The runtime path is simply:

WebUI → jackailocald (Rust) → local Ollama / llama.cpp model

There is no cloud inference fallback. A capability is shown as unavailable when its local binary or model is missing — it is never quietly replaced by a remote call.

---

🏆 Why this wins (rubric alignment)

Badge / award	How JackAILocal earns it
🌲 Track: Backyard AI	A real, polished tool that solves a real daily problem: private AI for people and SMBs who can't or won't use the cloud.
🔌 Off the Grid	The shipped runtime does 100% local inference. The demo video is recorded with the network physically disconnected.
🐤 Tiny Titan (≤4B)	The default runtime chat model is Qwen3.5 4B — the everyday experience runs on a genuinely tiny model.
🎨 Off Brand	A custom, themed builder console plus a fully hand-built offline WebUI (not default Gradio) shipped with the product.
🤖 Best Agent	The configuration step is a real multi-step decision engine: it can ASK_USER, ASK_HUMAN_REVIEW, REJECT, or emit a build action — with a policy gate that has final authority — and every step exposes its agent trace.
🦙 llama.cpp	Ships an optional llama.cpp OpenAI-compatible server path alongside Ollama for the runtime.
☁️ Best Use of Modal	The hosted configuration agent (Gemma 4 12B IT) is served from a Modal vLLM endpoint.
📓 Field Notes	A full build report is included — see submission/FIELD_NOTES.md.
🌟 Bonus Quest Champion	One submission, stacking the most bonus criteria at once.

---

🤖 The configuration agent (Best Agent)

The Builder does not treat the model as a chatbot. It treats it as an internal decision engine that must return schema-validated JSON. The UI sends the selected use case and target, the hardware profile, the available backend/model plan, client constraints, and any previous answers. The model must then return one of:

• ASK_USER — with concrete questions when critical info is missing,
• ASK_HUMAN_REVIEW — when a human operator should approve,
• REJECT — when the request is unsafe or non-compliant,
• a build action — the validated packaging plan.

A deterministic policy gate still has final authority: it blocks USB builds without a real target, backend mismatches, missing secrets, and unsafe manifest patches — even if the model says otherwise. The remote agent can only modify the packaging manifest; the shipped runtime stays local and offline once models are preloaded.

Crucially, no fake decision is ever generated. If no agent endpoint is configured, the panel reports the missing secrets explicitly rather than fabricating an "AI decision."

---

🧠 Small-model stack (everything ≤32B)

Role	Model	Size	Where
Default runtime chat	Qwen/Qwen3.5-4B	4B	On device (Ollama)
Power profile (24GB VRAM)	Qwen/Qwen3.6-27B	27B	On device (Ollama)
Configuration agent	google/gemma-4-12B-it	12B	Modal vLLM (build-time only)
Speech-to-text	whisper.cpp (ggml-base)	—	On device
Text-to-speech	Piper (en_US-libritts_r-medium, CC BY 4.0)	—	On device
Vision / OCR (SCOUT)	any installed vision Ollama model	≤32B	On device

No model above the 32B cap is ever offered. The default demo deliberately runs on a 4B model so it works on an ordinary laptop without a 24GB GPU.

---

🧩 What the shipped product includes

• Local chat through installed Ollama models, with an optional llama.cpp server path
• Persistent conversation threads (filter, rename, delete)
• SCOUT image analysis and text extraction via a local vision model
• Offline speech-to-text (whisper.cpp) and text-to-speech (Piper)
• Local documents and a bilingual Field Manual
• AES-256-GCM encrypted import/export packs for threads, documents, and safe settings
• Opt-in Phone Access on the local network with pairing token + QR code
• Hardware status, model availability, benchmark, and a privacy-safe support ZIP
• English and French WebUI
• Ed25519 offline licensing and signed, rollback-capable offline updates

---

🚀 Run locally

pip install -r requirements.txt
python app.py
# open http://127.0.0.1:7860

The hosted Space only builds and publishes packages (Windows / macOS / Linux-Docker ZIPs + a standalone PC analyzer). It never pretends to run a client runtime for you — when you ship the package and launch it on a real machine, that is where the offline AI lives.

Build targets (run locally on Windows/macOS):

BUILD-LOCAL.cmd      BUILD-USB.cmd      BUILD-SSD.cmd

Each builder validates the payload, installs the selected Ollama model(s), installs Voice assets, writes a SHA-256 manifest, and refuses to report success if required runtime or legal assets are missing.

---

🛡️ Security boundaries

• Loopback bind by default; pairing token required for any non-loopback access
• No shell tools exposed to the model
• No private update key in exported packages
• AES transfer-pack passphrases are never stored
• Support ZIP excludes conversations and documents
• SHA-256 package manifest + signed-manifest hooks

JackAILocal does not claim protection from an already-compromised host OS.

---

✅ Verification

cargo check
cargo test --no-run
python -m py_compile app.py saas/gradio/app.py
node --check webui/app-v15.js
python qa/static_audit.py

See submission/FIELD_NOTES.md for the build story. The full source, the Rust runtime, and the package builders live in the project's GitHub repository.

---

Built for the 🤗 Build Small Hackathon. Small models. Big privacy. Off the grid.

Post it - Social-media post : https://x.com/JacquesGariepy/status/2066340944329224593?s=20

Video demo : https://youtu.be/OON9hfPGqqk

Gradio app : https://huggingface.co/spaces/build-small-hackathon/jackailocal