Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -18,6 +18,26 @@ iPhone (or desktop) photo → local **Gemma 4 12B-it** vision extract → SQLite
|
|
| 18 |
GitHub: [`drowzeys/keys-Auto-Receipts-Studio`](https://github.com/drowzeys/keys-Auto-Receipts-Studio)
|
| 19 |
Hugging Face: [`drowzeys/keys-Auto-Receipts-Studio`](https://huggingface.co/drowzeys/keys-Auto-Receipts-Studio)
|
| 20 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
## Screenshots (v1.0 alpha)
|
| 22 |
|
| 23 |
iPhone Safari — take a photo, then completion notice:
|
|
@@ -40,34 +60,57 @@ Inbox drop zone, catalog SKUs, and backend health:
|
|
| 40 |
<img src="docs/screenshots/04-settings.png" alt="Settings backends" width="420"/>
|
| 41 |
</p>
|
| 42 |
|
| 43 |
-
##
|
| 44 |
|
| 45 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
|
| 47 |
```bash
|
| 48 |
-
git clone https://github.com/drowzeys/keys-Auto-Receipts-Studio.git
|
| 49 |
-
cd keys-Auto-Receipts-Studio
|
| 50 |
bash oneshot.sh
|
| 51 |
```
|
| 52 |
|
|
|
|
|
|
|
| 53 |
When it prints READY:
|
| 54 |
|
| 55 |
| | |
|
| 56 |
|---|---|
|
| 57 |
| Review | http://127.0.0.1:7860 |
|
| 58 |
-
| iPhone (same Wi‑Fi
|
|
|
|
| 59 |
|
| 60 |
-
|
| 61 |
|
| 62 |
-
|
|
|
|
|
|
|
| 63 |
|
| 64 |
-
|
| 65 |
|
| 66 |
-
|
| 67 |
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
|
| 72 |
```
|
| 73 |
RECEIPT_LLM_BASE_URL=http://<spark-lan-ip>:8080/v1
|
|
@@ -77,171 +120,126 @@ RECEIPT_EMBED_MODEL=google/gemma-4-12B-it
|
|
| 77 |
RECEIPT_EMBED_DIM=3840
|
| 78 |
```
|
| 79 |
|
| 80 |
-
|
| 81 |
|
| 82 |
-
|
| 83 |
|
| 84 |
-
|
| 85 |
|
| 86 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 87 |
|
| 88 |
```bash
|
| 89 |
-
git clone https://github.com/drowzeys/keys-Auto-Receipts-Studio.git
|
| 90 |
-
cd keys-Auto-Receipts-Studio
|
| 91 |
python3 -m venv .venv
|
| 92 |
.venv/bin/pip install -e ".[dev]"
|
| 93 |
cp .env.example .env
|
| 94 |
-
#
|
| 95 |
bash scripts/install-launcher.sh
|
| 96 |
```
|
| 97 |
|
| 98 |
-
Double-click **Desktop → Receipt Studio.command** (first time: right-click → **Open**).
|
| 99 |
|
| 100 |
-
|
| 101 |
|
| 102 |
-
|
| 103 |
|
| 104 |
-
|
|
|
|
|
|
|
|
|
|
| 105 |
|
| 106 |
-
|
| 107 |
|
| 108 |
-
|
| 109 |
|
| 110 |
-
|
| 111 |
-
|
| 112 |
-
**No.** Lamp is 8-core ARM64 with **6 GB RAM**. Gemma 4 12B Unified is a 12B dense omni model (`Gemma4UnifiedForConditionalGeneration`, hidden size **3840**). Weights alone will not boot beside Autonomous OS + HAL.
|
| 113 |
-
|
| 114 |
-
| Piece | Fits on Lamp? | Fits on GPU box? |
|
| 115 |
-
|---|---|---|
|
| 116 |
-
| `skills/keys-receipt-scanner/` (this skill) | yes — built-in skill format | yes |
|
| 117 |
-
| HAL `GET /camera/snapshot` | yes | n/a |
|
| 118 |
-
| SQLite + sqlite-vec + HTTP client | yes | yes |
|
| 119 |
-
| **Gemma 4 12B Unified weights** | **no** | yes (vLLM, util **0.15**) |
|
| 120 |
-
| Qwen3.8-27B ADay777 VLM | no | yes |
|
| 121 |
-
| Nemotron-3-Embed-1B | no | yes |
|
| 122 |
-
|
| 123 |
-
The skill is **built-in to Autonomous OS** (markdown + `skill.json` `capabilities: ["vision"]`). The brain is remote. Intern has no camera, so this skill will not install there.
|
| 124 |
-
|
| 125 |
-
## Built-in skill (Lamp)
|
| 126 |
|
| 127 |
```
|
| 128 |
skills/keys-receipt-scanner/
|
| 129 |
-
SKILL.md
|
| 130 |
-
skill.json
|
| 131 |
scripts/scan.py
|
|
|
|
| 132 |
```
|
| 133 |
|
| 134 |
-
Install
|
| 135 |
|
| 136 |
```bash
|
| 137 |
make push-skill SKILL=./skills/keys-receipt-scanner TARGET=pi@lamp-xxxx.local
|
| 138 |
```
|
| 139 |
|
| 140 |
-
On the Lamp
|
| 141 |
|
| 142 |
```bash
|
| 143 |
-
export
|
| 144 |
-
export
|
| 145 |
-
export
|
| 146 |
-
export RECEIPT_EMBED_MODEL=google/gemma-4-12B-it
|
| 147 |
-
export RECEIPT_EMBED_DIM=3840
|
| 148 |
-
export RECEIPT_EMBED_BACKEND=omni
|
| 149 |
```
|
| 150 |
|
| 151 |
-
Say **“scan this receipt”**
|
| 152 |
|
| 153 |
-
|
| 154 |
|
| 155 |
-
|
| 156 |
|
| 157 |
-
|
| 158 |
-
|
| 159 |
-
```bash
|
| 160 |
-
bash scripts/serve-gemma.sh
|
| 161 |
-
# util 0.15, FP8, max-model-len 8192 (or: bash oneshot.sh)
|
| 162 |
-
```
|
| 163 |
-
|
| 164 |
-
If `/v1/embeddings` 404s on a generate-only runner, either:
|
| 165 |
-
|
| 166 |
-
- serve a pooling convert on another port and set `RECEIPT_EMBED_BASE_URL`, or
|
| 167 |
-
- set `RECEIPT_EMBED_BACKEND=nvidia`, `RECEIPT_EMBED_DIM=2048`, and run Nemotron-3-Embed-1B. **Never mix 3840 and 2048 in one DB.**
|
| 168 |
-
|
| 169 |
-
### Fallback: Qwen3.8-27B ADay777 (vision only)
|
| 170 |
-
|
| 171 |
-
```bash
|
| 172 |
-
export RECEIPT_LLM_BACKEND=nvidia
|
| 173 |
-
export RECEIPT_LLM_BASE_URL=http://127.0.0.1:8078/v1
|
| 174 |
-
export RECEIPT_LLM_MODEL=qwen38-nvfp4
|
| 175 |
-
# thinking off is sent automatically
|
| 176 |
-
```
|
| 177 |
|
| 178 |
-
|
| 179 |
|
| 180 |
-
|
| 181 |
|
| 182 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 183 |
|
| 184 |
-
|
| 185 |
-
|---|---|
|
| 186 |
-
| **Linux** | `bash scripts/install-launcher.sh` once → Desktop **Receipt Studio** |
|
| 187 |
-
| **macOS** | `bash scripts/install-launcher.sh` once → Desktop **Receipt Studio.command** (first time: right-click → Open) |
|
| 188 |
-
| **Windows** | Copy `scripts/start-ui.bat` to the Desktop (or double-click it in the repo). First run creates `.venv`. |
|
| 189 |
|
| 190 |
-
|
| 191 |
|
| 192 |
```bash
|
| 193 |
-
|
| 194 |
-
.
|
| 195 |
-
|
| 196 |
-
# Windows
|
| 197 |
-
scripts\start-ui.bat
|
| 198 |
```
|
| 199 |
|
| 200 |
-
|
| 201 |
-
|
| 202 |
-
On a **CUDA box** (this Spark), the same click also starts Gemma 4 12B if `:8080` is down:
|
| 203 |
-
|
| 204 |
-
- `--gpu-memory-utilization **0.15**` (~18.3 GiB of 121.7 GiB; never above **0.85**)
|
| 205 |
-
- **FP8** — BF16 weights are ~23GB and cannot fit in that pool
|
| 206 |
-
- `--max-model-len **8192**` (receipt photo is ~280 vision tokens + JSON)
|
| 207 |
-
|
| 208 |
-
Context at util 0.15 (after ~12.5GB FP8 weights):
|
| 209 |
|
| 210 |
| Estimate | Tokens |
|
| 211 |
|---|---|
|
| 212 |
-
| Conservative (
|
| 213 |
| Hybrid (8 full + 40× sliding-1024) | **~65k** |
|
| 214 |
| Model native window | 262,144 (not at 0.15) |
|
| 215 |
|
| 216 |
-
A receipt scan uses ~1–2k tokens. Raise `RECEIPT_VLLM_MAX_MODEL_LEN` only after
|
| 217 |
-
|
| 218 |
-
If Gemma is already running, the launcher leaves it alone. To apply the 15GB cap, stop the current `vllm` process, then click again.
|
| 219 |
|
| 220 |
-
```
|
| 221 |
-
# apply 15GB cap (stops the current unconstrained serve)
|
| 222 |
-
pkill -x vllm # only if you intend to restart it
|
| 223 |
-
./scripts/start-ui.sh
|
| 224 |
-
```
|
| 225 |
|
| 226 |
-
|
| 227 |
-
|
| 228 |
-
```
|
| 229 |
-
RECEIPT_LLM_BASE_URL=http://<spark-lan-ip>:8080/v1
|
| 230 |
-
RECEIPT_EMBED_BASE_URL=http://<spark-lan-ip>:8080/v1
|
| 231 |
-
```
|
| 232 |
|
| 233 |
-
##
|
| 234 |
|
| 235 |
```bash
|
| 236 |
-
|
| 237 |
pip install -e ".[dev]"
|
| 238 |
cp .env.example .env
|
| 239 |
-
|
|
|
|
| 240 |
```
|
| 241 |
|
| 242 |
-
|
| 243 |
-
- Phone on LAN: `RECEIPT_UI_SHARE_LAN=true` → `http://<lan-ip>:7860/phone`
|
| 244 |
-
- Syncthing: phone camera/share folder → `inbox/`
|
| 245 |
|
| 246 |
```bash
|
| 247 |
python -m app.cli scan --image path/to/receipt.jpg
|
|
@@ -249,11 +247,13 @@ python -m app.cli query --category groceries
|
|
| 249 |
pytest
|
| 250 |
```
|
| 251 |
|
|
|
|
|
|
|
| 252 |
## What you still run yourself
|
| 253 |
|
| 254 |
-
-
|
| 255 |
-
-
|
| 256 |
-
-
|
| 257 |
-
- Optional: Syncthing
|
| 258 |
|
| 259 |
No model weights in this repo. No PyInstaller in this release.
|
|
|
|
| 18 |
GitHub: [`drowzeys/keys-Auto-Receipts-Studio`](https://github.com/drowzeys/keys-Auto-Receipts-Studio)
|
| 19 |
Hugging Face: [`drowzeys/keys-Auto-Receipts-Studio`](https://huggingface.co/drowzeys/keys-Auto-Receipts-Studio)
|
| 20 |
|
| 21 |
+
## Pathways (pick a recipe)
|
| 22 |
+
|
| 23 |
+
Gemma 4 12B-it **never** runs on the Lamp (6 GB) and typically **not** on a Windows/Mac laptop. The GPU box is the brain. PC / Mac / Linux run the inbox + Review UI. iPhone is a Safari capture client on the same Wi‑Fi.
|
| 24 |
+
|
| 25 |
+
| Pathway | One-shot recipe | Gemma 4 12B | iPhone page |
|
| 26 |
+
|---|---|---|---|
|
| 27 |
+
| **A. Linux NVIDIA GPU box** (Spark / Omen) | `bash oneshot.sh` | started here (`:8080`, util **0.15**, FP8, never above **0.85**) | `http://<this-box-lan-ip>:7860/phone` |
|
| 28 |
+
| **B. Linux, no NVIDIA** | `bash oneshot.sh` then edit `.env` | skip local vLLM; point `.env` at pathway A | same, this machine’s LAN IP |
|
| 29 |
+
| **C. Windows PC** | `oneshot.bat` then `scripts\install-launcher.bat` | on pathway A (or WSL2) | `http://<this-pc-lan-ip>:7860/phone` |
|
| 30 |
+
| **D. macOS** | venv + `bash scripts/install-launcher.sh` | on pathway A | `http://<this-mac-lan-ip>:7860/phone` |
|
| 31 |
+
| **E. iPhone** | Safari only, after A/B/C/D UI is up | n/a (uploads to the UI box) | `/phone` — **Take photo** or **Choose file** |
|
| 32 |
+
| **F. Autonomous Lamp** | `skills/keys-receipt-scanner/` | on pathway A | n/a (HAL camera → Studio `/api/inbox`) |
|
| 33 |
+
|
| 34 |
+
```bash
|
| 35 |
+
git clone https://github.com/drowzeys/keys-Auto-Receipts-Studio.git
|
| 36 |
+
cd keys-Auto-Receipts-Studio
|
| 37 |
+
```
|
| 38 |
+
|
| 39 |
+
Desktop icon after first UI start: Linux/macOS `bash scripts/install-launcher.sh` · Windows `scripts\install-launcher.bat`. Step-by-step for each letter is [below](#a-linux-nvidia-gpu-box-gemma--ui--iphone).
|
| 40 |
+
|
| 41 |
## Screenshots (v1.0 alpha)
|
| 42 |
|
| 43 |
iPhone Safari — take a photo, then completion notice:
|
|
|
|
| 60 |
<img src="docs/screenshots/04-settings.png" alt="Settings backends" width="420"/>
|
| 61 |
</p>
|
| 62 |
|
| 63 |
+
## Promo (v1.0 alpha)
|
| 64 |
|
| 65 |
+
[~50s story cut](docs/promo/Keys-Auto-Receipts-Studio-promo.mp4) — messy dorm → iPhone (lay the receipt flat, then photograph) → charcoal Autonomous Lamp scans the last receipt → trash → clean room. Bed: Beethoven Symphony No. 5, I. Allegro con brio (public-domain recording).
|
| 66 |
+
|
| 67 |
+
Also attached on the [v1.0-alpha-promo release](https://github.com/drowzeys/keys-Auto-Receipts-Studio/releases/tag/v1.0-alpha-promo) (plays in the browser).
|
| 68 |
+
|
| 69 |
+
## Pathway details
|
| 70 |
+
|
| 71 |
+
One-click / `app.launch` binds the UI on the LAN (`RECEIPT_UI_SHARE_LAN=true`) and opens Review in the browser. Phone URL is printed in the terminal.
|
| 72 |
+
|
| 73 |
+
Desktop icons (do **not** copy `scripts/start-ui.bat` itself to the Desktop):
|
| 74 |
+
|
| 75 |
+
| OS | Command | Opens |
|
| 76 |
+
|---|---|---|
|
| 77 |
+
| Linux | `bash scripts/install-launcher.sh` | Desktop **Receipt Studio.desktop** |
|
| 78 |
+
| macOS | `bash scripts/install-launcher.sh` | Desktop **Receipt Studio.command** (first time: right-click → **Open**) |
|
| 79 |
+
| Windows | `scripts\install-launcher.bat` | Desktop **Receipt Studio.bat** |
|
| 80 |
+
|
| 81 |
+
### A. Linux NVIDIA GPU box (Gemma + UI + iPhone)
|
| 82 |
|
| 83 |
```bash
|
|
|
|
|
|
|
| 84 |
bash oneshot.sh
|
| 85 |
```
|
| 86 |
|
| 87 |
+
Idempotent. Installs `.venv`, fetches **google/gemma-4-12B-it** to `~/models-gemma4-12b-it` if missing (`hf auth login` once), starts `scripts/serve-gemma.sh` if `:8080` is down, starts the UI on **0.0.0.0:7860**.
|
| 88 |
+
|
| 89 |
When it prints READY:
|
| 90 |
|
| 91 |
| | |
|
| 92 |
|---|---|
|
| 93 |
| Review | http://127.0.0.1:7860 |
|
| 94 |
+
| iPhone Safari (same Wi‑Fi) | http://<this-box-lan-ip>:7860/phone |
|
| 95 |
+
| Gemma OpenAI-compat | http://127.0.0.1:8080/v1 |
|
| 96 |
|
| 97 |
+
vLLM flags (also used by `scripts/start-ui.sh` on a CUDA box if Gemma is down):
|
| 98 |
|
| 99 |
+
- `--gpu-memory-utilization **0.15**` (~18.3 GiB of 121.7 GiB; **never above 0.85**)
|
| 100 |
+
- **FP8** (BF16 weights ~23 GB cannot fit in that pool)
|
| 101 |
+
- `--max-model-len **8192**`
|
| 102 |
|
| 103 |
+
If Gemma is already running, oneshot / launch leave it alone. To apply the 0.15 cap: stop that `vllm` process, then `bash oneshot.sh` again.
|
| 104 |
|
| 105 |
+
Gemma-only (UI already running): `bash scripts/serve-gemma.sh`
|
| 106 |
|
| 107 |
+
### B. Linux without NVIDIA (UI + iPhone only)
|
| 108 |
+
|
| 109 |
+
```bash
|
| 110 |
+
bash oneshot.sh
|
| 111 |
+
```
|
| 112 |
+
|
| 113 |
+
If `vllm` is not on PATH and nothing answers on `:8080`, oneshot **skips** weights and vLLM (it does **not** die) and still starts the UI. Then set `.env` at the GPU box from pathway A:
|
| 114 |
|
| 115 |
```
|
| 116 |
RECEIPT_LLM_BASE_URL=http://<spark-lan-ip>:8080/v1
|
|
|
|
| 120 |
RECEIPT_EMBED_DIM=3840
|
| 121 |
```
|
| 122 |
|
| 123 |
+
Restart the UI (`./scripts/start-ui.sh` or the desktop icon).
|
| 124 |
|
| 125 |
+
### C. Windows PC (UI + iPhone hotspot)
|
| 126 |
|
| 127 |
+
12B is **not** started by the Windows scripts. Serve Gemma on pathway A (or WSL2 + `bash oneshot.sh`).
|
| 128 |
|
| 129 |
+
1. Install [Python 3.12](https://www.python.org/downloads/) — check **Add python.exe to PATH**.
|
| 130 |
+
2. In the cloned repo, double-click **`oneshot.bat`** (creates `.venv`, opens Review).
|
| 131 |
+
3. Edit `.env` as in pathway B (Spark LAN IP, port **8080**).
|
| 132 |
+
4. Optional desktop icon: `scripts\install-launcher.bat` → **Desktop\Receipt Studio.bat**.
|
| 133 |
+
5. iPhone: `http://<this-windows-lan-ip>:7860/phone` in **Safari**.
|
| 134 |
+
|
| 135 |
+
Do not copy `scripts\start-ui.bat` to the Desktop; it must run from the repo (or from the wrapper `install-launcher.bat` writes).
|
| 136 |
+
|
| 137 |
+
If this PC **is** NVIDIA + `vllm` on PATH (Git Bash / WSL): point `.env` at `127.0.0.1:8080` and `bash scripts/serve-gemma.sh`.
|
| 138 |
+
|
| 139 |
+
### D. macOS (UI + iPhone)
|
| 140 |
+
|
| 141 |
+
Apple Silicon does not load 12B with this vLLM-NVIDIA recipe. Use pathway A for Gemma.
|
| 142 |
|
| 143 |
```bash
|
|
|
|
|
|
|
| 144 |
python3 -m venv .venv
|
| 145 |
.venv/bin/pip install -e ".[dev]"
|
| 146 |
cp .env.example .env
|
| 147 |
+
# edit RECEIPT_LLM_BASE_URL / RECEIPT_EMBED_BASE_URL like pathway B
|
| 148 |
bash scripts/install-launcher.sh
|
| 149 |
```
|
| 150 |
|
| 151 |
+
Double-click **Desktop → Receipt Studio.command** (first time: right-click → **Open**). Same as `./scripts/start-ui.sh`.
|
| 152 |
|
| 153 |
+
### E. iPhone (Safari)
|
| 154 |
|
| 155 |
+
Works against **whichever machine runs the UI** (A, B, C, or D), same Wi‑Fi. Chrome/Firefox on iOS are not the supported path.
|
| 156 |
|
| 157 |
+
1. Open `http://<ui-machine-lan-ip>:7860/phone`
|
| 158 |
+
2. **Take photo** (camera) or **Choose file** (Photos / Files, including HEIC)
|
| 159 |
+
3. Wait for **Completed: vendor · total · category · #id**
|
| 160 |
+
4. Confirm / edit / delete on the desktop **Review** tab
|
| 161 |
|
| 162 |
+
The phone page POSTs `/api/inbox` and polls `/api/jobs/{id}`. The GPU box (or the UI machine talking to Gemma) does extract + embed. Inbox drop and Syncthing into `inbox/` use the same pipeline (files idle 30s unless the phone API processes immediately).
|
| 163 |
|
| 164 |
+
### F. Autonomous Lamp skill
|
| 165 |
|
| 166 |
+
Repo path: **`skills/keys-receipt-scanner/`** (`SKILL.md`, `skill.json` `capabilities: ["vision"]`, `scripts/scan.py`).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 167 |
|
| 168 |
```
|
| 169 |
skills/keys-receipt-scanner/
|
| 170 |
+
SKILL.md
|
| 171 |
+
skill.json
|
| 172 |
scripts/scan.py
|
| 173 |
+
references/hardware.md
|
| 174 |
```
|
| 175 |
|
| 176 |
+
Install (no reboot):
|
| 177 |
|
| 178 |
```bash
|
| 179 |
make push-skill SKILL=./skills/keys-receipt-scanner TARGET=pi@lamp-xxxx.local
|
| 180 |
```
|
| 181 |
|
| 182 |
+
On the Lamp (camera only — **do not** load 12B):
|
| 183 |
|
| 184 |
```bash
|
| 185 |
+
export RECEIPT_GPU_HOST=<spark-lan-ip>
|
| 186 |
+
export RECEIPT_STUDIO_URL=http://<spark-lan-ip>:7860
|
| 187 |
+
export RECEIPT_CAMERA_URL=http://127.0.0.1:5001
|
|
|
|
|
|
|
|
|
|
| 188 |
```
|
| 189 |
|
| 190 |
+
Say **“scan this receipt”**. `scripts/scan.py` snapshots HAL (`:5001/camera/snapshot`, 1280 px), then POSTs to Receipt Studio `/api/inbox` on the GPU box. Fallback: `RECEIPT_LLM_BASE_URL=http://<spark-lan-ip>:8080/v1` (direct vLLM) or `RECEIPT_HERMES_BASE_URL` (same OpenAI-compat URL Hermes registered). On the GPU box, `python scripts/register-hermes-gemma.py` adds `custom_providers` `gemma4-12b-it`.
|
| 191 |
|
| 192 |
+
Intern has no camera; this skill will not install there.
|
| 193 |
|
| 194 |
+
To ship in a fork of [autonomous-os](https://github.com/autonomous-ai/autonomous-os): copy `skills/keys-receipt-scanner/` into `skills/`, run `python skills/skill-creator/scripts/quick_validate.py`, `make skills-catalog`, open the PR.
|
| 195 |
|
| 196 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 197 |
|
| 198 |
+
## Does Gemma 4 12B Unified fit on the Lamp?
|
| 199 |
|
| 200 |
+
**No.** Lamp is 8-core ARM64 with **6 GB RAM**. Gemma 4 12B Unified is a 12B dense omni model (`Gemma4UnifiedForConditionalGeneration`, hidden size **3840**).
|
| 201 |
|
| 202 |
+
| Piece | Lamp (6 GB) | GPU box |
|
| 203 |
+
|---|---|---|
|
| 204 |
+
| `skills/keys-receipt-scanner/` | yes | yes |
|
| 205 |
+
| HAL snapshot | yes | n/a |
|
| 206 |
+
| SQLite + HTTP client | yes | yes |
|
| 207 |
+
| **Gemma 4 12B Unified weights** | **no** | yes (vLLM util **0.15**) |
|
| 208 |
|
| 209 |
+
## Gemma 4 12B Unified (omni) on the GPU box
|
|
|
|
|
|
|
|
|
|
|
|
|
| 210 |
|
| 211 |
+
One OpenAI-compat server: vision `POST /v1/chat/completions` **and** `POST /v1/embeddings` (dim **3840**). Do not raise util above **0.85**.
|
| 212 |
|
| 213 |
```bash
|
| 214 |
+
bash scripts/serve-gemma.sh
|
| 215 |
+
# util 0.15, FP8, max-model-len 8192
|
|
|
|
|
|
|
|
|
|
| 216 |
```
|
| 217 |
|
| 218 |
+
Context at util 0.15 (after ~12.5 GB FP8 weights):
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 219 |
|
| 220 |
| Estimate | Tokens |
|
| 221 |
|---|---|
|
| 222 |
+
| Conservative (48 layers full attn) | **~12k** |
|
| 223 |
| Hybrid (8 full + 40× sliding-1024) | **~65k** |
|
| 224 |
| Model native window | 262,144 (not at 0.15) |
|
| 225 |
|
| 226 |
+
A receipt scan uses ~1–2k tokens. Raise `RECEIPT_VLLM_MAX_MODEL_LEN` only after the boot log shows a large enough `GPU KV cache size`.
|
|
|
|
|
|
|
| 227 |
|
| 228 |
+
If `/v1/embeddings` 404s, either serve a pooling convert on another port (`RECEIPT_EMBED_BASE_URL`) or `RECEIPT_EMBED_BACKEND=nvidia` + Nemotron-3-Embed-1B at dim **2048**. **Never mix 3840 and 2048 in one DB.**
|
|
|
|
|
|
|
|
|
|
|
|
|
| 229 |
|
| 230 |
+
Fallback vision (not omni embed): Qwen3.8-27B ADay777 at `:8078` (`RECEIPT_LLM_BACKEND=nvidia`, `RECEIPT_LLM_MODEL=qwen38-nvfp4`). Lightning is **text-only** — never send images.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 231 |
|
| 232 |
+
## Manual UI (any OS, after venv)
|
| 233 |
|
| 234 |
```bash
|
| 235 |
+
python3 -m venv .venv && source .venv/bin/activate # Windows: .venv\Scripts\activate
|
| 236 |
pip install -e ".[dev]"
|
| 237 |
cp .env.example .env
|
| 238 |
+
# set RECEIPT_LLM_* like pathway B unless Gemma is local
|
| 239 |
+
RECEIPT_UI_SHARE_LAN=true python -m app.cli ui
|
| 240 |
```
|
| 241 |
|
| 242 |
+
CLI:
|
|
|
|
|
|
|
| 243 |
|
| 244 |
```bash
|
| 245 |
python -m app.cli scan --image path/to/receipt.jpg
|
|
|
|
| 247 |
pytest
|
| 248 |
```
|
| 249 |
|
| 250 |
+
Syncthing: camera/share folder → `inbox/` (idle **30s** then process).
|
| 251 |
+
|
| 252 |
## What you still run yourself
|
| 253 |
|
| 254 |
+
- Pathway A (or equivalent) so Gemma is reachable on the LAN
|
| 255 |
+
- iPhone on the **same Wi‑Fi**, Safari
|
| 256 |
+
- Lamp: pair in the Autonomous app, `make push-skill`, set `RECEIPT_STUDIO_URL`
|
| 257 |
+
- Optional: Syncthing
|
| 258 |
|
| 259 |
No model weights in this repo. No PyInstaller in this release.
|