Aarya2004 Claude Opus 4.8 (1M context) commited on
Commit Β·
fa87836
1
Parent(s): 7761d00
docs(readme): Artifacts + two-stacks + Cohere credit; bundle eval graphs
Browse filesBrings the Space README to the current state: Artifacts section (fine-tuned
LoRA links, eval table), the Private-vs-Best two-stacks comparison, and the
sponsor:cohere credit. Includes docs/img/*.png so the Artifacts graphs render
on the Space page (the Space deploy doesn't otherwise ship docs/).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- README.md +47 -0
- docs/img/brain_f1.png +0 -0
- docs/img/finetune.png +0 -0
- docs/img/recall.png +0 -0
README.md
CHANGED
|
@@ -16,6 +16,7 @@ tags:
|
|
| 16 |
- track:backyard
|
| 17 |
- sponsor:openbmb
|
| 18 |
- sponsor:nvidia
|
|
|
|
| 19 |
- sponsor:modal
|
| 20 |
- achievement:welltuned
|
| 21 |
- achievement:offbrand
|
|
@@ -54,6 +55,11 @@ a GPU:
|
|
| 54 |
Stack, both run **Nemotron-Omni**, one multimodal deployment serving vision and speech.
|
| 55 |
(Locally, perception runs MiniCPM-V from OpenBMB β see _Backend resolution_.)
|
| 56 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
The point of the build is the **same family at two tiers**:
|
| 58 |
|
| 59 |
- **Private Stack (local / Airplane-Mode)** β **Nemotron-3-Nano 4B** via Ollama, on your
|
|
@@ -67,6 +73,27 @@ One agent, one tool contract β flip `FF_BACKEND` and the **Nemotron brain** mo
|
|
| 67 |
on your laptop to a 30B on a GPU without touching the agent code. Facts-from-Tools holds at
|
| 68 |
both tiers: the model never invents a price.
|
| 69 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
## What's real
|
| 71 |
|
| 72 |
- **Vision** β MiniCPM-V (OpenBMB) reads job photos β observations, locally via Ollama.
|
|
@@ -77,6 +104,26 @@ both tiers: the model never invents a price.
|
|
| 77 |
- **Phone capture** β call a Twilio number (it forges a draft + texts the PDF) or scan a QR to capture a photo + voice note on your phone and forge live on the desktop.
|
| 78 |
- **Frontend** β a bespoke web UI served by `gradio.Server` (FastAPI under the hood): streaming "Digital Apprentice" trace, editable estimate, PDF export.
|
| 79 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 80 |
## Run
|
| 81 |
|
| 82 |
```
|
|
|
|
| 16 |
- track:backyard
|
| 17 |
- sponsor:openbmb
|
| 18 |
- sponsor:nvidia
|
| 19 |
+
- sponsor:cohere
|
| 20 |
- sponsor:modal
|
| 21 |
- achievement:welltuned
|
| 22 |
- achievement:offbrand
|
|
|
|
| 55 |
Stack, both run **Nemotron-Omni**, one multimodal deployment serving vision and speech.
|
| 56 |
(Locally, perception runs MiniCPM-V from OpenBMB β see _Backend resolution_.)
|
| 57 |
|
| 58 |
+
The rest of the orchestra is small models from the other sponsors: **Cohere** powers the
|
| 59 |
+
multilingual role (**Aya / Aya-Expanse**, customer-facing copy in Spanish/French/Mandarin)
|
| 60 |
+
and the on-device **Transcribe** for the voice note; **OpenBMB**'s **MiniCPM-V** reads the
|
| 61 |
+
job photo on the local stack (and is the model we fine-tuned β see _Artifacts_).
|
| 62 |
+
|
| 63 |
The point of the build is the **same family at two tiers**:
|
| 64 |
|
| 65 |
- **Private Stack (local / Airplane-Mode)** β **Nemotron-3-Nano 4B** via Ollama, on your
|
|
|
|
| 73 |
on your laptop to a 30B on a GPU without touching the agent code. Facts-from-Tools holds at
|
| 74 |
both tiers: the model never invents a price.
|
| 75 |
|
| 76 |
+
### The two stacks, side by side
|
| 77 |
+
|
| 78 |
+
Same agent, same tools, same Facts-from-Tools guarantee β only the models behind each role
|
| 79 |
+
change:
|
| 80 |
+
|
| 81 |
+
| Role | π Private Stack (local, Ollama / on-device) | β‘ Best Stack (hosted, Modal GPUs) |
|
| 82 |
+
| ----------------- | -------------------------------------------- | ---------------------------------- |
|
| 83 |
+
| **Brain** | Nemotron-3-Nano **4B** (NVIDIA) | Nemotron-3-Nano **30B** (NVIDIA) |
|
| 84 |
+
| **Perception** | MiniCPM-V (OpenBMB) | Nemotron-Omni **30B** (NVIDIA) |
|
| 85 |
+
| **Audio** | Cohere Transcribe (on-device) | Nemotron-Omni **30B** (NVIDIA) |
|
| 86 |
+
| **Multilingual** | Aya (Cohere) | Aya-Expanse **8B** (Cohere) |
|
| 87 |
+
| **Embedding** | on-device (sentence-transformers) | _same on-device path_ |
|
| 88 |
+
| **Extraction** | _no local path_ | Parse extractor (fine-tuned) |
|
| 89 |
+
| **Runs offline?** | β
Yes β Airplane-Mode Proof | β No β public HTTPS GPU endpoints |
|
| 90 |
+
| **Cost / GPU** | $0, your hardware | scales to zero when idle |
|
| 91 |
+
|
| 92 |
+
**Switch with env, not code:** the local stack is the default; set `FF_BACKEND=modal` (+ the
|
| 93 |
+
`FF_MODAL_*_URL` secrets) to ride the Best Stack. Each role opts in independently β see
|
| 94 |
+
_[Backend resolution](#backend-resolution--read-this-before-am-i-on-modal)_ for the full
|
| 95 |
+
matrix (including the stub mode the public Space falls back to).
|
| 96 |
+
|
| 97 |
## What's real
|
| 98 |
|
| 99 |
- **Vision** β MiniCPM-V (OpenBMB) reads job photos β observations, locally via Ollama.
|
|
|
|
| 104 |
- **Phone capture** β call a Twilio number (it forges a draft + texts the PDF) or scan a QR to capture a photo + voice note on your phone and forge live on the desktop.
|
| 105 |
- **Frontend** β a bespoke web UI served by `gradio.Server` (FastAPI under the hood): streaming "Digital Apprentice" trace, editable estimate, PDF export.
|
| 106 |
|
| 107 |
+
## Artifacts
|
| 108 |
+
|
| 109 |
+
**Fine-tuned models** (MiniCPM-V LoRA adapters, on the Hub):
|
| 110 |
+
|
| 111 |
+
- π― [`Aarya2004/minicpmv-trade-lora`](https://huggingface.co/Aarya2004/minicpmv-trade-lora) β fine-tuned on a grounded-synthetic set of trade invoices (built from a real 381-entry catalog). In-distribution item F1 **0.703 β 0.933** (+0.23), price accuracy β 1.00.
|
| 112 |
+
- [`Aarya2004/minicpmv-cord-lora`](https://huggingface.co/Aarya2004/minicpmv-cord-lora) β the conservative, out-of-domain baseline on the public [CORD](https://huggingface.co/datasets/naver-clova-ix/cord-v2) receipt benchmark: item F1 **0.588 β 0.681** (+0.09).
|
| 113 |
+
|
| 114 |
+
**Evals** β every headline number is reproducible (`scripts/run_brain_eval.py`, `scripts/run_recall_eval.py`):
|
| 115 |
+
|
| 116 |
+
| Metric | Before | After |
|
| 117 |
+
| ------------------------------------ | ------ | ----- |
|
| 118 |
+
| Agent Brain item F1 | 0.367 | 0.967 |
|
| 119 |
+
| Episodic recall@1 | 0.750 | 0.875 |
|
| 120 |
+
| MiniCPM-V item F1 (trade, in-domain) | 0.703 | 0.933 |
|
| 121 |
+
| MiniCPM-V item F1 (CORD, OOD) | 0.588 | 0.681 |
|
| 122 |
+
|
| 123 |
+

|
| 124 |
+

|
| 125 |
+

|
| 126 |
+
|
| 127 |
## Run
|
| 128 |
|
| 129 |
```
|
docs/img/brain_f1.png
ADDED
|
docs/img/finetune.png
ADDED
|
docs/img/recall.png
ADDED
|