Aarya2004 Claude Opus 4.8 (1M context) commited on
Commit
fa87836
Β·
1 Parent(s): 7761d00

docs(readme): Artifacts + two-stacks + Cohere credit; bundle eval graphs

Browse files

Brings the Space README to the current state: Artifacts section (fine-tuned
LoRA links, eval table), the Private-vs-Best two-stacks comparison, and the
sponsor:cohere credit. Includes docs/img/*.png so the Artifacts graphs render
on the Space page (the Space deploy doesn't otherwise ship docs/).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

README.md CHANGED
@@ -16,6 +16,7 @@ tags:
16
  - track:backyard
17
  - sponsor:openbmb
18
  - sponsor:nvidia
 
19
  - sponsor:modal
20
  - achievement:welltuned
21
  - achievement:offbrand
@@ -54,6 +55,11 @@ a GPU:
54
  Stack, both run **Nemotron-Omni**, one multimodal deployment serving vision and speech.
55
  (Locally, perception runs MiniCPM-V from OpenBMB β€” see _Backend resolution_.)
56
 
 
 
 
 
 
57
  The point of the build is the **same family at two tiers**:
58
 
59
  - **Private Stack (local / Airplane-Mode)** β€” **Nemotron-3-Nano 4B** via Ollama, on your
@@ -67,6 +73,27 @@ One agent, one tool contract β€” flip `FF_BACKEND` and the **Nemotron brain** mo
67
  on your laptop to a 30B on a GPU without touching the agent code. Facts-from-Tools holds at
68
  both tiers: the model never invents a price.
69
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
70
  ## What's real
71
 
72
  - **Vision** β€” MiniCPM-V (OpenBMB) reads job photos β†’ observations, locally via Ollama.
@@ -77,6 +104,26 @@ both tiers: the model never invents a price.
77
  - **Phone capture** β€” call a Twilio number (it forges a draft + texts the PDF) or scan a QR to capture a photo + voice note on your phone and forge live on the desktop.
78
  - **Frontend** β€” a bespoke web UI served by `gradio.Server` (FastAPI under the hood): streaming "Digital Apprentice" trace, editable estimate, PDF export.
79
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
80
  ## Run
81
 
82
  ```
 
16
  - track:backyard
17
  - sponsor:openbmb
18
  - sponsor:nvidia
19
+ - sponsor:cohere
20
  - sponsor:modal
21
  - achievement:welltuned
22
  - achievement:offbrand
 
55
  Stack, both run **Nemotron-Omni**, one multimodal deployment serving vision and speech.
56
  (Locally, perception runs MiniCPM-V from OpenBMB β€” see _Backend resolution_.)
57
 
58
+ The rest of the orchestra is small models from the other sponsors: **Cohere** powers the
59
+ multilingual role (**Aya / Aya-Expanse**, customer-facing copy in Spanish/French/Mandarin)
60
+ and the on-device **Transcribe** for the voice note; **OpenBMB**'s **MiniCPM-V** reads the
61
+ job photo on the local stack (and is the model we fine-tuned β€” see _Artifacts_).
62
+
63
  The point of the build is the **same family at two tiers**:
64
 
65
  - **Private Stack (local / Airplane-Mode)** β€” **Nemotron-3-Nano 4B** via Ollama, on your
 
73
  on your laptop to a 30B on a GPU without touching the agent code. Facts-from-Tools holds at
74
  both tiers: the model never invents a price.
75
 
76
+ ### The two stacks, side by side
77
+
78
+ Same agent, same tools, same Facts-from-Tools guarantee β€” only the models behind each role
79
+ change:
80
+
81
+ | Role | πŸ”’ Private Stack (local, Ollama / on-device) | ⚑ Best Stack (hosted, Modal GPUs) |
82
+ | ----------------- | -------------------------------------------- | ---------------------------------- |
83
+ | **Brain** | Nemotron-3-Nano **4B** (NVIDIA) | Nemotron-3-Nano **30B** (NVIDIA) |
84
+ | **Perception** | MiniCPM-V (OpenBMB) | Nemotron-Omni **30B** (NVIDIA) |
85
+ | **Audio** | Cohere Transcribe (on-device) | Nemotron-Omni **30B** (NVIDIA) |
86
+ | **Multilingual** | Aya (Cohere) | Aya-Expanse **8B** (Cohere) |
87
+ | **Embedding** | on-device (sentence-transformers) | _same on-device path_ |
88
+ | **Extraction** | _no local path_ | Parse extractor (fine-tuned) |
89
+ | **Runs offline?** | βœ… Yes β€” Airplane-Mode Proof | ❌ No β€” public HTTPS GPU endpoints |
90
+ | **Cost / GPU** | $0, your hardware | scales to zero when idle |
91
+
92
+ **Switch with env, not code:** the local stack is the default; set `FF_BACKEND=modal` (+ the
93
+ `FF_MODAL_*_URL` secrets) to ride the Best Stack. Each role opts in independently β€” see
94
+ _[Backend resolution](#backend-resolution--read-this-before-am-i-on-modal)_ for the full
95
+ matrix (including the stub mode the public Space falls back to).
96
+
97
  ## What's real
98
 
99
  - **Vision** β€” MiniCPM-V (OpenBMB) reads job photos β†’ observations, locally via Ollama.
 
104
  - **Phone capture** β€” call a Twilio number (it forges a draft + texts the PDF) or scan a QR to capture a photo + voice note on your phone and forge live on the desktop.
105
  - **Frontend** β€” a bespoke web UI served by `gradio.Server` (FastAPI under the hood): streaming "Digital Apprentice" trace, editable estimate, PDF export.
106
 
107
+ ## Artifacts
108
+
109
+ **Fine-tuned models** (MiniCPM-V LoRA adapters, on the Hub):
110
+
111
+ - 🎯 [`Aarya2004/minicpmv-trade-lora`](https://huggingface.co/Aarya2004/minicpmv-trade-lora) β€” fine-tuned on a grounded-synthetic set of trade invoices (built from a real 381-entry catalog). In-distribution item F1 **0.703 β†’ 0.933** (+0.23), price accuracy β†’ 1.00.
112
+ - [`Aarya2004/minicpmv-cord-lora`](https://huggingface.co/Aarya2004/minicpmv-cord-lora) β€” the conservative, out-of-domain baseline on the public [CORD](https://huggingface.co/datasets/naver-clova-ix/cord-v2) receipt benchmark: item F1 **0.588 β†’ 0.681** (+0.09).
113
+
114
+ **Evals** β€” every headline number is reproducible (`scripts/run_brain_eval.py`, `scripts/run_recall_eval.py`):
115
+
116
+ | Metric | Before | After |
117
+ | ------------------------------------ | ------ | ----- |
118
+ | Agent Brain item F1 | 0.367 | 0.967 |
119
+ | Episodic recall@1 | 0.750 | 0.875 |
120
+ | MiniCPM-V item F1 (trade, in-domain) | 0.703 | 0.933 |
121
+ | MiniCPM-V item F1 (CORD, OOD) | 0.588 | 0.681 |
122
+
123
+ ![Agent Brain item F1](docs/img/brain_f1.png)
124
+ ![Episodic recall](docs/img/recall.png)
125
+ ![MiniCPM-V fine-tune](docs/img/finetune.png)
126
+
127
  ## Run
128
 
129
  ```
docs/img/brain_f1.png ADDED
docs/img/finetune.png ADDED
docs/img/recall.png ADDED