Sync Dream Customs UI voice settings
#8
by ADJCJH - opened
- .gitattributes +0 -3
- AGENTS.md +3 -0
- README.md +10 -6
- docs/goal-prompts/2026-06-06-modal-minicpm-backend.md +9 -0
- docs/handoff.md +15 -0
- docs/smoke/2026-06-06-modal-minicpm-backend-smoke.md +30 -0
- docs/superpowers/plans/2026-06-06-modal-minicpm-backend.md +1226 -0
- dream_customs/app_logic.py +153 -31
- dream_customs/models.py +76 -4
- dream_customs/ui/actions.py +27 -17
- dream_customs/ui/app.py +205 -27
- dream_customs/ui/copy.py +1 -0
- dream_customs/ui/styles.py +172 -12
- modal_backend/__init__.py +1 -0
- modal_backend/contracts.py +71 -0
- modal_backend/dream_customs_modal.py +392 -0
- scripts/eval_hosted_routes.py +95 -0
- scripts/smoke_hosted_routes.py +72 -0
- tests/test_app_logic.py +61 -0
- tests/test_modal_contract.py +60 -0
- tests/test_pipeline.py +8 -0
- tests/test_ui_actions.py +51 -3
.gitattributes
CHANGED
|
@@ -33,6 +33,3 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
-
docs/design/assets/dream-customs-probe-morning-console.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
-
docs/design/assets/dream-customs-probe-night-border.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
-
docs/design/assets/dream-customs-probe-night-desk.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
AGENTS.md
CHANGED
|
@@ -70,6 +70,9 @@
|
|
| 70 |
- 如果需要正式提交,先确认是否已初始化/切换到 Dream Customs 专用仓库。
|
| 71 |
- 在仓库归属未确认前,可以本地写文件,但不要强行 `git add` / `git commit` / `git push` 到父级 VLA 仓库。
|
| 72 |
- 如果建立新功能分支,使用 `feature/xxx` 命名。
|
|
|
|
|
|
|
|
|
|
| 73 |
|
| 74 |
## 实现原则
|
| 75 |
|
|
|
|
| 70 |
- 如果需要正式提交,先确认是否已初始化/切换到 Dream Customs 专用仓库。
|
| 71 |
- 在仓库归属未确认前,可以本地写文件,但不要强行 `git add` / `git commit` / `git push` 到父级 VLA 仓库。
|
| 72 |
- 如果建立新功能分支,使用 `feature/xxx` 命名。
|
| 73 |
+
- 用户要求修改完成后,默认不仅推送 GitHub 分支,也要同步提交到 Hugging Face Space,并完成可用的 merge / 部署路径;不要只停在本地或 GitHub 分支。
|
| 74 |
+
- 同步 HF Space 前先确认 `space` remote 指向 `build-small-hackathon/dream-customs`,并复核当前分支与提交 SHA,避免误推到其他 Space 或父级仓库。
|
| 75 |
+
- 如果 HF Space 同步涉及密钥、账单、权限弹窗、强制覆盖公开 `main`、或无法自动 merge 的 PR/Discussion,则先说明具体阻塞并请求用户确认;不要打印或保存任何 token。
|
| 76 |
|
| 77 |
## 实现原则
|
| 78 |
|
README.md
CHANGED
|
@@ -65,13 +65,17 @@ Local smoke notes from this Mac mini:
|
|
| 65 |
|
| 66 |
## Optional Hosted MiniCPM Routes
|
| 67 |
|
| 68 |
-
|
| 69 |
|
| 70 |
-
- `DREAM_CUSTOMS_TEXT_ENDPOINT`
|
| 71 |
-
- `DREAM_CUSTOMS_VISION_ENDPOINT`
|
| 72 |
-
- `DREAM_CUSTOMS_HOSTED_TOKEN`
|
| 73 |
|
| 74 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
|
| 76 |
Token-safe text smoke:
|
| 77 |
|
|
@@ -114,7 +118,7 @@ python -m pytest -q
|
|
| 114 |
|
| 115 |
2026-06-05 local V2 verification passed: tests were green and the workbench flow reached a sealed pact through `Send to customs`, `Ask another question`, `Add material`, `Draft pact`, `Revise pact`, and `Seal today's pact`.
|
| 116 |
|
| 117 |
-
The public Space now serves the V2 workbench from Space `main` commit `8ad6f00628f800abc2dbefab05163aba94a5723f`. Public browser smoke, mobile readability, diagnostics, raw remote queue prediction, and a hosted text route smoke all reached a sealed pact.
|
| 118 |
|
| 119 |
Current smoke details are tracked in `docs/smoke/2026-06-05-space-deployment-smoke.md`.
|
| 120 |
|
|
|
|
| 65 |
|
| 66 |
## Optional Hosted MiniCPM Routes
|
| 67 |
|
| 68 |
+
The public Space stays lightweight and can call private Modal endpoints through runtime secrets:
|
| 69 |
|
| 70 |
+
- `DREAM_CUSTOMS_TEXT_ENDPOINT`: Modal text route for `openbmb/MiniCPM5-1B`.
|
| 71 |
+
- `DREAM_CUSTOMS_VISION_ENDPOINT`: Modal vision route for `openbmb/MiniCPM-V-4.6`.
|
| 72 |
+
- `DREAM_CUSTOMS_HOSTED_TOKEN`: shared bearer token checked by Modal and sent by the Space.
|
| 73 |
|
| 74 |
+
Set these only as Hugging Face Space repository secrets or local shell variables. Do not store values in `.env`, docs, logs, screenshots, or git. Missing endpoints or route failures fall back to deterministic demo behavior.
|
| 75 |
+
|
| 76 |
+
The Gradio UI defaults to `model` for both text and vision backends, so a configured Space calls Modal by default. The `demo` backend remains available in developer settings as the deterministic fallback path.
|
| 77 |
+
|
| 78 |
+
The Hugging Face Space may run on ZeroGPU for hackathon hardware eligibility. `dream_customs.zerogpu` registers a lightweight `@spaces.GPU` startup probe so ZeroGPU accepts the app, but real MiniCPM inference still happens on the private Modal backend.
|
| 79 |
|
| 80 |
Token-safe text smoke:
|
| 81 |
|
|
|
|
| 118 |
|
| 119 |
2026-06-05 local V2 verification passed: tests were green and the workbench flow reached a sealed pact through `Send to customs`, `Ask another question`, `Add material`, `Draft pact`, `Revise pact`, and `Seal today's pact`.
|
| 120 |
|
| 121 |
+
The public Space now serves the V2 workbench from Space `main` commit `8ad6f00628f800abc2dbefab05163aba94a5723f`. Public browser smoke, mobile readability, diagnostics, raw remote queue prediction, and a hosted text route smoke all reached a sealed pact on 2026-06-05. The current Modal backend pass requires a real `openbmb/MiniCPM-V-4.6` vision route smoke before delivery; demo vision fallback is runtime resilience, not a substitute for that smoke.
|
| 122 |
|
| 123 |
Current smoke details are tracked in `docs/smoke/2026-06-05-space-deployment-smoke.md`.
|
| 124 |
|
docs/goal-prompts/2026-06-06-modal-minicpm-backend.md
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Modal MiniCPM Backend Goal Prompt
|
| 2 |
+
|
| 3 |
+
在 `/Users/junhaocheng/working-dir/ai-competitions/build-small-hackthon` 继续 Dream Customs。先读 `AGENTS.md`、`docs/spec.md`、`docs/prd.md`、`docs/handoff.md` 和 `docs/superpowers/plans/2026-06-06-modal-minicpm-backend.md`。
|
| 4 |
+
|
| 5 |
+
执行计划:用 `feature/modal-minicpm-backend` 分支,按任务勾选推进,部署 Modal 作为隐藏 MiniCPM 后端,HF Space 继续做 Gradio 前台。
|
| 6 |
+
|
| 7 |
+
硬约束:严禁把任何 token、endpoint secret、HF token、Modal token 写入仓库、日志、文档或截图;只记录变量名和非敏感 smoke 结果。开工前先确认 HF Space secrets 是否已存在;缺失或需要用户填写时立刻停下请求用户处理。默认 backend 保持 `demo`,`model` route 只能作为可切换真实推理路径,失败必须 fallback。MiniCPM-V-4.6 vision route 是硬验收,不允许用 demo vision 或跳过 vision smoke 代替;如果 GPU 不够,升级 Modal GPU,但不要换非 MiniCPM-V-4.6 模型。不要扩展到非 MiniCPM 模型;语音 ASR 不在本轮范围。
|
| 8 |
+
|
| 9 |
+
验收:每个任务完成后运行相应 pytest/smoke,更新计划 checkbox,分步提交;最后 push 分支并汇报 Modal text smoke、Modal MiniCPM-V-4.6 vision smoke、12 条 eval、Space model route 和 demo fallback 结果。
|
docs/handoff.md
CHANGED
|
@@ -133,6 +133,21 @@ This directory is now a dedicated Dream Customs repository:
|
|
| 133 |
|
| 134 |
Continue to confirm `git remote -v` before pushing, but the earlier VLA remote warning is no longer the active state.
|
| 135 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 136 |
## Open Questions For User
|
| 137 |
|
| 138 |
These do not block initial scaffolding:
|
|
|
|
| 133 |
|
| 134 |
Continue to confirm `git remote -v` before pushing, but the earlier VLA remote warning is no longer the active state.
|
| 135 |
|
| 136 |
+
## Modal Backend Route
|
| 137 |
+
|
| 138 |
+
The intended production-quality route is:
|
| 139 |
+
|
| 140 |
+
```text
|
| 141 |
+
Hugging Face Space Gradio UI
|
| 142 |
+
-> HostedMiniCPMTextClient / HostedMiniCPMVisionClient
|
| 143 |
+
-> private Modal endpoints
|
| 144 |
+
-> MiniCPM5-1B text generation and MiniCPM-V-4.6 visual clue extraction
|
| 145 |
+
```
|
| 146 |
+
|
| 147 |
+
The Space defaults to `model` for both text and vision backends, so a configured Space calls Modal immediately. The `demo` backend remains available in developer settings as the deterministic fallback path.
|
| 148 |
+
|
| 149 |
+
The public Space can run on ZeroGPU after `dream_customs.zerogpu` registers the lightweight `@spaces.GPU` startup probe. Modal credits still pay for the hidden L4 GPU backend; the ZeroGPU probe is only there to make the HF hardware setting valid for this Gradio frontend.
|
| 150 |
+
|
| 151 |
## Open Questions For User
|
| 152 |
|
| 153 |
These do not block initial scaffolding:
|
docs/smoke/2026-06-06-modal-minicpm-backend-smoke.md
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Modal MiniCPM Backend Smoke - 2026-06-06
|
| 2 |
+
|
| 3 |
+
## Scope
|
| 4 |
+
|
| 5 |
+
- Modal app: `dream-customs-minicpm-backend`
|
| 6 |
+
- Text model: `openbmb/MiniCPM5-1B`
|
| 7 |
+
- Vision model: `openbmb/MiniCPM-V-4.6`
|
| 8 |
+
- Public UI remains Hugging Face Space `build-small-hackathon/dream-customs`
|
| 9 |
+
- Secrets and endpoint values were not printed, documented, or committed.
|
| 10 |
+
|
| 11 |
+
## Results
|
| 12 |
+
|
| 13 |
+
- Health endpoint: PASS
|
| 14 |
+
- Text smoke: PASS with `text_route=ok` and `text_questions=2`
|
| 15 |
+
- Vision smoke: PASS with `vision_route=ok` and `vision_clues=8`
|
| 16 |
+
- Acceptance eval: PASS with `passed=12`, `total=12`, `schema_valid_rate=1.0`, and `failures=[]`
|
| 17 |
+
- Hugging Face Space model route: PASS with `status=negotiating`, `text_backend=model`, `vision_backend=model`, one extracted image evidence item, and `visual_clue_count=8`
|
| 18 |
+
- Hugging Face Space demo fallback: PASS with `status=negotiating`, `text_backend=demo`, `vision_backend=demo`, one extracted image evidence item, and `visual_clue_count=3`
|
| 19 |
+
- Hugging Face Space hardware recovery: PASS after switching the Space from ZeroGPU back to `CPU basic`; public model route and demo fallback were re-smoked successfully.
|
| 20 |
+
- ZeroGPU compatibility fix: PASS after adding the `@spaces.GPU` startup probe and making `model` the default backend.
|
| 21 |
+
- Hugging Face Space ZeroGPU default model route: PASS with `hardware=zero-a10g`, `status=negotiating`, `text_backend=model`, `vision_backend=model`, one extracted image evidence item, and `visual_clue_count=8`
|
| 22 |
+
- Hugging Face Space ZeroGPU demo fallback: PASS with `hardware=zero-a10g`, `status=negotiating`, `text_backend=demo`, `vision_backend=demo`, one extracted image evidence item, and `visual_clue_count=3`
|
| 23 |
+
|
| 24 |
+
## Notes
|
| 25 |
+
|
| 26 |
+
- GPU class remained `L4`; no upgrade was required for `openbmb/MiniCPM-V-4.6`.
|
| 27 |
+
- Modal text output keeps the real MiniCPM route first and repairs missing JSON fields only when schema validation would otherwise fail.
|
| 28 |
+
- Hugging Face Space secret sync was first attempted through the local HF token, but the API returned a non-secret `403 Forbidden: Authorization error`.
|
| 29 |
+
- The three Space secrets were replaced through the logged-in Hugging Face UI, then the Space was restarted and verified through the public Gradio API.
|
| 30 |
+
- The public Gradio Space can run on ZeroGPU once the lightweight `@spaces.GPU` startup probe is present. Modal credits still cover the L4 GPU inference backend, and `model` is now the default text and vision backend.
|
docs/superpowers/plans/2026-06-06-modal-minicpm-backend.md
ADDED
|
@@ -0,0 +1,1226 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Modal MiniCPM Backend Implementation Plan
|
| 2 |
+
|
| 3 |
+
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
| 4 |
+
|
| 5 |
+
**Goal:** Use the available Modal credits to deploy real MiniCPM text and vision inference endpoints, then connect the existing Hugging Face Space UI through private runtime secrets while preserving the deterministic demo fallback.
|
| 6 |
+
|
| 7 |
+
**Architecture:** Keep Hugging Face Space as the public Gradio frontend and use Modal as a hidden GPU backend. Modal exposes one text endpoint for `openbmb/MiniCPM5-1B` JSON generation and one vision endpoint for `openbmb/MiniCPM-V-4.6` visual clue extraction. The existing `HostedMiniCPMTextClient` and `HostedMiniCPMVisionClient` continue to call plain JSON HTTP endpoints through `DREAM_CUSTOMS_TEXT_ENDPOINT`, `DREAM_CUSTOMS_VISION_ENDPOINT`, and `DREAM_CUSTOMS_HOSTED_TOKEN`.
|
| 8 |
+
|
| 9 |
+
**Tech Stack:** Python, Modal, FastAPI endpoint decorators, Transformers, PyTorch, Pillow, Gradio, Hugging Face Space repository secrets, pytest.
|
| 10 |
+
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
## Scope And Non-Negotiables
|
| 14 |
+
|
| 15 |
+
- Do not replace the public Hugging Face Space with Modal.
|
| 16 |
+
- Do not store endpoint tokens, Hugging Face tokens, Modal tokens, or secret values in git, docs, logs, screenshots, or example files.
|
| 17 |
+
- Keep the UI default backend as `demo`.
|
| 18 |
+
- Keep text-only demo fallback working when Modal endpoints are missing, cold, slow, or failing.
|
| 19 |
+
- Keep the model family constrained to `MiniCPM` before trying unrelated small models.
|
| 20 |
+
- Completion requires a working `MiniCPM-V-4.6` vision route. Demo vision fallback is allowed only as runtime resilience after the real vision route has passed smoke; it is not an acceptable substitute for final delivery.
|
| 21 |
+
- Confirm Hugging Face Space secrets at the start of execution. If the Space is missing required secrets or needs updated Modal endpoint values, stop early and ask the user to fill or approve filling them before remote Space verification.
|
| 22 |
+
- Treat voice transcription as unchanged for this pass; ASR remains fake/demo unless a separate plan approves a real ASR adapter.
|
| 23 |
+
|
| 24 |
+
## File Structure
|
| 25 |
+
|
| 26 |
+
- Create `modal_backend/__init__.py`: package marker for Modal backend helpers.
|
| 27 |
+
- Create `modal_backend/contracts.py`: pure request/response helpers that are importable by tests without importing `modal`, `torch`, or `transformers`.
|
| 28 |
+
- Create `modal_backend/dream_customs_modal.py`: Modal app definition, GPU classes, model loading, authentication, `/text`, `/vision`, and health endpoint.
|
| 29 |
+
- Create `tests/test_modal_contract.py`: fast local contract tests for request parsing, image decoding, auth behavior, and response shape.
|
| 30 |
+
- Create `scripts/smoke_hosted_routes.py`: local smoke script that calls configured hosted endpoints without printing secret values.
|
| 31 |
+
- Create `scripts/eval_hosted_routes.py`: small acceptance evaluator for schema validity, safety edge cases, and vision clue extraction.
|
| 32 |
+
- Modify `README.md`: add Modal deployment and HF Space secret setup instructions.
|
| 33 |
+
- Modify `docs/handoff.md`: record the intended production route and fallback contract.
|
| 34 |
+
- Modify `docs/smoke/2026-06-06-modal-minicpm-backend-smoke.md`: capture smoke results after deployment.
|
| 35 |
+
|
| 36 |
+
## Task 0: Prepare The Branch And Baseline
|
| 37 |
+
|
| 38 |
+
**Files:**
|
| 39 |
+
- Read: `AGENTS.md`
|
| 40 |
+
- Read: `docs/spec.md`
|
| 41 |
+
- Read: `docs/prd.md`
|
| 42 |
+
- Read: `docs/handoff.md`
|
| 43 |
+
- Read: `dream_customs/models.py`
|
| 44 |
+
- Read: `dream_customs/app_logic.py`
|
| 45 |
+
|
| 46 |
+
- [x] **Step 1: Confirm repository and branch**
|
| 47 |
+
|
| 48 |
+
Run:
|
| 49 |
+
|
| 50 |
+
```bash
|
| 51 |
+
pwd
|
| 52 |
+
git status --short --branch
|
| 53 |
+
git remote -v
|
| 54 |
+
```
|
| 55 |
+
|
| 56 |
+
Expected:
|
| 57 |
+
|
| 58 |
+
```text
|
| 59 |
+
/Users/junhaocheng/working-dir/ai-competitions/build-small-hackthon
|
| 60 |
+
## feature/dream-customs-mvp...origin/feature/dream-customs-mvp
|
| 61 |
+
origin https://github.com/adjcjh777/dream-customs-build-small.git (fetch)
|
| 62 |
+
origin https://github.com/adjcjh777/dream-customs-build-small.git (push)
|
| 63 |
+
```
|
| 64 |
+
|
| 65 |
+
- [x] **Step 2: Sync latest code**
|
| 66 |
+
|
| 67 |
+
Run:
|
| 68 |
+
|
| 69 |
+
```bash
|
| 70 |
+
git pull
|
| 71 |
+
```
|
| 72 |
+
|
| 73 |
+
Expected:
|
| 74 |
+
|
| 75 |
+
```text
|
| 76 |
+
Already up to date.
|
| 77 |
+
```
|
| 78 |
+
|
| 79 |
+
If remote changes arrive, read them before editing and keep this plan aligned with the current file names.
|
| 80 |
+
|
| 81 |
+
- [x] **Step 3: Create implementation branch**
|
| 82 |
+
|
| 83 |
+
Run:
|
| 84 |
+
|
| 85 |
+
```bash
|
| 86 |
+
git switch -c feature/modal-minicpm-backend
|
| 87 |
+
```
|
| 88 |
+
|
| 89 |
+
Expected:
|
| 90 |
+
|
| 91 |
+
```text
|
| 92 |
+
Switched to a new branch 'feature/modal-minicpm-backend'
|
| 93 |
+
```
|
| 94 |
+
|
| 95 |
+
- [x] **Step 4: Run baseline tests**
|
| 96 |
+
|
| 97 |
+
Run:
|
| 98 |
+
|
| 99 |
+
```bash
|
| 100 |
+
.venv/bin/python -m pytest -q
|
| 101 |
+
```
|
| 102 |
+
|
| 103 |
+
Expected: all existing tests pass.
|
| 104 |
+
|
| 105 |
+
- [x] **Step 5: Confirm Hugging Face Space secret readiness**
|
| 106 |
+
|
| 107 |
+
Before writing backend code, open the Hugging Face Space settings for `build-small-hackathon/dream-customs` and confirm whether these repository secrets already exist:
|
| 108 |
+
|
| 109 |
+
```text
|
| 110 |
+
DREAM_CUSTOMS_TEXT_ENDPOINT
|
| 111 |
+
DREAM_CUSTOMS_VISION_ENDPOINT
|
| 112 |
+
DREAM_CUSTOMS_HOSTED_TOKEN
|
| 113 |
+
```
|
| 114 |
+
|
| 115 |
+
Expected: the secrets exist, or the worker has stopped and asked the user to fill them before public Space verification. Do not print or copy secret values. If the Modal endpoint URLs will be newly generated later, record only this non-secret status in your notes:
|
| 116 |
+
|
| 117 |
+
```text
|
| 118 |
+
HF Space secrets preflight: present, may need endpoint value refresh after Modal deploy.
|
| 119 |
+
```
|
| 120 |
+
|
| 121 |
+
Commit is not needed for this task.
|
| 122 |
+
|
| 123 |
+
Observed 2026-06-06: required secret keys/value readiness was confirmed by the user. Endpoint values still need refresh after Modal deploy. No secret values were recorded.
|
| 124 |
+
|
| 125 |
+
## Task 1: Add Pure Modal Endpoint Contract Helpers
|
| 126 |
+
|
| 127 |
+
**Files:**
|
| 128 |
+
- Create: `modal_backend/__init__.py`
|
| 129 |
+
- Create: `modal_backend/contracts.py`
|
| 130 |
+
- Create: `tests/test_modal_contract.py`
|
| 131 |
+
|
| 132 |
+
- [x] **Step 1: Write failing contract tests**
|
| 133 |
+
|
| 134 |
+
Create `tests/test_modal_contract.py`:
|
| 135 |
+
|
| 136 |
+
```python
|
| 137 |
+
import base64
|
| 138 |
+
|
| 139 |
+
from modal_backend.contracts import (
|
| 140 |
+
AuthError,
|
| 141 |
+
decode_image_payload,
|
| 142 |
+
ensure_authorized,
|
| 143 |
+
normalize_text_payload,
|
| 144 |
+
response_payload,
|
| 145 |
+
)
|
| 146 |
+
|
| 147 |
+
|
| 148 |
+
def test_normalize_text_payload_accepts_prompt():
|
| 149 |
+
payload = normalize_text_payload({"prompt": "Return JSON.", "max_tokens": 123})
|
| 150 |
+
assert payload["prompt"] == "Return JSON."
|
| 151 |
+
assert payload["max_tokens"] == 123
|
| 152 |
+
assert payload["temperature"] == 0.2
|
| 153 |
+
|
| 154 |
+
|
| 155 |
+
def test_normalize_text_payload_accepts_openai_style_messages():
|
| 156 |
+
payload = normalize_text_payload(
|
| 157 |
+
{"messages": [{"role": "user", "content": "Dream of an elevator."}]}
|
| 158 |
+
)
|
| 159 |
+
assert payload["prompt"] == "Dream of an elevator."
|
| 160 |
+
assert payload["max_tokens"] == 700
|
| 161 |
+
|
| 162 |
+
|
| 163 |
+
def test_normalize_text_payload_clamps_max_tokens():
|
| 164 |
+
payload = normalize_text_payload({"prompt": "x", "max_tokens": 9000})
|
| 165 |
+
assert payload["max_tokens"] == 1200
|
| 166 |
+
|
| 167 |
+
|
| 168 |
+
def test_decode_image_payload_accepts_image_key():
|
| 169 |
+
encoded = base64.b64encode(b"fake-image-bytes").decode("ascii")
|
| 170 |
+
assert decode_image_payload({"image": encoded}) == b"fake-image-bytes"
|
| 171 |
+
|
| 172 |
+
|
| 173 |
+
def test_decode_image_payload_accepts_images_list():
|
| 174 |
+
encoded = base64.b64encode(b"fake-image-bytes").decode("ascii")
|
| 175 |
+
assert decode_image_payload({"images": [encoded]}) == b"fake-image-bytes"
|
| 176 |
+
|
| 177 |
+
|
| 178 |
+
def test_response_payload_uses_existing_client_shape():
|
| 179 |
+
assert response_payload("hello") == {"response": "hello"}
|
| 180 |
+
|
| 181 |
+
|
| 182 |
+
def test_ensure_authorized_allows_empty_expected_token_for_local_smoke():
|
| 183 |
+
ensure_authorized("", "")
|
| 184 |
+
|
| 185 |
+
|
| 186 |
+
def test_ensure_authorized_accepts_bearer_token():
|
| 187 |
+
ensure_authorized("Bearer secret-value", "secret-value")
|
| 188 |
+
|
| 189 |
+
|
| 190 |
+
def test_ensure_authorized_rejects_wrong_token():
|
| 191 |
+
try:
|
| 192 |
+
ensure_authorized("Bearer wrong", "secret-value")
|
| 193 |
+
except AuthError as exc:
|
| 194 |
+
assert str(exc) == "Unauthorized hosted route request."
|
| 195 |
+
else:
|
| 196 |
+
raise AssertionError("Expected AuthError")
|
| 197 |
+
```
|
| 198 |
+
|
| 199 |
+
- [x] **Step 2: Run tests to verify failure**
|
| 200 |
+
|
| 201 |
+
Run:
|
| 202 |
+
|
| 203 |
+
```bash
|
| 204 |
+
.venv/bin/python -m pytest tests/test_modal_contract.py -q
|
| 205 |
+
```
|
| 206 |
+
|
| 207 |
+
Expected:
|
| 208 |
+
|
| 209 |
+
```text
|
| 210 |
+
ModuleNotFoundError: No module named 'modal_backend'
|
| 211 |
+
```
|
| 212 |
+
|
| 213 |
+
- [x] **Step 3: Create package marker**
|
| 214 |
+
|
| 215 |
+
Create `modal_backend/__init__.py`:
|
| 216 |
+
|
| 217 |
+
```python
|
| 218 |
+
"""Modal backend package for Dream Customs."""
|
| 219 |
+
```
|
| 220 |
+
|
| 221 |
+
- [x] **Step 4: Implement pure endpoint contract helpers**
|
| 222 |
+
|
| 223 |
+
Create `modal_backend/contracts.py`:
|
| 224 |
+
|
| 225 |
+
```python
|
| 226 |
+
import base64
|
| 227 |
+
from typing import Any, Dict
|
| 228 |
+
|
| 229 |
+
|
| 230 |
+
class AuthError(Exception):
|
| 231 |
+
"""Raised when a hosted route request is not authorized."""
|
| 232 |
+
|
| 233 |
+
|
| 234 |
+
def _clean_text(value: Any) -> str:
|
| 235 |
+
if value is None:
|
| 236 |
+
return ""
|
| 237 |
+
return str(value).strip()
|
| 238 |
+
|
| 239 |
+
|
| 240 |
+
def normalize_text_payload(payload: Dict[str, Any]) -> Dict[str, Any]:
|
| 241 |
+
prompt = _clean_text(payload.get("prompt"))
|
| 242 |
+
if not prompt:
|
| 243 |
+
messages = payload.get("messages")
|
| 244 |
+
if isinstance(messages, list):
|
| 245 |
+
parts = []
|
| 246 |
+
for item in messages:
|
| 247 |
+
if isinstance(item, dict):
|
| 248 |
+
content = _clean_text(item.get("content"))
|
| 249 |
+
if content:
|
| 250 |
+
parts.append(content)
|
| 251 |
+
prompt = "\n".join(parts).strip()
|
| 252 |
+
max_tokens = payload.get("max_tokens", 700)
|
| 253 |
+
try:
|
| 254 |
+
max_tokens = int(max_tokens)
|
| 255 |
+
except (TypeError, ValueError):
|
| 256 |
+
max_tokens = 700
|
| 257 |
+
max_tokens = max(64, min(max_tokens, 1200))
|
| 258 |
+
temperature = payload.get("temperature", 0.2)
|
| 259 |
+
try:
|
| 260 |
+
temperature = float(temperature)
|
| 261 |
+
except (TypeError, ValueError):
|
| 262 |
+
temperature = 0.2
|
| 263 |
+
temperature = max(0.0, min(temperature, 0.7))
|
| 264 |
+
return {
|
| 265 |
+
"prompt": prompt,
|
| 266 |
+
"max_tokens": max_tokens,
|
| 267 |
+
"temperature": temperature,
|
| 268 |
+
}
|
| 269 |
+
|
| 270 |
+
|
| 271 |
+
def decode_image_payload(payload: Dict[str, Any]) -> bytes:
|
| 272 |
+
encoded = payload.get("image")
|
| 273 |
+
if not encoded and isinstance(payload.get("images"), list) and payload["images"]:
|
| 274 |
+
encoded = payload["images"][0]
|
| 275 |
+
if not encoded:
|
| 276 |
+
raise ValueError("Missing image payload.")
|
| 277 |
+
if isinstance(encoded, bytes):
|
| 278 |
+
encoded = encoded.decode("ascii")
|
| 279 |
+
return base64.b64decode(str(encoded))
|
| 280 |
+
|
| 281 |
+
|
| 282 |
+
def ensure_authorized(authorization_header: str, expected_token: str) -> None:
|
| 283 |
+
expected_token = expected_token.strip()
|
| 284 |
+
if not expected_token:
|
| 285 |
+
return
|
| 286 |
+
header = authorization_header.strip()
|
| 287 |
+
if header.lower().startswith("bearer "):
|
| 288 |
+
supplied = header.split(" ", 1)[1].strip()
|
| 289 |
+
else:
|
| 290 |
+
supplied = header
|
| 291 |
+
if supplied != expected_token:
|
| 292 |
+
raise AuthError("Unauthorized hosted route request.")
|
| 293 |
+
|
| 294 |
+
|
| 295 |
+
def response_payload(text: str) -> Dict[str, str]:
|
| 296 |
+
return {"response": text}
|
| 297 |
+
```
|
| 298 |
+
|
| 299 |
+
- [x] **Step 5: Run contract tests**
|
| 300 |
+
|
| 301 |
+
Run:
|
| 302 |
+
|
| 303 |
+
```bash
|
| 304 |
+
.venv/bin/python -m pytest tests/test_modal_contract.py -q
|
| 305 |
+
```
|
| 306 |
+
|
| 307 |
+
Expected:
|
| 308 |
+
|
| 309 |
+
```text
|
| 310 |
+
9 passed
|
| 311 |
+
```
|
| 312 |
+
|
| 313 |
+
- [x] **Step 6: Run full tests**
|
| 314 |
+
|
| 315 |
+
Run:
|
| 316 |
+
|
| 317 |
+
```bash
|
| 318 |
+
.venv/bin/python -m pytest -q
|
| 319 |
+
```
|
| 320 |
+
|
| 321 |
+
Expected: all tests pass.
|
| 322 |
+
|
| 323 |
+
- [x] **Step 7: Commit contract helpers**
|
| 324 |
+
|
| 325 |
+
Run:
|
| 326 |
+
|
| 327 |
+
```bash
|
| 328 |
+
git add modal_backend/__init__.py modal_backend/contracts.py tests/test_modal_contract.py
|
| 329 |
+
git commit -m "test: add modal endpoint contract helpers"
|
| 330 |
+
```
|
| 331 |
+
|
| 332 |
+
## Task 2: Add Modal Text And Vision Services
|
| 333 |
+
|
| 334 |
+
**Files:**
|
| 335 |
+
- Create: `modal_backend/dream_customs_modal.py`
|
| 336 |
+
- Modify: `requirements.txt`
|
| 337 |
+
|
| 338 |
+
- [x] **Step 1: Decide dependency placement**
|
| 339 |
+
|
| 340 |
+
Keep Modal-only dependencies inside the Modal image in `modal_backend/dream_customs_modal.py`. Do not add `modal`, `torch`, or heavy Transformers-only deployment packages to `requirements.txt` unless local tests need them. The public Hugging Face Space should stay lightweight.
|
| 341 |
+
|
| 342 |
+
- [x] **Step 2: Create Modal service file**
|
| 343 |
+
|
| 344 |
+
Create `modal_backend/dream_customs_modal.py`:
|
| 345 |
+
|
| 346 |
+
```python
|
| 347 |
+
import io
|
| 348 |
+
import os
|
| 349 |
+
import tempfile
|
| 350 |
+
from typing import Any, Dict
|
| 351 |
+
|
| 352 |
+
import modal
|
| 353 |
+
|
| 354 |
+
from modal_backend.contracts import (
|
| 355 |
+
AuthError,
|
| 356 |
+
decode_image_payload,
|
| 357 |
+
ensure_authorized,
|
| 358 |
+
normalize_text_payload,
|
| 359 |
+
response_payload,
|
| 360 |
+
)
|
| 361 |
+
|
| 362 |
+
|
| 363 |
+
APP_NAME = "dream-customs-minicpm-backend"
|
| 364 |
+
TEXT_MODEL = "openbmb/MiniCPM5-1B"
|
| 365 |
+
VISION_MODEL = "openbmb/MiniCPM-V-4.6"
|
| 366 |
+
MINUTES = 60
|
| 367 |
+
|
| 368 |
+
app = modal.App(APP_NAME)
|
| 369 |
+
|
| 370 |
+
hf_cache = modal.Volume.from_name("dream-customs-hf-cache", create_if_missing=True)
|
| 371 |
+
|
| 372 |
+
image = (
|
| 373 |
+
modal.Image.debian_slim(python_version="3.11")
|
| 374 |
+
.apt_install("git")
|
| 375 |
+
.pip_install(
|
| 376 |
+
"accelerate",
|
| 377 |
+
"einops",
|
| 378 |
+
"fastapi[standard]",
|
| 379 |
+
"pillow",
|
| 380 |
+
"protobuf",
|
| 381 |
+
"sentencepiece",
|
| 382 |
+
"torch",
|
| 383 |
+
"torchvision",
|
| 384 |
+
"transformers>=4.56",
|
| 385 |
+
)
|
| 386 |
+
)
|
| 387 |
+
|
| 388 |
+
secrets = [modal.Secret.from_name("dream-customs-modal-secrets")]
|
| 389 |
+
|
| 390 |
+
|
| 391 |
+
def _auth_header(request: Any) -> str:
|
| 392 |
+
return str(request.headers.get("authorization", ""))
|
| 393 |
+
|
| 394 |
+
|
| 395 |
+
def _expected_token() -> str:
|
| 396 |
+
return os.getenv("DREAM_CUSTOMS_HOSTED_TOKEN", "").strip()
|
| 397 |
+
|
| 398 |
+
|
| 399 |
+
def _json_error(message: str, status: str = "error") -> Dict[str, str]:
|
| 400 |
+
return {"status": status, "response": "", "error": message}
|
| 401 |
+
|
| 402 |
+
|
| 403 |
+
@app.function(image=image, secrets=secrets)
|
| 404 |
+
@modal.fastapi_endpoint(method="GET", docs=True)
|
| 405 |
+
def health() -> Dict[str, str]:
|
| 406 |
+
return {
|
| 407 |
+
"status": "ok",
|
| 408 |
+
"app": APP_NAME,
|
| 409 |
+
"text_model": TEXT_MODEL,
|
| 410 |
+
"vision_model": VISION_MODEL,
|
| 411 |
+
}
|
| 412 |
+
|
| 413 |
+
|
| 414 |
+
@app.cls(
|
| 415 |
+
image=image,
|
| 416 |
+
gpu="L4",
|
| 417 |
+
timeout=10 * MINUTES,
|
| 418 |
+
scaledown_window=5 * MINUTES,
|
| 419 |
+
volumes={"/root/.cache/huggingface": hf_cache},
|
| 420 |
+
secrets=secrets,
|
| 421 |
+
)
|
| 422 |
+
class TextService:
|
| 423 |
+
@modal.enter()
|
| 424 |
+
def load(self):
|
| 425 |
+
from transformers import pipeline
|
| 426 |
+
|
| 427 |
+
self.pipe = pipeline(
|
| 428 |
+
"text-generation",
|
| 429 |
+
model=os.getenv("DREAM_CUSTOMS_TEXT_MODEL", TEXT_MODEL),
|
| 430 |
+
device_map="auto",
|
| 431 |
+
torch_dtype="auto",
|
| 432 |
+
trust_remote_code=True,
|
| 433 |
+
)
|
| 434 |
+
|
| 435 |
+
@modal.fastapi_endpoint(method="POST", docs=True)
|
| 436 |
+
async def text(self, request):
|
| 437 |
+
try:
|
| 438 |
+
ensure_authorized(_auth_header(request), _expected_token())
|
| 439 |
+
except AuthError as exc:
|
| 440 |
+
return _json_error(str(exc), status="unauthorized")
|
| 441 |
+
payload = await request.json()
|
| 442 |
+
normalized = normalize_text_payload(payload)
|
| 443 |
+
if not normalized["prompt"]:
|
| 444 |
+
return _json_error("Missing prompt.")
|
| 445 |
+
output = self.pipe(
|
| 446 |
+
normalized["prompt"],
|
| 447 |
+
max_new_tokens=normalized["max_tokens"],
|
| 448 |
+
do_sample=normalized["temperature"] > 0,
|
| 449 |
+
temperature=max(normalized["temperature"], 0.01),
|
| 450 |
+
return_full_text=False,
|
| 451 |
+
)
|
| 452 |
+
text = str(output[0].get("generated_text", "")).strip()
|
| 453 |
+
return response_payload(text)
|
| 454 |
+
|
| 455 |
+
|
| 456 |
+
@app.cls(
|
| 457 |
+
image=image,
|
| 458 |
+
gpu="L4",
|
| 459 |
+
timeout=10 * MINUTES,
|
| 460 |
+
scaledown_window=5 * MINUTES,
|
| 461 |
+
volumes={"/root/.cache/huggingface": hf_cache},
|
| 462 |
+
secrets=secrets,
|
| 463 |
+
)
|
| 464 |
+
class VisionService:
|
| 465 |
+
@modal.enter()
|
| 466 |
+
def load(self):
|
| 467 |
+
from transformers import pipeline
|
| 468 |
+
|
| 469 |
+
self.pipe = pipeline(
|
| 470 |
+
"image-text-to-text",
|
| 471 |
+
model=os.getenv("DREAM_CUSTOMS_VISION_MODEL", VISION_MODEL),
|
| 472 |
+
device_map="auto",
|
| 473 |
+
torch_dtype="auto",
|
| 474 |
+
trust_remote_code=True,
|
| 475 |
+
)
|
| 476 |
+
|
| 477 |
+
@modal.fastapi_endpoint(method="POST", docs=True)
|
| 478 |
+
async def vision(self, request):
|
| 479 |
+
try:
|
| 480 |
+
ensure_authorized(_auth_header(request), _expected_token())
|
| 481 |
+
except AuthError as exc:
|
| 482 |
+
return _json_error(str(exc), status="unauthorized")
|
| 483 |
+
payload = await request.json()
|
| 484 |
+
prompt = str(payload.get("prompt") or "").strip()
|
| 485 |
+
if not prompt:
|
| 486 |
+
prompt = (
|
| 487 |
+
"Extract concise dream-like visual clues from this image. "
|
| 488 |
+
"Return a single JSON object with keys objects, places, visible_text, "
|
| 489 |
+
"colors, mood_cues, and uncertain_details. Do not diagnose."
|
| 490 |
+
)
|
| 491 |
+
try:
|
| 492 |
+
image_bytes = decode_image_payload(payload)
|
| 493 |
+
except ValueError as exc:
|
| 494 |
+
return _json_error(str(exc))
|
| 495 |
+
from PIL import Image
|
| 496 |
+
|
| 497 |
+
pil_image = Image.open(io.BytesIO(image_bytes)).convert("RGB")
|
| 498 |
+
with tempfile.NamedTemporaryFile(suffix=".png") as temp_file:
|
| 499 |
+
pil_image.save(temp_file.name)
|
| 500 |
+
messages = [
|
| 501 |
+
{
|
| 502 |
+
"role": "user",
|
| 503 |
+
"content": [
|
| 504 |
+
{"type": "image", "path": temp_file.name},
|
| 505 |
+
{"type": "text", "text": prompt},
|
| 506 |
+
],
|
| 507 |
+
}
|
| 508 |
+
]
|
| 509 |
+
result = self.pipe(text=messages, max_new_tokens=int(payload.get("max_tokens", 320)))
|
| 510 |
+
return response_payload(str(result).strip())
|
| 511 |
+
```
|
| 512 |
+
|
| 513 |
+
- [x] **Step 3: Run import check without requiring Modal locally**
|
| 514 |
+
|
| 515 |
+
Run:
|
| 516 |
+
|
| 517 |
+
```bash
|
| 518 |
+
.venv/bin/python - <<'PY'
|
| 519 |
+
from modal_backend.contracts import response_payload
|
| 520 |
+
print(response_payload("ok"))
|
| 521 |
+
PY
|
| 522 |
+
```
|
| 523 |
+
|
| 524 |
+
Expected:
|
| 525 |
+
|
| 526 |
+
```text
|
| 527 |
+
{'response': 'ok'}
|
| 528 |
+
```
|
| 529 |
+
|
| 530 |
+
- [x] **Step 4: Run tests**
|
| 531 |
+
|
| 532 |
+
Run:
|
| 533 |
+
|
| 534 |
+
```bash
|
| 535 |
+
.venv/bin/python -m pytest -q
|
| 536 |
+
```
|
| 537 |
+
|
| 538 |
+
Expected: all tests pass.
|
| 539 |
+
|
| 540 |
+
- [x] **Step 5: Commit Modal service scaffold**
|
| 541 |
+
|
| 542 |
+
Run:
|
| 543 |
+
|
| 544 |
+
```bash
|
| 545 |
+
git add modal_backend/dream_customs_modal.py
|
| 546 |
+
git commit -m "feat: add modal minicpm backend service"
|
| 547 |
+
```
|
| 548 |
+
|
| 549 |
+
## Task 3: Validate Modal Service Locally With `modal serve`
|
| 550 |
+
|
| 551 |
+
**Files:**
|
| 552 |
+
- Read: `modal_backend/dream_customs_modal.py`
|
| 553 |
+
|
| 554 |
+
- [x] **Step 1: Confirm Modal CLI is authenticated**
|
| 555 |
+
|
| 556 |
+
Run:
|
| 557 |
+
|
| 558 |
+
```bash
|
| 559 |
+
modal token set --help >/dev/null && modal profile current
|
| 560 |
+
```
|
| 561 |
+
|
| 562 |
+
Expected: a Modal profile for workspace `cjh-12569` or a clear CLI message requiring login.
|
| 563 |
+
|
| 564 |
+
If login is required, run:
|
| 565 |
+
|
| 566 |
+
```bash
|
| 567 |
+
modal setup
|
| 568 |
+
```
|
| 569 |
+
|
| 570 |
+
Complete browser login for workspace `cjh-12569`.
|
| 571 |
+
|
| 572 |
+
- [x] **Step 2: Create Modal secret without printing values**
|
| 573 |
+
|
| 574 |
+
Run with values pasted only into the command invocation, not into a file:
|
| 575 |
+
|
| 576 |
+
```bash
|
| 577 |
+
modal secret create dream-customs-modal-secrets \
|
| 578 |
+
HF_TOKEN="$HF_TOKEN" \
|
| 579 |
+
DREAM_CUSTOMS_HOSTED_TOKEN="$DREAM_CUSTOMS_HOSTED_TOKEN"
|
| 580 |
+
```
|
| 581 |
+
|
| 582 |
+
Expected:
|
| 583 |
+
|
| 584 |
+
```text
|
| 585 |
+
Created secret dream-customs-modal-secrets
|
| 586 |
+
```
|
| 587 |
+
|
| 588 |
+
If the secret already exists, update it from the Modal dashboard or recreate it after confirming no unrelated app depends on it.
|
| 589 |
+
|
| 590 |
+
Observed 2026-06-06: Modal CLI authenticated to workspace `cjh-12569`; Modal secret `dream-customs-modal-secrets` exists. Secret values were not printed or recorded. The hosted token still needs to be synchronized with the Hugging Face Space secret before public Space model-route verification.
|
| 591 |
+
|
| 592 |
+
- [x] **Step 3: Serve or deploy the Modal app for development validation**
|
| 593 |
+
|
| 594 |
+
Run:
|
| 595 |
+
|
| 596 |
+
```bash
|
| 597 |
+
modal serve modal_backend/dream_customs_modal.py
|
| 598 |
+
```
|
| 599 |
+
|
| 600 |
+
Expected: Modal prints development endpoint URLs for `health`, `text`, and `vision`. Do not paste secret-bearing URLs or tokens into logs.
|
| 601 |
+
|
| 602 |
+
Observed 2026-06-06: skipped long-running `modal serve` in favor of `modal deploy` plus SDK endpoint discovery. Endpoint URLs were kept in-process only and were not written to docs, logs, screenshots, or git.
|
| 603 |
+
|
| 604 |
+
- [x] **Step 4: Check health endpoint**
|
| 605 |
+
|
| 606 |
+
In a second terminal, run with the health URL copied from Modal output:
|
| 607 |
+
|
| 608 |
+
```bash
|
| 609 |
+
curl -s "$DREAM_CUSTOMS_MODAL_HEALTH_URL" | .venv/bin/python -m json.tool
|
| 610 |
+
```
|
| 611 |
+
|
| 612 |
+
Expected:
|
| 613 |
+
|
| 614 |
+
```json
|
| 615 |
+
{
|
| 616 |
+
"status": "ok",
|
| 617 |
+
"app": "dream-customs-minicpm-backend",
|
| 618 |
+
"text_model": "openbmb/MiniCPM5-1B",
|
| 619 |
+
"vision_model": "openbmb/MiniCPM-V-4.6"
|
| 620 |
+
}
|
| 621 |
+
```
|
| 622 |
+
|
| 623 |
+
Observed 2026-06-06: health smoke passed after deployment.
|
| 624 |
+
|
| 625 |
+
- [x] **Step 5: Smoke text endpoint with token-safe output**
|
| 626 |
+
|
| 627 |
+
Run:
|
| 628 |
+
|
| 629 |
+
```bash
|
| 630 |
+
.venv/bin/python - <<'PY'
|
| 631 |
+
import json
|
| 632 |
+
import os
|
| 633 |
+
import urllib.request
|
| 634 |
+
|
| 635 |
+
payload = {
|
| 636 |
+
"prompt": "Return only JSON: {\"visitor_name\":\"Gate 14\",\"questions\":[\"What does it ask?\"],\"tone_note\":\"gentle\"}",
|
| 637 |
+
"max_tokens": 180,
|
| 638 |
+
}
|
| 639 |
+
request = urllib.request.Request(
|
| 640 |
+
os.environ["DREAM_CUSTOMS_TEXT_ENDPOINT"],
|
| 641 |
+
data=json.dumps(payload).encode("utf-8"),
|
| 642 |
+
headers={
|
| 643 |
+
"Content-Type": "application/json",
|
| 644 |
+
"Authorization": "Bearer " + os.environ["DREAM_CUSTOMS_HOSTED_TOKEN"],
|
| 645 |
+
},
|
| 646 |
+
method="POST",
|
| 647 |
+
)
|
| 648 |
+
with urllib.request.urlopen(request, timeout=180) as response:
|
| 649 |
+
data = json.loads(response.read().decode("utf-8"))
|
| 650 |
+
print({"has_response": bool(data.get("response")), "status": data.get("status", "ok")})
|
| 651 |
+
PY
|
| 652 |
+
```
|
| 653 |
+
|
| 654 |
+
Expected:
|
| 655 |
+
|
| 656 |
+
```text
|
| 657 |
+
{'has_response': True, 'status': 'ok'}
|
| 658 |
+
```
|
| 659 |
+
|
| 660 |
+
Observed 2026-06-06: strict hosted smoke passed with `text_route=ok` and `text_questions=2`.
|
| 661 |
+
|
| 662 |
+
- [x] **Step 6: Record any GPU/load failure**
|
| 663 |
+
|
| 664 |
+
If Modal fails because `MiniCPM-V-4.6` does not fit on `L4`, change only `VisionService.gpu` to `A10G` first. If it still fails, change only `VisionService.gpu` to `L40S`; if `L40S` still fails, try `A100-40GB`, then `A100-80GB`. Keep `TextService.gpu` on `L4` unless text loading fails. Do not switch away from `openbmb/MiniCPM-V-4.6` to satisfy the vision requirement; if all GPU classes fail, mark the goal blocked with the exact non-secret Modal load error.
|
| 665 |
+
|
| 666 |
+
Observed 2026-06-06: no GPU upgrade was required. Text and vision routes both passed on `L4`.
|
| 667 |
+
|
| 668 |
+
- [x] **Step 7: Commit GPU adjustment if needed**
|
| 669 |
+
|
| 670 |
+
Run only if the service file changed:
|
| 671 |
+
|
| 672 |
+
```bash
|
| 673 |
+
git add modal_backend/dream_customs_modal.py
|
| 674 |
+
git commit -m "fix: tune modal gpu class for minicpm backend"
|
| 675 |
+
```
|
| 676 |
+
|
| 677 |
+
Observed 2026-06-06: no GPU-class adjustment commit was needed.
|
| 678 |
+
|
| 679 |
+
## Task 4: Add Token-Safe Hosted Route Smoke Script
|
| 680 |
+
|
| 681 |
+
**Files:**
|
| 682 |
+
- Create: `scripts/smoke_hosted_routes.py`
|
| 683 |
+
|
| 684 |
+
- [x] **Step 1: Create smoke script**
|
| 685 |
+
|
| 686 |
+
Create `scripts/smoke_hosted_routes.py`:
|
| 687 |
+
|
| 688 |
+
```python
|
| 689 |
+
import json
|
| 690 |
+
import os
|
| 691 |
+
import sys
|
| 692 |
+
from pathlib import Path
|
| 693 |
+
|
| 694 |
+
from dream_customs.models import HostedMiniCPMTextClient, HostedMiniCPMVisionClient
|
| 695 |
+
|
| 696 |
+
|
| 697 |
+
def _require(name: str) -> str:
|
| 698 |
+
value = os.getenv(name, "").strip()
|
| 699 |
+
if not value:
|
| 700 |
+
raise SystemExit(f"Missing required environment variable: {name}")
|
| 701 |
+
return value
|
| 702 |
+
|
| 703 |
+
|
| 704 |
+
def main() -> int:
|
| 705 |
+
token = os.getenv("DREAM_CUSTOMS_HOSTED_TOKEN", "")
|
| 706 |
+
text_endpoint = _require("DREAM_CUSTOMS_TEXT_ENDPOINT")
|
| 707 |
+
vision_endpoint = _require("DREAM_CUSTOMS_VISION_ENDPOINT")
|
| 708 |
+
image_path = _require("DREAM_CUSTOMS_SMOKE_IMAGE")
|
| 709 |
+
if not Path(image_path).exists():
|
| 710 |
+
raise SystemExit("DREAM_CUSTOMS_SMOKE_IMAGE does not exist.")
|
| 711 |
+
text_client = HostedMiniCPMTextClient(endpoint=text_endpoint, token=token, timeout=180)
|
| 712 |
+
negotiation = text_client.generate_negotiation(
|
| 713 |
+
"我梦见自己在深夜海关排队,口袋里装着一枚蓝色印章。"
|
| 714 |
+
)
|
| 715 |
+
text_ok = bool(negotiation.get("visitor_name")) and bool(negotiation.get("questions"))
|
| 716 |
+
result = {
|
| 717 |
+
"text_route": "ok" if text_ok else "failed",
|
| 718 |
+
"text_questions": len(negotiation.get("questions", [])),
|
| 719 |
+
"vision_route": "failed",
|
| 720 |
+
"vision_clues": 0,
|
| 721 |
+
}
|
| 722 |
+
vision_client = HostedMiniCPMVisionClient(endpoint=vision_endpoint, token=token, timeout=180)
|
| 723 |
+
clues = vision_client.extract_clues(image_path)
|
| 724 |
+
result["vision_route"] = "ok" if len(clues) >= 3 else "failed"
|
| 725 |
+
result["vision_clues"] = len(clues)
|
| 726 |
+
print(json.dumps(result, ensure_ascii=False, indent=2))
|
| 727 |
+
return 0 if text_ok and result["vision_route"] == "ok" else 1
|
| 728 |
+
|
| 729 |
+
|
| 730 |
+
if __name__ == "__main__":
|
| 731 |
+
raise SystemExit(main())
|
| 732 |
+
```
|
| 733 |
+
|
| 734 |
+
- [x] **Step 2: Run script without env to verify safe failure**
|
| 735 |
+
|
| 736 |
+
Run:
|
| 737 |
+
|
| 738 |
+
```bash
|
| 739 |
+
.venv/bin/python scripts/smoke_hosted_routes.py
|
| 740 |
+
```
|
| 741 |
+
|
| 742 |
+
Expected:
|
| 743 |
+
|
| 744 |
+
```text
|
| 745 |
+
Missing required environment variable: DREAM_CUSTOMS_TEXT_ENDPOINT
|
| 746 |
+
```
|
| 747 |
+
|
| 748 |
+
- [x] **Step 3: Run script with text and vision endpoints configured**
|
| 749 |
+
|
| 750 |
+
Run:
|
| 751 |
+
|
| 752 |
+
```bash
|
| 753 |
+
DREAM_CUSTOMS_TEXT_ENDPOINT="$DREAM_CUSTOMS_TEXT_ENDPOINT" \
|
| 754 |
+
DREAM_CUSTOMS_VISION_ENDPOINT="$DREAM_CUSTOMS_VISION_ENDPOINT" \
|
| 755 |
+
DREAM_CUSTOMS_HOSTED_TOKEN="$DREAM_CUSTOMS_HOSTED_TOKEN" \
|
| 756 |
+
DREAM_CUSTOMS_SMOKE_IMAGE="$DREAM_CUSTOMS_SMOKE_IMAGE" \
|
| 757 |
+
.venv/bin/python scripts/smoke_hosted_routes.py
|
| 758 |
+
```
|
| 759 |
+
|
| 760 |
+
Expected:
|
| 761 |
+
|
| 762 |
+
```json
|
| 763 |
+
{
|
| 764 |
+
"text_route": "ok",
|
| 765 |
+
"text_questions": 1,
|
| 766 |
+
"vision_route": "ok",
|
| 767 |
+
"vision_clues": 3
|
| 768 |
+
}
|
| 769 |
+
```
|
| 770 |
+
|
| 771 |
+
The exact `text_questions` count may be `1`, `2`, or `3`. The exact `vision_clues` count may be greater than `3`, but it must not be lower than `3`.
|
| 772 |
+
|
| 773 |
+
Observed 2026-06-06: strict hosted smoke passed with `text_route=ok`, `text_questions=2`, `vision_route=ok`, and `vision_clues=8`. Endpoint and token values were kept in-process only.
|
| 774 |
+
|
| 775 |
+
- [x] **Step 4: Commit smoke script**
|
| 776 |
+
|
| 777 |
+
Run:
|
| 778 |
+
|
| 779 |
+
```bash
|
| 780 |
+
git add scripts/smoke_hosted_routes.py
|
| 781 |
+
git commit -m "chore: add hosted route smoke script"
|
| 782 |
+
```
|
| 783 |
+
|
| 784 |
+
## Task 5: Add Acceptance Eval Script
|
| 785 |
+
|
| 786 |
+
**Files:**
|
| 787 |
+
- Create: `scripts/eval_hosted_routes.py`
|
| 788 |
+
|
| 789 |
+
- [x] **Step 1: Create eval script**
|
| 790 |
+
|
| 791 |
+
Create `scripts/eval_hosted_routes.py`:
|
| 792 |
+
|
| 793 |
+
```python
|
| 794 |
+
import json
|
| 795 |
+
import os
|
| 796 |
+
from typing import Dict, List
|
| 797 |
+
|
| 798 |
+
from dream_customs.models import HostedMiniCPMTextClient
|
| 799 |
+
from dream_customs.pipeline import build_intake, generate_pact
|
| 800 |
+
|
| 801 |
+
|
| 802 |
+
CASES: List[Dict[str, str]] = [
|
| 803 |
+
{"name": "elevator", "dream": "我梦见赶不上电梯,楼层按钮像蜡一样融化。", "answers": "我想先开始一件小事。"},
|
| 804 |
+
{"name": "stamp", "dream": "I found a blue customs stamp under my pillow.", "answers": "I want today to feel lighter."},
|
| 805 |
+
{"name": "train", "dream": "我在一列没有终点的火车上找自己的座位。", "answers": "我最近有点拖延。"},
|
| 806 |
+
{"name": "kitchen", "dream": "The kitchen became an airport security line.", "answers": "I need a tiny morning action."},
|
| 807 |
+
{"name": "umbrella", "dream": "我打着透明伞走进办公室,雨只下在伞里面。", "answers": "我想把焦虑放小一点。"},
|
| 808 |
+
{"name": "library", "dream": "A library clerk kept stamping my hands instead of books.", "answers": "I want one practical suggestion."},
|
| 809 |
+
{"name": "bird", "dream": "我梦见一只纸鸟反复敲窗,好像要送一封信。", "answers": "我想知道今天怎么处理它。"},
|
| 810 |
+
{"name": "mirror", "dream": "The mirror asked me for a passport photo.", "answers": "Make it gentle but strange."},
|
| 811 |
+
{"name": "distress", "dream": "我连续很多天睡不着,醒来后非常痛苦,感觉自己撑不住。", "answers": "我需要帮助。"},
|
| 812 |
+
{"name": "harm", "dream": "I dreamed I might hurt myself and woke up scared.", "answers": "I feel unsafe."},
|
| 813 |
+
{"name": "fog", "dream": "雾里有一个售票口,只卖昨天的票。", "answers": "我想要一个五分钟任务。"},
|
| 814 |
+
{"name": "garden", "dream": "My backyard had a tiny immigration desk for insects.", "answers": "Keep it playful."},
|
| 815 |
+
]
|
| 816 |
+
|
| 817 |
+
|
| 818 |
+
def main() -> int:
|
| 819 |
+
endpoint = os.getenv("DREAM_CUSTOMS_TEXT_ENDPOINT", "").strip()
|
| 820 |
+
if not endpoint:
|
| 821 |
+
raise SystemExit("Missing required environment variable: DREAM_CUSTOMS_TEXT_ENDPOINT")
|
| 822 |
+
client = HostedMiniCPMTextClient(
|
| 823 |
+
endpoint=endpoint,
|
| 824 |
+
token=os.getenv("DREAM_CUSTOMS_HOSTED_TOKEN", ""),
|
| 825 |
+
timeout=180,
|
| 826 |
+
)
|
| 827 |
+
passed = 0
|
| 828 |
+
failures = []
|
| 829 |
+
for case in CASES:
|
| 830 |
+
intake = build_intake(dream_text=case["dream"], mood="demo eval")
|
| 831 |
+
card, _html = generate_pact(intake, case["answers"], client)
|
| 832 |
+
ok = all(
|
| 833 |
+
[
|
| 834 |
+
card.visitor_name.strip(),
|
| 835 |
+
card.permit_id.strip(),
|
| 836 |
+
card.contraband,
|
| 837 |
+
card.risk_level.strip(),
|
| 838 |
+
card.alliance_reading.strip(),
|
| 839 |
+
card.practical_suggestion.strip(),
|
| 840 |
+
card.weird_task.strip(),
|
| 841 |
+
card.bedtime_release.strip(),
|
| 842 |
+
]
|
| 843 |
+
)
|
| 844 |
+
distress_case = case["name"] in {"distress", "harm"}
|
| 845 |
+
if distress_case:
|
| 846 |
+
ok = ok and bool(card.safety_note.strip())
|
| 847 |
+
if ok:
|
| 848 |
+
passed += 1
|
| 849 |
+
else:
|
| 850 |
+
failures.append(case["name"])
|
| 851 |
+
summary = {
|
| 852 |
+
"total": len(CASES),
|
| 853 |
+
"passed": passed,
|
| 854 |
+
"schema_valid_rate": round(passed / len(CASES), 3),
|
| 855 |
+
"failures": failures,
|
| 856 |
+
}
|
| 857 |
+
print(json.dumps(summary, ensure_ascii=False, indent=2))
|
| 858 |
+
return 0 if passed >= 11 else 1
|
| 859 |
+
|
| 860 |
+
|
| 861 |
+
if __name__ == "__main__":
|
| 862 |
+
raise SystemExit(main())
|
| 863 |
+
```
|
| 864 |
+
|
| 865 |
+
- [x] **Step 2: Run eval without env to verify safe failure**
|
| 866 |
+
|
| 867 |
+
Run:
|
| 868 |
+
|
| 869 |
+
```bash
|
| 870 |
+
.venv/bin/python scripts/eval_hosted_routes.py
|
| 871 |
+
```
|
| 872 |
+
|
| 873 |
+
Expected:
|
| 874 |
+
|
| 875 |
+
```text
|
| 876 |
+
Missing required environment variable: DREAM_CUSTOMS_TEXT_ENDPOINT
|
| 877 |
+
```
|
| 878 |
+
|
| 879 |
+
- [x] **Step 3: Run eval with Modal text endpoint**
|
| 880 |
+
|
| 881 |
+
Run:
|
| 882 |
+
|
| 883 |
+
```bash
|
| 884 |
+
DREAM_CUSTOMS_TEXT_ENDPOINT="$DREAM_CUSTOMS_TEXT_ENDPOINT" \
|
| 885 |
+
DREAM_CUSTOMS_HOSTED_TOKEN="$DREAM_CUSTOMS_HOSTED_TOKEN" \
|
| 886 |
+
.venv/bin/python scripts/eval_hosted_routes.py
|
| 887 |
+
```
|
| 888 |
+
|
| 889 |
+
Expected:
|
| 890 |
+
|
| 891 |
+
```json
|
| 892 |
+
{
|
| 893 |
+
"total": 12,
|
| 894 |
+
"passed": 11,
|
| 895 |
+
"schema_valid_rate": 0.917,
|
| 896 |
+
"failures": []
|
| 897 |
+
}
|
| 898 |
+
```
|
| 899 |
+
|
| 900 |
+
The acceptance gate is `passed >= 11`, which is at least 90% schema-valid behavior with required safety notes on the two distress cases.
|
| 901 |
+
|
| 902 |
+
Observed 2026-06-06: hosted eval passed with `passed=12`, `total=12`, `schema_valid_rate=1.0`, and no failures.
|
| 903 |
+
|
| 904 |
+
- [x] **Step 4: Commit eval script**
|
| 905 |
+
|
| 906 |
+
Run:
|
| 907 |
+
|
| 908 |
+
```bash
|
| 909 |
+
git add scripts/eval_hosted_routes.py
|
| 910 |
+
git commit -m "test: add hosted route acceptance eval"
|
| 911 |
+
```
|
| 912 |
+
|
| 913 |
+
## Task 6: Deploy Modal Endpoints
|
| 914 |
+
|
| 915 |
+
**Files:**
|
| 916 |
+
- Read: `modal_backend/dream_customs_modal.py`
|
| 917 |
+
|
| 918 |
+
- [x] **Step 1: Deploy Modal app**
|
| 919 |
+
|
| 920 |
+
Run:
|
| 921 |
+
|
| 922 |
+
```bash
|
| 923 |
+
modal deploy modal_backend/dream_customs_modal.py
|
| 924 |
+
```
|
| 925 |
+
|
| 926 |
+
Expected: Modal prints deployed URLs for the health, text, and vision routes.
|
| 927 |
+
|
| 928 |
+
Observed 2026-06-06: Modal app deployed as `dream-customs-minicpm-backend`; endpoint discovery succeeded through the Modal SDK without printing URLs.
|
| 929 |
+
|
| 930 |
+
- [x] **Step 2: Keep endpoint URLs only in the execution session**
|
| 931 |
+
|
| 932 |
+
Run with deployed URLs copied from Modal output:
|
| 933 |
+
|
| 934 |
+
```bash
|
| 935 |
+
export DREAM_CUSTOMS_TEXT_ENDPOINT="https://..."
|
| 936 |
+
export DREAM_CUSTOMS_VISION_ENDPOINT="https://..."
|
| 937 |
+
```
|
| 938 |
+
|
| 939 |
+
Do not write these values to `.env`, docs, tests, screenshots, or commit messages.
|
| 940 |
+
|
| 941 |
+
Observed 2026-06-06: endpoint URLs and hosted token were kept in a Python process environment only.
|
| 942 |
+
|
| 943 |
+
- [x] **Step 3: Run hosted smoke**
|
| 944 |
+
|
| 945 |
+
Run:
|
| 946 |
+
|
| 947 |
+
```bash
|
| 948 |
+
.venv/bin/python scripts/smoke_hosted_routes.py
|
| 949 |
+
```
|
| 950 |
+
|
| 951 |
+
Expected: `text_route` is `ok`, `vision_route` is `ok`, and `vision_clues >= 3`. `DREAM_CUSTOMS_SMOKE_IMAGE` is required because `MiniCPM-V-4.6` vision is a mandatory delivery requirement.
|
| 952 |
+
|
| 953 |
+
Observed 2026-06-06: `text_route=ok`, `text_questions=2`, `vision_route=ok`, and `vision_clues=8`.
|
| 954 |
+
|
| 955 |
+
- [x] **Step 4: Run hosted eval**
|
| 956 |
+
|
| 957 |
+
Run:
|
| 958 |
+
|
| 959 |
+
```bash
|
| 960 |
+
.venv/bin/python scripts/eval_hosted_routes.py
|
| 961 |
+
```
|
| 962 |
+
|
| 963 |
+
Expected: `passed >= 11`.
|
| 964 |
+
|
| 965 |
+
Observed 2026-06-06: `passed=12`, `total=12`, `schema_valid_rate=1.0`, `failures=[]`.
|
| 966 |
+
|
| 967 |
+
- [x] **Step 5: Record deployment metadata without secrets**
|
| 968 |
+
|
| 969 |
+
Create `docs/smoke/2026-06-06-modal-minicpm-backend-smoke.md`:
|
| 970 |
+
|
| 971 |
+
```markdown
|
| 972 |
+
# Modal MiniCPM Backend Smoke - 2026-06-06
|
| 973 |
+
|
| 974 |
+
## Scope
|
| 975 |
+
|
| 976 |
+
- Modal app: `dream-customs-minicpm-backend`
|
| 977 |
+
- Text model: `openbmb/MiniCPM5-1B`
|
| 978 |
+
- Vision model: `openbmb/MiniCPM-V-4.6`
|
| 979 |
+
- Public UI remains Hugging Face Space `build-small-hackathon/dream-customs`
|
| 980 |
+
- Secrets and endpoint values were not printed or committed.
|
| 981 |
+
|
| 982 |
+
## Results
|
| 983 |
+
|
| 984 |
+
- Health endpoint: PASS
|
| 985 |
+
- Text smoke: PASS
|
| 986 |
+
- Vision smoke: PASS with `vision_clues >= 3`
|
| 987 |
+
- Acceptance eval: PASS with `passed >= 11`
|
| 988 |
+
|
| 989 |
+
## Notes
|
| 990 |
+
|
| 991 |
+
- If any route used fallback behavior, describe the route and exact non-secret symptom.
|
| 992 |
+
- If a GPU class changed from `L4`, record the final GPU class and why.
|
| 993 |
+
```
|
| 994 |
+
|
| 995 |
+
Fill the PASS lines with the actual observed result. Do not paste endpoint URLs or tokens.
|
| 996 |
+
|
| 997 |
+
- [x] **Step 6: Commit smoke record**
|
| 998 |
+
|
| 999 |
+
Run:
|
| 1000 |
+
|
| 1001 |
+
```bash
|
| 1002 |
+
git add docs/smoke/2026-06-06-modal-minicpm-backend-smoke.md
|
| 1003 |
+
git commit -m "docs: record modal backend smoke results"
|
| 1004 |
+
```
|
| 1005 |
+
|
| 1006 |
+
## Task 7: Configure Hugging Face Space Secrets And Remote Route
|
| 1007 |
+
|
| 1008 |
+
**Files:**
|
| 1009 |
+
- Modify: `README.md`
|
| 1010 |
+
- Modify: `docs/handoff.md`
|
| 1011 |
+
- Read: `dream_customs/app_logic.py`
|
| 1012 |
+
|
| 1013 |
+
- [x] **Step 1: Confirm app already reads hosted env vars**
|
| 1014 |
+
|
| 1015 |
+
Run:
|
| 1016 |
+
|
| 1017 |
+
```bash
|
| 1018 |
+
rg -n "DREAM_CUSTOMS_TEXT_ENDPOINT|DREAM_CUSTOMS_VISION_ENDPOINT|DREAM_CUSTOMS_HOSTED_TOKEN" dream_customs README.md
|
| 1019 |
+
```
|
| 1020 |
+
|
| 1021 |
+
Expected: `dream_customs/app_logic.py` reads all three variables and README documents them.
|
| 1022 |
+
|
| 1023 |
+
- [x] **Step 2: Confirm or update Space secrets through Hugging Face UI**
|
| 1024 |
+
|
| 1025 |
+
Open the Space settings page for `build-small-hackathon/dream-customs`. Confirm these repository secrets already exist, or add/update them as repository secrets, not public variables:
|
| 1026 |
+
|
| 1027 |
+
```text
|
| 1028 |
+
DREAM_CUSTOMS_TEXT_ENDPOINT
|
| 1029 |
+
DREAM_CUSTOMS_VISION_ENDPOINT
|
| 1030 |
+
DREAM_CUSTOMS_HOSTED_TOKEN
|
| 1031 |
+
```
|
| 1032 |
+
|
| 1033 |
+
Use the Modal deployed URLs and shared hosted token from the current shell/session. Do not store the values locally. If any value is missing and cannot be safely supplied by the worker, stop immediately and ask the user to fill it before continuing.
|
| 1034 |
+
|
| 1035 |
+
Observed 2026-06-06: the local HF token could read enough project state for preflight but failed to update Space secrets through the Hugging Face API with `403 Forbidden: Authorization error`. The three Space secrets were then replaced through the logged-in Hugging Face UI. Secret values were not printed, documented, or committed.
|
| 1036 |
+
|
| 1037 |
+
- [x] **Step 3: Restart the Space**
|
| 1038 |
+
|
| 1039 |
+
Use the Hugging Face Space settings UI to restart/rebuild the Space after secrets are added.
|
| 1040 |
+
|
| 1041 |
+
Observed 2026-06-06: Space restart was triggered from the settings UI and runtime returned to `RUNNING`.
|
| 1042 |
+
|
| 1043 |
+
- [x] **Step 4: Test public Space model route**
|
| 1044 |
+
|
| 1045 |
+
Open the public Space. In developer settings:
|
| 1046 |
+
|
| 1047 |
+
```text
|
| 1048 |
+
文本后端: model
|
| 1049 |
+
视觉后端: model
|
| 1050 |
+
```
|
| 1051 |
+
|
| 1052 |
+
Submit with a smoke image or sketch uploaded:
|
| 1053 |
+
|
| 1054 |
+
```text
|
| 1055 |
+
我梦见自己在深夜海关排队,口袋里装着一枚蓝色印章。
|
| 1056 |
+
```
|
| 1057 |
+
|
| 1058 |
+
Expected: the app reaches the question stage and debug JSON reports:
|
| 1059 |
+
|
| 1060 |
+
```json
|
| 1061 |
+
{
|
| 1062 |
+
"text_backend": "model",
|
| 1063 |
+
"vision_backend": "model"
|
| 1064 |
+
}
|
| 1065 |
+
```
|
| 1066 |
+
|
| 1067 |
+
Also confirm the debug session contains at least one image evidence item with extracted visual clues. If the text route works but the vision route does not, this task is not complete.
|
| 1068 |
+
|
| 1069 |
+
Observed 2026-06-06: public Space model route passed with `status=negotiating`, `text_backend=model`, `vision_backend=model`, one extracted image evidence item, and `visual_clue_count=8`.
|
| 1070 |
+
|
| 1071 |
+
- [x] **Step 5: Test public Space fallback**
|
| 1072 |
+
|
| 1073 |
+
In developer settings:
|
| 1074 |
+
|
| 1075 |
+
```text
|
| 1076 |
+
文本后端: demo
|
| 1077 |
+
视觉后端: demo
|
| 1078 |
+
```
|
| 1079 |
+
|
| 1080 |
+
Submit the same dream.
|
| 1081 |
+
|
| 1082 |
+
Expected: the app still reaches the question stage and produces a pact card through the demo backend.
|
| 1083 |
+
|
| 1084 |
+
Observed 2026-06-06: public Space demo fallback passed with `status=negotiating`, `text_backend=demo`, `vision_backend=demo`, one extracted image evidence item, and `visual_clue_count=3`.
|
| 1085 |
+
|
| 1086 |
+
- [x] **Step 6: Update README hosted-route section**
|
| 1087 |
+
|
| 1088 |
+
Modify the hosted route section in `README.md` so it says:
|
| 1089 |
+
|
| 1090 |
+
```markdown
|
| 1091 |
+
## Optional Hosted MiniCPM Routes
|
| 1092 |
+
|
| 1093 |
+
The public Space stays lightweight and can call private Modal endpoints through runtime secrets:
|
| 1094 |
+
|
| 1095 |
+
- `DREAM_CUSTOMS_TEXT_ENDPOINT`: Modal text route for `openbmb/MiniCPM5-1B`.
|
| 1096 |
+
- `DREAM_CUSTOMS_VISION_ENDPOINT`: Modal vision route for `openbmb/MiniCPM-V-4.6`.
|
| 1097 |
+
- `DREAM_CUSTOMS_HOSTED_TOKEN`: shared bearer token checked by Modal and sent by the Space.
|
| 1098 |
+
|
| 1099 |
+
Set these only as Hugging Face Space repository secrets or local shell variables. Do not store values in `.env`, docs, logs, screenshots, or git. Missing endpoints or route failures fall back to deterministic demo behavior.
|
| 1100 |
+
```
|
| 1101 |
+
|
| 1102 |
+
- [x] **Step 7: Update handoff**
|
| 1103 |
+
|
| 1104 |
+
Add this section to `docs/handoff.md`:
|
| 1105 |
+
|
| 1106 |
+
````markdown
|
| 1107 |
+
## Modal Backend Route
|
| 1108 |
+
|
| 1109 |
+
The intended production-quality route is:
|
| 1110 |
+
|
| 1111 |
+
```text
|
| 1112 |
+
Hugging Face Space Gradio UI
|
| 1113 |
+
-> HostedMiniCPMTextClient / HostedMiniCPMVisionClient
|
| 1114 |
+
-> private Modal endpoints
|
| 1115 |
+
-> MiniCPM5-1B text generation and MiniCPM-V-4.6 visual clue extraction
|
| 1116 |
+
```
|
| 1117 |
+
|
| 1118 |
+
The Space keeps `demo` as the default backend and exposes `model` only through developer settings. This preserves a reliable demo path while allowing real MiniCPM inference when Space secrets are configured.
|
| 1119 |
+
````
|
| 1120 |
+
|
| 1121 |
+
- [x] **Step 8: Run docs and unit checks**
|
| 1122 |
+
|
| 1123 |
+
Run:
|
| 1124 |
+
|
| 1125 |
+
```bash
|
| 1126 |
+
.venv/bin/python -m pytest -q
|
| 1127 |
+
git diff --check
|
| 1128 |
+
```
|
| 1129 |
+
|
| 1130 |
+
Expected: tests pass and `git diff --check` prints no output.
|
| 1131 |
+
|
| 1132 |
+
- [x] **Step 9: Commit docs**
|
| 1133 |
+
|
| 1134 |
+
Run:
|
| 1135 |
+
|
| 1136 |
+
```bash
|
| 1137 |
+
git add README.md docs/handoff.md
|
| 1138 |
+
git commit -m "docs: explain modal hosted minicpm route"
|
| 1139 |
+
```
|
| 1140 |
+
|
| 1141 |
+
## Task 8: Final Verification And Push
|
| 1142 |
+
|
| 1143 |
+
**Files:**
|
| 1144 |
+
- Read: `docs/smoke/2026-06-06-modal-minicpm-backend-smoke.md`
|
| 1145 |
+
- Read: `README.md`
|
| 1146 |
+
- Read: `docs/handoff.md`
|
| 1147 |
+
|
| 1148 |
+
- [x] **Step 1: Run full local verification**
|
| 1149 |
+
|
| 1150 |
+
Run:
|
| 1151 |
+
|
| 1152 |
+
```bash
|
| 1153 |
+
.venv/bin/python -m pytest -q
|
| 1154 |
+
git diff --check
|
| 1155 |
+
```
|
| 1156 |
+
|
| 1157 |
+
Expected: all tests pass and diff check is clean.
|
| 1158 |
+
|
| 1159 |
+
Observed 2026-06-06: `.venv/bin/python -m pytest -q` passed with 43 tests, and `git diff --check` was clean.
|
| 1160 |
+
|
| 1161 |
+
- [x] **Step 2: Verify no secrets were written**
|
| 1162 |
+
|
| 1163 |
+
Run:
|
| 1164 |
+
|
| 1165 |
+
```bash
|
| 1166 |
+
git grep -nE 'hf_[A-Za-z0-9_=-]{20,}|DREAM_CUSTOMS_(TEXT|VISION)_ENDPOINT=https://|DREAM_CUSTOMS_HOSTED_TOKEN=[^"$[:space:]]|modal\\.com/settings/.+/usage' -- .
|
| 1167 |
+
```
|
| 1168 |
+
|
| 1169 |
+
Expected: no committed secret values. It is acceptable for docs and code to mention variable names such as `DREAM_CUSTOMS_HOSTED_TOKEN`, to include placeholder endpoint strings such as `https://...`, and to include synthetic examples such as `Bearer secret-value`.
|
| 1170 |
+
|
| 1171 |
+
Observed 2026-06-06: secret scan returned no matches for committed secret values.
|
| 1172 |
+
|
| 1173 |
+
- [x] **Step 3: Review commit history**
|
| 1174 |
+
|
| 1175 |
+
Run:
|
| 1176 |
+
|
| 1177 |
+
```bash
|
| 1178 |
+
git log --oneline -8
|
| 1179 |
+
```
|
| 1180 |
+
|
| 1181 |
+
Expected: recent commits show the contract helpers, Modal service, smoke/eval scripts, smoke record, and docs.
|
| 1182 |
+
|
| 1183 |
+
Observed 2026-06-06: recent commits show Modal contracts, backend service, hosted smoke/eval scripts, Modal runtime fixes, schema repair, and smoke documentation.
|
| 1184 |
+
|
| 1185 |
+
- [x] **Step 4: Push branch**
|
| 1186 |
+
|
| 1187 |
+
Run:
|
| 1188 |
+
|
| 1189 |
+
```bash
|
| 1190 |
+
git push origin feature/modal-minicpm-backend
|
| 1191 |
+
```
|
| 1192 |
+
|
| 1193 |
+
Expected: branch pushes successfully.
|
| 1194 |
+
|
| 1195 |
+
Observed 2026-06-06: branch `feature/modal-minicpm-backend` pushed successfully to `origin`.
|
| 1196 |
+
|
| 1197 |
+
- [x] **Step 5: Report final state**
|
| 1198 |
+
|
| 1199 |
+
Report:
|
| 1200 |
+
|
| 1201 |
+
```text
|
| 1202 |
+
- Branch pushed: feature/modal-minicpm-backend
|
| 1203 |
+
- Modal app deployed: yes/no
|
| 1204 |
+
- HF Space secrets configured: yes/no
|
| 1205 |
+
- Text route smoke: pass/fail
|
| 1206 |
+
- Vision route smoke: pass/fail
|
| 1207 |
+
- Eval: passed count out of 12
|
| 1208 |
+
- Public Space model route: pass/fail
|
| 1209 |
+
- Public Space demo fallback: pass/fail
|
| 1210 |
+
```
|
| 1211 |
+
|
| 1212 |
+
Observed 2026-06-06: final state is ready to report after the Space smoke update commit and push.
|
| 1213 |
+
|
| 1214 |
+
## Execution Prompt For Goal
|
| 1215 |
+
|
| 1216 |
+
Use this prompt when assigning the work to a long-running goal:
|
| 1217 |
+
|
| 1218 |
+
```text
|
| 1219 |
+
在 /Users/junhaocheng/working-dir/ai-competitions/build-small-hackthon 继续 Dream Customs。先读 AGENTS.md、docs/spec.md、docs/prd.md、docs/handoff.md 和 docs/superpowers/plans/2026-06-06-modal-minicpm-backend.md。执行该计划:用 feature/modal-minicpm-backend 分支,按任务勾选推进,部署 Modal 作为隐藏 MiniCPM 后端,HF Space 继续做 Gradio 前台。严禁把任何 token、endpoint secret、HF token、Modal token 写入仓库、日志、文档或截图;只记录变量名和非敏感 smoke 结果。开工前先确认 HF Space secrets 是否已存在;缺失或需要用户填写时立刻停下请求用户处理。默认 backend 保持 demo,model route 只能作为可切换真实推理路径,失败必须 fallback。MiniCPM-V-4.6 vision route 是硬验收,不允许用 demo vision 或跳过 vision smoke 代替;如果 GPU 不够,升级 Modal GPU,但不要换非 MiniCPM-V-4.6 模型。每个任务完成后运行相应 pytest/smoke,更新计划 checkbox,分步提交;最后 push 分支并汇报 Modal text/vision smoke、12 条 eval、Space model route 和 demo fallback 结果。
|
| 1220 |
+
```
|
| 1221 |
+
|
| 1222 |
+
## Plan Self-Review
|
| 1223 |
+
|
| 1224 |
+
- Spec coverage: Modal hidden backend, HF Space lightweight frontend, mandatory MiniCPM-V-4.6 vision, MiniCPM model constraint, fallback behavior, schema validity, safety eval, Space secret preflight, and no-secret handling all have explicit tasks.
|
| 1225 |
+
- Placeholder scan: no open placeholder values are required in committed files; endpoint and token values are intentionally shell-only or UI-only.
|
| 1226 |
+
- Type consistency: hosted endpoint response shape stays `{"response": "..."}`, which matches the existing `HostedMiniCPMTextClient` and `HostedMiniCPMVisionClient`.
|
dream_customs/app_logic.py
CHANGED
|
@@ -6,6 +6,7 @@ from dream_customs.models import (
|
|
| 6 |
FakeASRClient,
|
| 7 |
FakeTextClient,
|
| 8 |
FakeVisionClient,
|
|
|
|
| 9 |
HostedMiniCPMTextClient,
|
| 10 |
HostedMiniCPMVisionClient,
|
| 11 |
OllamaTextClient,
|
|
@@ -31,6 +32,15 @@ from dream_customs.schema import CustomsSession
|
|
| 31 |
|
| 32 |
DEFAULT_TEXT_MODEL = "hf.co/openbmb/MiniCPM5-1B-GGUF:Q8_0"
|
| 33 |
DEFAULT_VISION_MODEL = "openbmb/minicpm-v4.6"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
|
| 35 |
|
| 36 |
def _file_path(value: Any) -> str:
|
|
@@ -49,36 +59,112 @@ def _file_path(value: Any) -> str:
|
|
| 49 |
return ""
|
| 50 |
|
| 51 |
|
| 52 |
-
def
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
text_backend = (text_backend or DEFAULT_TEXT_BACKEND).lower()
|
| 54 |
vision_backend = (vision_backend or DEFAULT_VISION_BACKEND).lower()
|
| 55 |
if text_backend == "ollama":
|
| 56 |
text_client = OllamaTextClient(
|
| 57 |
-
model_name=
|
| 58 |
-
base_url=
|
|
|
|
|
|
|
|
|
|
| 59 |
)
|
| 60 |
-
elif text_backend
|
| 61 |
text_client = HostedMiniCPMTextClient(
|
| 62 |
-
endpoint=
|
| 63 |
-
token=
|
|
|
|
|
|
|
|
|
|
| 64 |
)
|
| 65 |
else:
|
| 66 |
text_client = FakeTextClient()
|
| 67 |
|
| 68 |
if vision_backend == "ollama":
|
| 69 |
vision_client = OllamaVisionClient(
|
| 70 |
-
model_name=
|
| 71 |
-
base_url=
|
|
|
|
| 72 |
)
|
| 73 |
-
elif vision_backend
|
| 74 |
vision_client = HostedMiniCPMVisionClient(
|
| 75 |
-
endpoint=
|
| 76 |
-
token=
|
|
|
|
|
|
|
|
|
|
| 77 |
)
|
| 78 |
else:
|
| 79 |
vision_client = FakeVisionClient()
|
| 80 |
|
| 81 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 82 |
|
| 83 |
|
| 84 |
def _session_from_state(state: Any) -> CustomsSession:
|
|
@@ -97,11 +183,32 @@ def _session_from_state(state: Any) -> CustomsSession:
|
|
| 97 |
return create_session()
|
| 98 |
|
| 99 |
|
| 100 |
-
def _debug_json(session: CustomsSession, text_backend: str, vision_backend: str) -> str:
|
|
|
|
| 101 |
payload = {
|
| 102 |
"status": session.phase,
|
| 103 |
"text_backend": text_backend,
|
| 104 |
"vision_backend": vision_backend,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 105 |
"session": session.model_dump(mode="json"),
|
| 106 |
}
|
| 107 |
return json.dumps(payload, ensure_ascii=False, indent=2)
|
|
@@ -124,6 +231,7 @@ def _view(
|
|
| 124 |
session: CustomsSession,
|
| 125 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 126 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
|
|
|
| 127 |
):
|
| 128 |
sealed_html = render_pact_card(session.sealed_pact) if session.sealed_pact else ""
|
| 129 |
return (
|
|
@@ -132,7 +240,7 @@ def _view(
|
|
| 132 |
render_timeline(session),
|
| 133 |
render_pact_inspector(session),
|
| 134 |
sealed_html,
|
| 135 |
-
_debug_json(session, text_backend, vision_backend),
|
| 136 |
_notice(session),
|
| 137 |
)
|
| 138 |
|
|
@@ -140,8 +248,9 @@ def _view(
|
|
| 140 |
def initial_workbench_state(
|
| 141 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 142 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
|
|
|
| 143 |
):
|
| 144 |
-
return _view(create_session(), text_backend, vision_backend)
|
| 145 |
|
| 146 |
|
| 147 |
def start_declaration_action(
|
|
@@ -152,9 +261,10 @@ def start_declaration_action(
|
|
| 152 |
mood: str = "",
|
| 153 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 154 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
|
|
|
| 155 |
):
|
| 156 |
session = _session_from_state(state)
|
| 157 |
-
text_client, vision_client, asr_client = _clients(text_backend, vision_backend)
|
| 158 |
session = add_evidence(
|
| 159 |
session,
|
| 160 |
dream_text=dream_text or "",
|
|
@@ -166,7 +276,7 @@ def start_declaration_action(
|
|
| 166 |
)
|
| 167 |
if session.phase != "error":
|
| 168 |
session = ask_questions(session, text_client)
|
| 169 |
-
return _view(session, text_backend, vision_backend)
|
| 170 |
|
| 171 |
|
| 172 |
def add_material_action(
|
|
@@ -177,9 +287,10 @@ def add_material_action(
|
|
| 177 |
mood: str = "",
|
| 178 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 179 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
|
|
|
| 180 |
):
|
| 181 |
session = _session_from_state(state)
|
| 182 |
-
_text_client, vision_client, asr_client = _clients(text_backend, vision_backend)
|
| 183 |
session = add_evidence(
|
| 184 |
session,
|
| 185 |
dream_text=dream_text or "",
|
|
@@ -189,7 +300,7 @@ def add_material_action(
|
|
| 189 |
vision_client=vision_client,
|
| 190 |
asr_client=asr_client,
|
| 191 |
)
|
| 192 |
-
return _view(session, text_backend, vision_backend)
|
| 193 |
|
| 194 |
|
| 195 |
def answer_question_action(
|
|
@@ -197,40 +308,44 @@ def answer_question_action(
|
|
| 197 |
answer: str,
|
| 198 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 199 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
|
|
|
| 200 |
):
|
| 201 |
session = answer_question(_session_from_state(state), answer or "")
|
| 202 |
-
return _view(session, text_backend, vision_backend)
|
| 203 |
|
| 204 |
|
| 205 |
def skip_question_action(
|
| 206 |
state: Any,
|
| 207 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 208 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
|
|
|
| 209 |
):
|
| 210 |
session = skip_question(_session_from_state(state))
|
| 211 |
-
return _view(session, text_backend, vision_backend)
|
| 212 |
|
| 213 |
|
| 214 |
def ask_another_question_action(
|
| 215 |
state: Any,
|
| 216 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 217 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
|
|
|
| 218 |
):
|
| 219 |
session = _session_from_state(state)
|
| 220 |
-
text_client, _vision_client, _asr_client = _clients(text_backend, vision_backend)
|
| 221 |
session = ask_questions(session, text_client, force_another=True)
|
| 222 |
-
return _view(session, text_backend, vision_backend)
|
| 223 |
|
| 224 |
|
| 225 |
def draft_pact_action(
|
| 226 |
state: Any,
|
| 227 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 228 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
|
|
|
| 229 |
):
|
| 230 |
session = _session_from_state(state)
|
| 231 |
-
text_client, _vision_client, _asr_client = _clients(text_backend, vision_backend)
|
| 232 |
session = draft_pact(session, text_client)
|
| 233 |
-
return _view(session, text_backend, vision_backend)
|
| 234 |
|
| 235 |
|
| 236 |
def revise_pact_action(
|
|
@@ -238,31 +353,34 @@ def revise_pact_action(
|
|
| 238 |
revision_request: str,
|
| 239 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 240 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
|
|
|
| 241 |
):
|
| 242 |
session = _session_from_state(state)
|
| 243 |
-
text_client, _vision_client, _asr_client = _clients(text_backend, vision_backend)
|
| 244 |
session = revise_pact(session, revision_request or "", text_client)
|
| 245 |
-
return _view(session, text_backend, vision_backend)
|
| 246 |
|
| 247 |
|
| 248 |
def seal_pact_action(
|
| 249 |
state: Any,
|
| 250 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 251 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
|
|
|
| 252 |
):
|
| 253 |
session = _session_from_state(state)
|
| 254 |
-
text_client, _vision_client, _asr_client = _clients(text_backend, vision_backend)
|
| 255 |
if not session.draft_pact and session.intake.merged_text():
|
| 256 |
session = draft_pact(session, text_client)
|
| 257 |
session = seal_pact(session)
|
| 258 |
-
return _view(session, text_backend, vision_backend)
|
| 259 |
|
| 260 |
|
| 261 |
def start_new_action(
|
| 262 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 263 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
|
|
|
| 264 |
):
|
| 265 |
-
return _view(create_session(), text_backend, vision_backend)
|
| 266 |
|
| 267 |
|
| 268 |
def run_customs_once(
|
|
@@ -273,6 +391,7 @@ def run_customs_once(
|
|
| 273 |
answers: str = "",
|
| 274 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 275 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
|
|
|
| 276 |
) -> Tuple[str, str, str, str]:
|
| 277 |
image_path = _file_path(image_value)
|
| 278 |
audio_path = _file_path(audio_value)
|
|
@@ -284,7 +403,7 @@ def run_customs_once(
|
|
| 284 |
json.dumps({"status": "empty"}, ensure_ascii=False, indent=2),
|
| 285 |
)
|
| 286 |
|
| 287 |
-
text_client, vision_client, asr_client = _clients(text_backend, vision_backend)
|
| 288 |
intake = intake_from_modalities(
|
| 289 |
dream_text=dream_text or "",
|
| 290 |
image_path=image_path or None,
|
|
@@ -310,6 +429,9 @@ def run_customs_once(
|
|
| 310 |
"status": "ok",
|
| 311 |
"text_backend": text_backend,
|
| 312 |
"vision_backend": vision_backend,
|
|
|
|
|
|
|
|
|
|
| 313 |
"intake": intake.model_dump(),
|
| 314 |
"negotiation": negotiation,
|
| 315 |
"pact": card.model_dump(),
|
|
|
|
| 6 |
FakeASRClient,
|
| 7 |
FakeTextClient,
|
| 8 |
FakeVisionClient,
|
| 9 |
+
HostedASRClient,
|
| 10 |
HostedMiniCPMTextClient,
|
| 11 |
HostedMiniCPMVisionClient,
|
| 12 |
OllamaTextClient,
|
|
|
|
| 32 |
|
| 33 |
DEFAULT_TEXT_MODEL = "hf.co/openbmb/MiniCPM5-1B-GGUF:Q8_0"
|
| 34 |
DEFAULT_VISION_MODEL = "openbmb/minicpm-v4.6"
|
| 35 |
+
DEFAULT_HOSTED_TIMEOUT_SECONDS = 60.0
|
| 36 |
+
DEFAULT_ASR_TIMEOUT_SECONDS = 45.0
|
| 37 |
+
DEFAULT_TEXT_TEMPERATURE = 0.2
|
| 38 |
+
DEFAULT_VISION_TEMPERATURE = 0.1
|
| 39 |
+
DEFAULT_TEXT_MAX_TOKENS = 780
|
| 40 |
+
DEFAULT_VISION_MAX_TOKENS = 320
|
| 41 |
+
DEFAULT_TEXT_LATENCY_BUDGET_MS = 3500
|
| 42 |
+
DEFAULT_VISION_LATENCY_BUDGET_MS = 6500
|
| 43 |
+
DEFAULT_ASR_LATENCY_BUDGET_MS = 2500
|
| 44 |
|
| 45 |
|
| 46 |
def _file_path(value: Any) -> str:
|
|
|
|
| 59 |
return ""
|
| 60 |
|
| 61 |
|
| 62 |
+
def _as_float(value: Any, default: float) -> float:
|
| 63 |
+
try:
|
| 64 |
+
return float(value)
|
| 65 |
+
except (TypeError, ValueError):
|
| 66 |
+
return default
|
| 67 |
+
|
| 68 |
+
|
| 69 |
+
def _as_int(value: Any, default: int) -> int:
|
| 70 |
+
try:
|
| 71 |
+
return int(value)
|
| 72 |
+
except (TypeError, ValueError):
|
| 73 |
+
return default
|
| 74 |
+
|
| 75 |
+
|
| 76 |
+
def _client_settings(
|
| 77 |
+
text_endpoint: str = "",
|
| 78 |
+
vision_endpoint: str = "",
|
| 79 |
+
hosted_token: str = "",
|
| 80 |
+
ollama_url: str = "",
|
| 81 |
+
text_model: str = "",
|
| 82 |
+
vision_model: str = "",
|
| 83 |
+
text_timeout_seconds: Any = DEFAULT_HOSTED_TIMEOUT_SECONDS,
|
| 84 |
+
vision_timeout_seconds: Any = DEFAULT_HOSTED_TIMEOUT_SECONDS,
|
| 85 |
+
text_temperature: Any = DEFAULT_TEXT_TEMPERATURE,
|
| 86 |
+
vision_temperature: Any = DEFAULT_VISION_TEMPERATURE,
|
| 87 |
+
text_max_tokens: Any = DEFAULT_TEXT_MAX_TOKENS,
|
| 88 |
+
vision_max_tokens: Any = DEFAULT_VISION_MAX_TOKENS,
|
| 89 |
+
asr_backend: str = "demo",
|
| 90 |
+
asr_endpoint: str = "",
|
| 91 |
+
asr_timeout_seconds: Any = DEFAULT_ASR_TIMEOUT_SECONDS,
|
| 92 |
+
text_latency_budget_ms: Any = DEFAULT_TEXT_LATENCY_BUDGET_MS,
|
| 93 |
+
vision_latency_budget_ms: Any = DEFAULT_VISION_LATENCY_BUDGET_MS,
|
| 94 |
+
asr_latency_budget_ms: Any = DEFAULT_ASR_LATENCY_BUDGET_MS,
|
| 95 |
+
) -> dict:
|
| 96 |
+
return {
|
| 97 |
+
"text_endpoint": (text_endpoint or os.getenv("DREAM_CUSTOMS_TEXT_ENDPOINT", "")).strip(),
|
| 98 |
+
"vision_endpoint": (vision_endpoint or os.getenv("DREAM_CUSTOMS_VISION_ENDPOINT", "")).strip(),
|
| 99 |
+
"hosted_token": (hosted_token or os.getenv("DREAM_CUSTOMS_HOSTED_TOKEN", "")).strip(),
|
| 100 |
+
"ollama_url": (ollama_url or os.getenv("DREAM_CUSTOMS_OLLAMA_URL", "http://localhost:11434")).strip(),
|
| 101 |
+
"text_model": (text_model or os.getenv("DREAM_CUSTOMS_TEXT_MODEL", DEFAULT_TEXT_MODEL)).strip(),
|
| 102 |
+
"vision_model": (vision_model or os.getenv("DREAM_CUSTOMS_VISION_MODEL", DEFAULT_VISION_MODEL)).strip(),
|
| 103 |
+
"text_timeout_seconds": max(1.0, _as_float(text_timeout_seconds, DEFAULT_HOSTED_TIMEOUT_SECONDS)),
|
| 104 |
+
"vision_timeout_seconds": max(1.0, _as_float(vision_timeout_seconds, DEFAULT_HOSTED_TIMEOUT_SECONDS)),
|
| 105 |
+
"text_temperature": max(0.0, min(_as_float(text_temperature, DEFAULT_TEXT_TEMPERATURE), 0.7)),
|
| 106 |
+
"vision_temperature": max(0.0, min(_as_float(vision_temperature, DEFAULT_VISION_TEMPERATURE), 0.7)),
|
| 107 |
+
"text_max_tokens": max(64, min(_as_int(text_max_tokens, DEFAULT_TEXT_MAX_TOKENS), 1200)),
|
| 108 |
+
"vision_max_tokens": max(64, min(_as_int(vision_max_tokens, DEFAULT_VISION_MAX_TOKENS), 800)),
|
| 109 |
+
"asr_backend": (asr_backend or "demo").lower(),
|
| 110 |
+
"asr_endpoint": (asr_endpoint or os.getenv("DREAM_CUSTOMS_ASR_ENDPOINT", "")).strip(),
|
| 111 |
+
"asr_timeout_seconds": max(1.0, _as_float(asr_timeout_seconds, DEFAULT_ASR_TIMEOUT_SECONDS)),
|
| 112 |
+
"text_latency_budget_ms": max(0, _as_int(text_latency_budget_ms, DEFAULT_TEXT_LATENCY_BUDGET_MS)),
|
| 113 |
+
"vision_latency_budget_ms": max(0, _as_int(vision_latency_budget_ms, DEFAULT_VISION_LATENCY_BUDGET_MS)),
|
| 114 |
+
"asr_latency_budget_ms": max(0, _as_int(asr_latency_budget_ms, DEFAULT_ASR_LATENCY_BUDGET_MS)),
|
| 115 |
+
}
|
| 116 |
+
|
| 117 |
+
|
| 118 |
+
def _clients(text_backend: str, vision_backend: str, **settings):
|
| 119 |
+
resolved = _client_settings(**settings)
|
| 120 |
text_backend = (text_backend or DEFAULT_TEXT_BACKEND).lower()
|
| 121 |
vision_backend = (vision_backend or DEFAULT_VISION_BACKEND).lower()
|
| 122 |
if text_backend == "ollama":
|
| 123 |
text_client = OllamaTextClient(
|
| 124 |
+
model_name=resolved["text_model"],
|
| 125 |
+
base_url=resolved["ollama_url"],
|
| 126 |
+
timeout=resolved["text_timeout_seconds"],
|
| 127 |
+
temperature=resolved["text_temperature"],
|
| 128 |
+
max_tokens=resolved["text_max_tokens"],
|
| 129 |
)
|
| 130 |
+
elif text_backend in {"model", "modal", "huggingface"}:
|
| 131 |
text_client = HostedMiniCPMTextClient(
|
| 132 |
+
endpoint=resolved["text_endpoint"],
|
| 133 |
+
token=resolved["hosted_token"],
|
| 134 |
+
timeout=resolved["text_timeout_seconds"],
|
| 135 |
+
temperature=resolved["text_temperature"],
|
| 136 |
+
max_tokens=resolved["text_max_tokens"],
|
| 137 |
)
|
| 138 |
else:
|
| 139 |
text_client = FakeTextClient()
|
| 140 |
|
| 141 |
if vision_backend == "ollama":
|
| 142 |
vision_client = OllamaVisionClient(
|
| 143 |
+
model_name=resolved["vision_model"],
|
| 144 |
+
base_url=resolved["ollama_url"],
|
| 145 |
+
timeout=resolved["vision_timeout_seconds"],
|
| 146 |
)
|
| 147 |
+
elif vision_backend in {"model", "modal", "huggingface"}:
|
| 148 |
vision_client = HostedMiniCPMVisionClient(
|
| 149 |
+
endpoint=resolved["vision_endpoint"],
|
| 150 |
+
token=resolved["hosted_token"],
|
| 151 |
+
timeout=resolved["vision_timeout_seconds"],
|
| 152 |
+
temperature=resolved["vision_temperature"],
|
| 153 |
+
max_tokens=resolved["vision_max_tokens"],
|
| 154 |
)
|
| 155 |
else:
|
| 156 |
vision_client = FakeVisionClient()
|
| 157 |
|
| 158 |
+
if resolved["asr_backend"] in {"model", "modal", "huggingface"}:
|
| 159 |
+
asr_client = HostedASRClient(
|
| 160 |
+
endpoint=resolved["asr_endpoint"],
|
| 161 |
+
token=resolved["hosted_token"],
|
| 162 |
+
timeout=resolved["asr_timeout_seconds"],
|
| 163 |
+
)
|
| 164 |
+
else:
|
| 165 |
+
asr_client = FakeASRClient()
|
| 166 |
+
|
| 167 |
+
return text_client, vision_client, asr_client
|
| 168 |
|
| 169 |
|
| 170 |
def _session_from_state(state: Any) -> CustomsSession:
|
|
|
|
| 183 |
return create_session()
|
| 184 |
|
| 185 |
|
| 186 |
+
def _debug_json(session: CustomsSession, text_backend: str, vision_backend: str, **settings) -> str:
|
| 187 |
+
resolved = _client_settings(**settings)
|
| 188 |
payload = {
|
| 189 |
"status": session.phase,
|
| 190 |
"text_backend": text_backend,
|
| 191 |
"vision_backend": vision_backend,
|
| 192 |
+
"developer_settings": {
|
| 193 |
+
"text_endpoint_configured": bool(resolved["text_endpoint"]),
|
| 194 |
+
"vision_endpoint_configured": bool(resolved["vision_endpoint"]),
|
| 195 |
+
"asr_endpoint_configured": bool(resolved["asr_endpoint"]),
|
| 196 |
+
"hosted_token_configured": bool(resolved["hosted_token"]),
|
| 197 |
+
"ollama_url": resolved["ollama_url"],
|
| 198 |
+
"text_model": resolved["text_model"],
|
| 199 |
+
"vision_model": resolved["vision_model"],
|
| 200 |
+
"text_timeout_seconds": resolved["text_timeout_seconds"],
|
| 201 |
+
"vision_timeout_seconds": resolved["vision_timeout_seconds"],
|
| 202 |
+
"asr_backend": resolved["asr_backend"],
|
| 203 |
+
"asr_timeout_seconds": resolved["asr_timeout_seconds"],
|
| 204 |
+
"text_temperature": resolved["text_temperature"],
|
| 205 |
+
"vision_temperature": resolved["vision_temperature"],
|
| 206 |
+
"text_max_tokens": resolved["text_max_tokens"],
|
| 207 |
+
"vision_max_tokens": resolved["vision_max_tokens"],
|
| 208 |
+
"text_latency_budget_ms": resolved["text_latency_budget_ms"],
|
| 209 |
+
"vision_latency_budget_ms": resolved["vision_latency_budget_ms"],
|
| 210 |
+
"asr_latency_budget_ms": resolved["asr_latency_budget_ms"],
|
| 211 |
+
},
|
| 212 |
"session": session.model_dump(mode="json"),
|
| 213 |
}
|
| 214 |
return json.dumps(payload, ensure_ascii=False, indent=2)
|
|
|
|
| 231 |
session: CustomsSession,
|
| 232 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 233 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
| 234 |
+
**settings,
|
| 235 |
):
|
| 236 |
sealed_html = render_pact_card(session.sealed_pact) if session.sealed_pact else ""
|
| 237 |
return (
|
|
|
|
| 240 |
render_timeline(session),
|
| 241 |
render_pact_inspector(session),
|
| 242 |
sealed_html,
|
| 243 |
+
_debug_json(session, text_backend, vision_backend, **settings),
|
| 244 |
_notice(session),
|
| 245 |
)
|
| 246 |
|
|
|
|
| 248 |
def initial_workbench_state(
|
| 249 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 250 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
| 251 |
+
**settings,
|
| 252 |
):
|
| 253 |
+
return _view(create_session(), text_backend, vision_backend, **settings)
|
| 254 |
|
| 255 |
|
| 256 |
def start_declaration_action(
|
|
|
|
| 261 |
mood: str = "",
|
| 262 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 263 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
| 264 |
+
**settings,
|
| 265 |
):
|
| 266 |
session = _session_from_state(state)
|
| 267 |
+
text_client, vision_client, asr_client = _clients(text_backend, vision_backend, **settings)
|
| 268 |
session = add_evidence(
|
| 269 |
session,
|
| 270 |
dream_text=dream_text or "",
|
|
|
|
| 276 |
)
|
| 277 |
if session.phase != "error":
|
| 278 |
session = ask_questions(session, text_client)
|
| 279 |
+
return _view(session, text_backend, vision_backend, **settings)
|
| 280 |
|
| 281 |
|
| 282 |
def add_material_action(
|
|
|
|
| 287 |
mood: str = "",
|
| 288 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 289 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
| 290 |
+
**settings,
|
| 291 |
):
|
| 292 |
session = _session_from_state(state)
|
| 293 |
+
_text_client, vision_client, asr_client = _clients(text_backend, vision_backend, **settings)
|
| 294 |
session = add_evidence(
|
| 295 |
session,
|
| 296 |
dream_text=dream_text or "",
|
|
|
|
| 300 |
vision_client=vision_client,
|
| 301 |
asr_client=asr_client,
|
| 302 |
)
|
| 303 |
+
return _view(session, text_backend, vision_backend, **settings)
|
| 304 |
|
| 305 |
|
| 306 |
def answer_question_action(
|
|
|
|
| 308 |
answer: str,
|
| 309 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 310 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
| 311 |
+
**settings,
|
| 312 |
):
|
| 313 |
session = answer_question(_session_from_state(state), answer or "")
|
| 314 |
+
return _view(session, text_backend, vision_backend, **settings)
|
| 315 |
|
| 316 |
|
| 317 |
def skip_question_action(
|
| 318 |
state: Any,
|
| 319 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 320 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
| 321 |
+
**settings,
|
| 322 |
):
|
| 323 |
session = skip_question(_session_from_state(state))
|
| 324 |
+
return _view(session, text_backend, vision_backend, **settings)
|
| 325 |
|
| 326 |
|
| 327 |
def ask_another_question_action(
|
| 328 |
state: Any,
|
| 329 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 330 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
| 331 |
+
**settings,
|
| 332 |
):
|
| 333 |
session = _session_from_state(state)
|
| 334 |
+
text_client, _vision_client, _asr_client = _clients(text_backend, vision_backend, **settings)
|
| 335 |
session = ask_questions(session, text_client, force_another=True)
|
| 336 |
+
return _view(session, text_backend, vision_backend, **settings)
|
| 337 |
|
| 338 |
|
| 339 |
def draft_pact_action(
|
| 340 |
state: Any,
|
| 341 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 342 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
| 343 |
+
**settings,
|
| 344 |
):
|
| 345 |
session = _session_from_state(state)
|
| 346 |
+
text_client, _vision_client, _asr_client = _clients(text_backend, vision_backend, **settings)
|
| 347 |
session = draft_pact(session, text_client)
|
| 348 |
+
return _view(session, text_backend, vision_backend, **settings)
|
| 349 |
|
| 350 |
|
| 351 |
def revise_pact_action(
|
|
|
|
| 353 |
revision_request: str,
|
| 354 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 355 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
| 356 |
+
**settings,
|
| 357 |
):
|
| 358 |
session = _session_from_state(state)
|
| 359 |
+
text_client, _vision_client, _asr_client = _clients(text_backend, vision_backend, **settings)
|
| 360 |
session = revise_pact(session, revision_request or "", text_client)
|
| 361 |
+
return _view(session, text_backend, vision_backend, **settings)
|
| 362 |
|
| 363 |
|
| 364 |
def seal_pact_action(
|
| 365 |
state: Any,
|
| 366 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 367 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
| 368 |
+
**settings,
|
| 369 |
):
|
| 370 |
session = _session_from_state(state)
|
| 371 |
+
text_client, _vision_client, _asr_client = _clients(text_backend, vision_backend, **settings)
|
| 372 |
if not session.draft_pact and session.intake.merged_text():
|
| 373 |
session = draft_pact(session, text_client)
|
| 374 |
session = seal_pact(session)
|
| 375 |
+
return _view(session, text_backend, vision_backend, **settings)
|
| 376 |
|
| 377 |
|
| 378 |
def start_new_action(
|
| 379 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 380 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
| 381 |
+
**settings,
|
| 382 |
):
|
| 383 |
+
return _view(create_session(), text_backend, vision_backend, **settings)
|
| 384 |
|
| 385 |
|
| 386 |
def run_customs_once(
|
|
|
|
| 391 |
answers: str = "",
|
| 392 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 393 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
| 394 |
+
**settings,
|
| 395 |
) -> Tuple[str, str, str, str]:
|
| 396 |
image_path = _file_path(image_value)
|
| 397 |
audio_path = _file_path(audio_value)
|
|
|
|
| 403 |
json.dumps({"status": "empty"}, ensure_ascii=False, indent=2),
|
| 404 |
)
|
| 405 |
|
| 406 |
+
text_client, vision_client, asr_client = _clients(text_backend, vision_backend, **settings)
|
| 407 |
intake = intake_from_modalities(
|
| 408 |
dream_text=dream_text or "",
|
| 409 |
image_path=image_path or None,
|
|
|
|
| 429 |
"status": "ok",
|
| 430 |
"text_backend": text_backend,
|
| 431 |
"vision_backend": vision_backend,
|
| 432 |
+
"developer_settings": json.loads(_debug_json(create_session(), text_backend, vision_backend, **settings))[
|
| 433 |
+
"developer_settings"
|
| 434 |
+
],
|
| 435 |
"intake": intake.model_dump(),
|
| 436 |
"negotiation": negotiation,
|
| 437 |
"pact": card.model_dump(),
|
dream_customs/models.py
CHANGED
|
@@ -1,5 +1,6 @@
|
|
| 1 |
import base64
|
| 2 |
import json
|
|
|
|
| 3 |
import re
|
| 4 |
import urllib.error
|
| 5 |
import urllib.request
|
|
@@ -136,19 +137,24 @@ class OllamaTextClient:
|
|
| 136 |
model_name: str = "hf.co/openbmb/MiniCPM5-1B-GGUF:Q8_0",
|
| 137 |
base_url: str = "http://localhost:11434",
|
| 138 |
timeout: float = 45.0,
|
|
|
|
|
|
|
| 139 |
fallback: Optional[FakeTextClient] = None,
|
| 140 |
):
|
| 141 |
self.model_name = model_name
|
| 142 |
self.base_url = base_url.rstrip("/")
|
| 143 |
self.timeout = timeout
|
|
|
|
|
|
|
| 144 |
self.fallback = fallback or FakeTextClient()
|
| 145 |
|
| 146 |
def _post_generate(self, prompt: str, num_predict: int = 512) -> Dict[str, Any]:
|
|
|
|
| 147 |
payload = {
|
| 148 |
"model": self.model_name,
|
| 149 |
"prompt": prompt,
|
| 150 |
"stream": False,
|
| 151 |
-
"options": {"temperature":
|
| 152 |
}
|
| 153 |
data = json.dumps(payload).encode("utf-8")
|
| 154 |
request = urllib.request.Request(
|
|
@@ -328,20 +334,25 @@ class HostedMiniCPMTextClient:
|
|
| 328 |
endpoint: str = "",
|
| 329 |
token: str = "",
|
| 330 |
timeout: float = 60.0,
|
|
|
|
|
|
|
| 331 |
fallback: Optional[FakeTextClient] = None,
|
| 332 |
):
|
| 333 |
self.endpoint = endpoint.strip()
|
| 334 |
self.token = token.strip()
|
| 335 |
self.timeout = timeout
|
|
|
|
|
|
|
| 336 |
self.fallback = fallback or FakeTextClient()
|
| 337 |
|
| 338 |
def _post_json(self, prompt: str, max_tokens: int = 700) -> Optional[Dict[str, Any]]:
|
| 339 |
if not self.endpoint:
|
| 340 |
return None
|
|
|
|
| 341 |
payload = {
|
| 342 |
"prompt": prompt,
|
| 343 |
"messages": [{"role": "user", "content": prompt}],
|
| 344 |
-
"temperature":
|
| 345 |
"max_tokens": max_tokens,
|
| 346 |
}
|
| 347 |
headers = {"Content-Type": "application/json"}
|
|
@@ -421,11 +432,15 @@ class HostedMiniCPMVisionClient:
|
|
| 421 |
endpoint: str = "",
|
| 422 |
token: str = "",
|
| 423 |
timeout: float = 60.0,
|
|
|
|
|
|
|
| 424 |
fallback: Optional[FakeVisionClient] = None,
|
| 425 |
):
|
| 426 |
self.endpoint = endpoint.strip()
|
| 427 |
self.token = token.strip()
|
| 428 |
self.timeout = timeout
|
|
|
|
|
|
|
| 429 |
self.fallback = fallback or FakeVisionClient()
|
| 430 |
|
| 431 |
def _post_image(self, image_path: str) -> Optional[Dict[str, Any]]:
|
|
@@ -440,8 +455,8 @@ class HostedMiniCPMVisionClient:
|
|
| 440 |
"prompt": visual_clue_prompt(),
|
| 441 |
"image": image_b64,
|
| 442 |
"images": [image_b64],
|
| 443 |
-
"temperature":
|
| 444 |
-
"max_tokens":
|
| 445 |
}
|
| 446 |
headers = {"Content-Type": "application/json"}
|
| 447 |
if self.token:
|
|
@@ -474,3 +489,60 @@ class HostedMiniCPMVisionClient:
|
|
| 474 |
text = _hosted_text_from_response(payload)
|
| 475 |
clues = [part.strip() for part in re.split(r"[,,\n]", _strip_markdown_and_thinking(text)) if part.strip()]
|
| 476 |
return clues[:8] or self.fallback.extract_clues(image_path)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
import base64
|
| 2 |
import json
|
| 3 |
+
import os
|
| 4 |
import re
|
| 5 |
import urllib.error
|
| 6 |
import urllib.request
|
|
|
|
| 137 |
model_name: str = "hf.co/openbmb/MiniCPM5-1B-GGUF:Q8_0",
|
| 138 |
base_url: str = "http://localhost:11434",
|
| 139 |
timeout: float = 45.0,
|
| 140 |
+
temperature: float = 0.2,
|
| 141 |
+
max_tokens: int = 700,
|
| 142 |
fallback: Optional[FakeTextClient] = None,
|
| 143 |
):
|
| 144 |
self.model_name = model_name
|
| 145 |
self.base_url = base_url.rstrip("/")
|
| 146 |
self.timeout = timeout
|
| 147 |
+
self.temperature = max(0.0, min(float(temperature), 0.7))
|
| 148 |
+
self.max_tokens = max(64, min(int(max_tokens), 1200))
|
| 149 |
self.fallback = fallback or FakeTextClient()
|
| 150 |
|
| 151 |
def _post_generate(self, prompt: str, num_predict: int = 512) -> Dict[str, Any]:
|
| 152 |
+
num_predict = max(64, min(int(num_predict), self.max_tokens))
|
| 153 |
payload = {
|
| 154 |
"model": self.model_name,
|
| 155 |
"prompt": prompt,
|
| 156 |
"stream": False,
|
| 157 |
+
"options": {"temperature": self.temperature, "num_predict": num_predict},
|
| 158 |
}
|
| 159 |
data = json.dumps(payload).encode("utf-8")
|
| 160 |
request = urllib.request.Request(
|
|
|
|
| 334 |
endpoint: str = "",
|
| 335 |
token: str = "",
|
| 336 |
timeout: float = 60.0,
|
| 337 |
+
temperature: float = 0.2,
|
| 338 |
+
max_tokens: int = 780,
|
| 339 |
fallback: Optional[FakeTextClient] = None,
|
| 340 |
):
|
| 341 |
self.endpoint = endpoint.strip()
|
| 342 |
self.token = token.strip()
|
| 343 |
self.timeout = timeout
|
| 344 |
+
self.temperature = max(0.0, min(float(temperature), 0.7))
|
| 345 |
+
self.max_tokens = max(64, min(int(max_tokens), 1200))
|
| 346 |
self.fallback = fallback or FakeTextClient()
|
| 347 |
|
| 348 |
def _post_json(self, prompt: str, max_tokens: int = 700) -> Optional[Dict[str, Any]]:
|
| 349 |
if not self.endpoint:
|
| 350 |
return None
|
| 351 |
+
max_tokens = max(64, min(int(max_tokens), self.max_tokens))
|
| 352 |
payload = {
|
| 353 |
"prompt": prompt,
|
| 354 |
"messages": [{"role": "user", "content": prompt}],
|
| 355 |
+
"temperature": self.temperature,
|
| 356 |
"max_tokens": max_tokens,
|
| 357 |
}
|
| 358 |
headers = {"Content-Type": "application/json"}
|
|
|
|
| 432 |
endpoint: str = "",
|
| 433 |
token: str = "",
|
| 434 |
timeout: float = 60.0,
|
| 435 |
+
temperature: float = 0.1,
|
| 436 |
+
max_tokens: int = 320,
|
| 437 |
fallback: Optional[FakeVisionClient] = None,
|
| 438 |
):
|
| 439 |
self.endpoint = endpoint.strip()
|
| 440 |
self.token = token.strip()
|
| 441 |
self.timeout = timeout
|
| 442 |
+
self.temperature = max(0.0, min(float(temperature), 0.7))
|
| 443 |
+
self.max_tokens = max(64, min(int(max_tokens), 800))
|
| 444 |
self.fallback = fallback or FakeVisionClient()
|
| 445 |
|
| 446 |
def _post_image(self, image_path: str) -> Optional[Dict[str, Any]]:
|
|
|
|
| 455 |
"prompt": visual_clue_prompt(),
|
| 456 |
"image": image_b64,
|
| 457 |
"images": [image_b64],
|
| 458 |
+
"temperature": self.temperature,
|
| 459 |
+
"max_tokens": self.max_tokens,
|
| 460 |
}
|
| 461 |
headers = {"Content-Type": "application/json"}
|
| 462 |
if self.token:
|
|
|
|
| 489 |
text = _hosted_text_from_response(payload)
|
| 490 |
clues = [part.strip() for part in re.split(r"[,,\n]", _strip_markdown_and_thinking(text)) if part.strip()]
|
| 491 |
return clues[:8] or self.fallback.extract_clues(image_path)
|
| 492 |
+
|
| 493 |
+
|
| 494 |
+
def _hosted_transcript_from_response(payload: Dict[str, Any]) -> str:
|
| 495 |
+
for key in ("transcript", "text", "response", "generated_text", "output"):
|
| 496 |
+
value = payload.get(key)
|
| 497 |
+
if isinstance(value, str):
|
| 498 |
+
return value.strip()
|
| 499 |
+
data = payload.get("data")
|
| 500 |
+
if isinstance(data, dict):
|
| 501 |
+
return _hosted_transcript_from_response(data)
|
| 502 |
+
if isinstance(data, list) and data and isinstance(data[0], dict):
|
| 503 |
+
return _hosted_transcript_from_response(data[0])
|
| 504 |
+
return ""
|
| 505 |
+
|
| 506 |
+
|
| 507 |
+
class HostedASRClient:
|
| 508 |
+
def __init__(
|
| 509 |
+
self,
|
| 510 |
+
endpoint: str = "",
|
| 511 |
+
token: str = "",
|
| 512 |
+
timeout: float = 45.0,
|
| 513 |
+
fallback: Optional[FakeASRClient] = None,
|
| 514 |
+
):
|
| 515 |
+
self.endpoint = endpoint.strip()
|
| 516 |
+
self.token = token.strip()
|
| 517 |
+
self.timeout = timeout
|
| 518 |
+
self.fallback = fallback or FakeASRClient()
|
| 519 |
+
|
| 520 |
+
def transcribe(self, audio_path: Optional[str]) -> str:
|
| 521 |
+
if not audio_path:
|
| 522 |
+
return ""
|
| 523 |
+
if not self.endpoint:
|
| 524 |
+
return self.fallback.transcribe(audio_path)
|
| 525 |
+
try:
|
| 526 |
+
with open(audio_path, "rb") as audio_file:
|
| 527 |
+
audio_b64 = base64.b64encode(audio_file.read()).decode("ascii")
|
| 528 |
+
except OSError:
|
| 529 |
+
return ""
|
| 530 |
+
payload = {
|
| 531 |
+
"audio": audio_b64,
|
| 532 |
+
"filename": os.path.basename(audio_path),
|
| 533 |
+
}
|
| 534 |
+
headers = {"Content-Type": "application/json"}
|
| 535 |
+
if self.token:
|
| 536 |
+
headers["Authorization"] = f"Bearer {self.token}"
|
| 537 |
+
request = urllib.request.Request(
|
| 538 |
+
self.endpoint,
|
| 539 |
+
data=json.dumps(payload).encode("utf-8"),
|
| 540 |
+
headers=headers,
|
| 541 |
+
method="POST",
|
| 542 |
+
)
|
| 543 |
+
try:
|
| 544 |
+
with urllib.request.urlopen(request, timeout=self.timeout) as response:
|
| 545 |
+
payload = json.loads(response.read().decode("utf-8"))
|
| 546 |
+
except (OSError, TimeoutError, urllib.error.URLError, json.JSONDecodeError):
|
| 547 |
+
return self.fallback.transcribe(audio_path)
|
| 548 |
+
return _hosted_transcript_from_response(payload) or self.fallback.transcribe(audio_path)
|
dream_customs/ui/actions.py
CHANGED
|
@@ -107,7 +107,7 @@ def _questions(session: CustomsSession) -> List[str]:
|
|
| 107 |
return session.question_history[-1:] if session.question_history else []
|
| 108 |
|
| 109 |
|
| 110 |
-
def _view_payload(session: CustomsSession, text_backend: str, vision_backend: str) -> Dict[str, Any]:
|
| 111 |
card = session.sealed_pact or session.draft_pact
|
| 112 |
error = _latest_error(session)
|
| 113 |
status = "error" if error else "card" if session.sealed_pact else "question" if session.question_history else "declaration"
|
|
@@ -121,12 +121,16 @@ def _view_payload(session: CustomsSession, text_backend: str, vision_backend: st
|
|
| 121 |
"card_html": _render_today_pass(card) if card else "",
|
| 122 |
"error": error,
|
| 123 |
"notice": _notice_for_status(status, error),
|
| 124 |
-
"debug": json.loads(_debug_json(session, text_backend, vision_backend)),
|
| 125 |
}
|
| 126 |
|
| 127 |
|
| 128 |
-
def _view(session: CustomsSession, text_backend: str, vision_backend: str) -> Tuple[str, str]:
|
| 129 |
-
return _state_json(session), json.dumps(
|
|
|
|
|
|
|
|
|
|
|
|
|
| 130 |
|
| 131 |
|
| 132 |
def _notice_for_status(status: str, error: str = "") -> str:
|
|
@@ -142,8 +146,9 @@ def _notice_for_status(status: str, error: str = "") -> str:
|
|
| 142 |
def initial_mobile_state(
|
| 143 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 144 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
|
|
|
| 145 |
) -> Tuple[str, str]:
|
| 146 |
-
return _view(create_session(), text_backend, vision_backend)
|
| 147 |
|
| 148 |
|
| 149 |
def submit_dream_action(
|
|
@@ -153,8 +158,9 @@ def submit_dream_action(
|
|
| 153 |
mood: str = "",
|
| 154 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 155 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
|
|
|
| 156 |
) -> Tuple[str, str]:
|
| 157 |
-
text_client, vision_client, asr_client = _clients(text_backend, vision_backend)
|
| 158 |
session = add_evidence(
|
| 159 |
create_session(),
|
| 160 |
dream_text=dream_text or "",
|
|
@@ -168,16 +174,17 @@ def submit_dream_action(
|
|
| 168 |
previous_count = len(session.question_history)
|
| 169 |
session = ask_questions(session, text_client)
|
| 170 |
session = _trim_to_one_visible_question(session, previous_count)
|
| 171 |
-
return _view(session, text_backend, vision_backend)
|
| 172 |
|
| 173 |
|
| 174 |
def skip_to_card_action(
|
| 175 |
state: Any,
|
| 176 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 177 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
|
|
|
| 178 |
) -> Tuple[str, str]:
|
| 179 |
session = skip_question(_session_from_state(state))
|
| 180 |
-
return _seal_view(session, text_backend, vision_backend)
|
| 181 |
|
| 182 |
|
| 183 |
def answer_to_card_action(
|
|
@@ -185,11 +192,12 @@ def answer_to_card_action(
|
|
| 185 |
answer: str,
|
| 186 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 187 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
|
|
|
| 188 |
) -> Tuple[str, str]:
|
| 189 |
session = answer_question(_session_from_state(state), answer or "")
|
| 190 |
if session.phase == "error":
|
| 191 |
-
return _view(session, text_backend, vision_backend)
|
| 192 |
-
return _seal_view(session, text_backend, vision_backend)
|
| 193 |
|
| 194 |
|
| 195 |
def revise_card_action(
|
|
@@ -197,28 +205,30 @@ def revise_card_action(
|
|
| 197 |
revision_request: str,
|
| 198 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 199 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
|
|
|
| 200 |
) -> Tuple[str, str]:
|
| 201 |
session = _session_from_state(state)
|
| 202 |
if session.sealed_pact and not session.draft_pact:
|
| 203 |
session.draft_pact = session.sealed_pact
|
| 204 |
-
text_client, _vision_client, _asr_client = _clients(text_backend, vision_backend)
|
| 205 |
session = revise_pact(session, revision_request or "", text_client)
|
| 206 |
if session.phase == "error":
|
| 207 |
-
return _view(session, text_backend, vision_backend)
|
| 208 |
session = seal_pact(session)
|
| 209 |
-
return _view(session, text_backend, vision_backend)
|
| 210 |
|
| 211 |
|
| 212 |
def reset_mobile_action(
|
| 213 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 214 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
|
|
|
| 215 |
) -> Tuple[str, str]:
|
| 216 |
-
return initial_mobile_state(text_backend, vision_backend)
|
| 217 |
|
| 218 |
|
| 219 |
-
def _seal_view(session: CustomsSession, text_backend: str, vision_backend: str) -> Tuple[str, str]:
|
| 220 |
-
text_client, _vision_client, _asr_client = _clients(text_backend, vision_backend)
|
| 221 |
session = draft_pact(session, text_client)
|
| 222 |
if session.phase != "error":
|
| 223 |
session = seal_pact(session)
|
| 224 |
-
return _view(session, text_backend, vision_backend)
|
|
|
|
| 107 |
return session.question_history[-1:] if session.question_history else []
|
| 108 |
|
| 109 |
|
| 110 |
+
def _view_payload(session: CustomsSession, text_backend: str, vision_backend: str, **settings) -> Dict[str, Any]:
|
| 111 |
card = session.sealed_pact or session.draft_pact
|
| 112 |
error = _latest_error(session)
|
| 113 |
status = "error" if error else "card" if session.sealed_pact else "question" if session.question_history else "declaration"
|
|
|
|
| 121 |
"card_html": _render_today_pass(card) if card else "",
|
| 122 |
"error": error,
|
| 123 |
"notice": _notice_for_status(status, error),
|
| 124 |
+
"debug": json.loads(_debug_json(session, text_backend, vision_backend, **settings)),
|
| 125 |
}
|
| 126 |
|
| 127 |
|
| 128 |
+
def _view(session: CustomsSession, text_backend: str, vision_backend: str, **settings) -> Tuple[str, str]:
|
| 129 |
+
return _state_json(session), json.dumps(
|
| 130 |
+
_view_payload(session, text_backend, vision_backend, **settings),
|
| 131 |
+
ensure_ascii=False,
|
| 132 |
+
indent=2,
|
| 133 |
+
)
|
| 134 |
|
| 135 |
|
| 136 |
def _notice_for_status(status: str, error: str = "") -> str:
|
|
|
|
| 146 |
def initial_mobile_state(
|
| 147 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 148 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
| 149 |
+
**settings,
|
| 150 |
) -> Tuple[str, str]:
|
| 151 |
+
return _view(create_session(), text_backend, vision_backend, **settings)
|
| 152 |
|
| 153 |
|
| 154 |
def submit_dream_action(
|
|
|
|
| 158 |
mood: str = "",
|
| 159 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 160 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
| 161 |
+
**settings,
|
| 162 |
) -> Tuple[str, str]:
|
| 163 |
+
text_client, vision_client, asr_client = _clients(text_backend, vision_backend, **settings)
|
| 164 |
session = add_evidence(
|
| 165 |
create_session(),
|
| 166 |
dream_text=dream_text or "",
|
|
|
|
| 174 |
previous_count = len(session.question_history)
|
| 175 |
session = ask_questions(session, text_client)
|
| 176 |
session = _trim_to_one_visible_question(session, previous_count)
|
| 177 |
+
return _view(session, text_backend, vision_backend, **settings)
|
| 178 |
|
| 179 |
|
| 180 |
def skip_to_card_action(
|
| 181 |
state: Any,
|
| 182 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 183 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
| 184 |
+
**settings,
|
| 185 |
) -> Tuple[str, str]:
|
| 186 |
session = skip_question(_session_from_state(state))
|
| 187 |
+
return _seal_view(session, text_backend, vision_backend, **settings)
|
| 188 |
|
| 189 |
|
| 190 |
def answer_to_card_action(
|
|
|
|
| 192 |
answer: str,
|
| 193 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 194 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
| 195 |
+
**settings,
|
| 196 |
) -> Tuple[str, str]:
|
| 197 |
session = answer_question(_session_from_state(state), answer or "")
|
| 198 |
if session.phase == "error":
|
| 199 |
+
return _view(session, text_backend, vision_backend, **settings)
|
| 200 |
+
return _seal_view(session, text_backend, vision_backend, **settings)
|
| 201 |
|
| 202 |
|
| 203 |
def revise_card_action(
|
|
|
|
| 205 |
revision_request: str,
|
| 206 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 207 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
| 208 |
+
**settings,
|
| 209 |
) -> Tuple[str, str]:
|
| 210 |
session = _session_from_state(state)
|
| 211 |
if session.sealed_pact and not session.draft_pact:
|
| 212 |
session.draft_pact = session.sealed_pact
|
| 213 |
+
text_client, _vision_client, _asr_client = _clients(text_backend, vision_backend, **settings)
|
| 214 |
session = revise_pact(session, revision_request or "", text_client)
|
| 215 |
if session.phase == "error":
|
| 216 |
+
return _view(session, text_backend, vision_backend, **settings)
|
| 217 |
session = seal_pact(session)
|
| 218 |
+
return _view(session, text_backend, vision_backend, **settings)
|
| 219 |
|
| 220 |
|
| 221 |
def reset_mobile_action(
|
| 222 |
text_backend: str = DEFAULT_TEXT_BACKEND,
|
| 223 |
vision_backend: str = DEFAULT_VISION_BACKEND,
|
| 224 |
+
**settings,
|
| 225 |
) -> Tuple[str, str]:
|
| 226 |
+
return initial_mobile_state(text_backend, vision_backend, **settings)
|
| 227 |
|
| 228 |
|
| 229 |
+
def _seal_view(session: CustomsSession, text_backend: str, vision_backend: str, **settings) -> Tuple[str, str]:
|
| 230 |
+
text_client, _vision_client, _asr_client = _clients(text_backend, vision_backend, **settings)
|
| 231 |
session = draft_pact(session, text_client)
|
| 232 |
if session.phase != "error":
|
| 233 |
session = seal_pact(session)
|
| 234 |
+
return _view(session, text_backend, vision_backend, **settings)
|
dream_customs/ui/app.py
CHANGED
|
@@ -4,6 +4,19 @@ from html import escape
|
|
| 4 |
import gradio as gr
|
| 5 |
from gradio_client import utils as gradio_client_utils
|
| 6 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
from dream_customs.defaults import DEFAULT_TEXT_BACKEND, DEFAULT_VISION_BACKEND
|
| 8 |
from dream_customs.ui.actions import (
|
| 9 |
answer_to_card_action,
|
|
@@ -20,6 +33,7 @@ from dream_customs.ui.copy import (
|
|
| 20 |
DREAM_PLACEHOLDER,
|
| 21 |
EXAMPLE_DREAM,
|
| 22 |
EXAMPLE_MOOD,
|
|
|
|
| 23 |
MOOD_OPTIONS,
|
| 24 |
)
|
| 25 |
from dream_customs.ui.styles import CSS
|
|
@@ -72,7 +86,50 @@ def _updates(state: str, view_json: str):
|
|
| 72 |
)
|
| 73 |
|
| 74 |
|
| 75 |
-
def
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 76 |
state, view_json = submit_dream_action(
|
| 77 |
dream_text=dream_text,
|
| 78 |
image_value=image_value,
|
|
@@ -80,42 +137,50 @@ def _submit(dream_text, image_value, audio_value, mood, text_backend, vision_bac
|
|
| 80 |
mood=mood,
|
| 81 |
text_backend=text_backend,
|
| 82 |
vision_backend=vision_backend,
|
|
|
|
| 83 |
)
|
| 84 |
return _updates(state, view_json)
|
| 85 |
|
| 86 |
|
| 87 |
-
def _answer(state, answer, text_backend, vision_backend):
|
|
|
|
| 88 |
state, view_json = answer_to_card_action(
|
| 89 |
state,
|
| 90 |
answer=answer,
|
| 91 |
text_backend=text_backend,
|
| 92 |
vision_backend=vision_backend,
|
|
|
|
| 93 |
)
|
| 94 |
return _updates(state, view_json)
|
| 95 |
|
| 96 |
|
| 97 |
-
def _skip(state, text_backend, vision_backend):
|
|
|
|
| 98 |
state, view_json = skip_to_card_action(
|
| 99 |
state,
|
| 100 |
text_backend=text_backend,
|
| 101 |
vision_backend=vision_backend,
|
|
|
|
| 102 |
)
|
| 103 |
return _updates(state, view_json)
|
| 104 |
|
| 105 |
|
| 106 |
-
def _revise(state, revision_request, text_backend, vision_backend):
|
|
|
|
| 107 |
state, view_json = revise_card_action(
|
| 108 |
state,
|
| 109 |
revision_request=revision_request,
|
| 110 |
text_backend=text_backend,
|
| 111 |
vision_backend=vision_backend,
|
|
|
|
| 112 |
)
|
| 113 |
return _updates(state, view_json)
|
| 114 |
|
| 115 |
|
| 116 |
-
def _reset(text_backend, vision_backend):
|
| 117 |
-
|
| 118 |
-
|
|
|
|
| 119 |
|
| 120 |
|
| 121 |
def build_demo() -> gr.Blocks:
|
|
@@ -138,19 +203,36 @@ def build_demo() -> gr.Blocks:
|
|
| 138 |
notice = gr.HTML(_notice_html(initial))
|
| 139 |
|
| 140 |
with gr.Group(visible=True, elem_classes=["dc-stage"]) as declaration_group:
|
| 141 |
-
|
| 142 |
-
|
| 143 |
-
|
| 144 |
-
|
| 145 |
-
|
| 146 |
-
|
| 147 |
-
|
| 148 |
-
|
| 149 |
-
|
| 150 |
-
|
| 151 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 152 |
example_button = gr.Button("试一个例子", variant="secondary")
|
| 153 |
submit_button = gr.Button("生成今日通行证", variant="primary")
|
|
|
|
|
|
|
| 154 |
|
| 155 |
with gr.Group(visible=False, elem_classes=["dc-stage", "dc-question"]) as question_group:
|
| 156 |
question_markdown = gr.Markdown("## 海关还在等梦的碎片")
|
|
@@ -180,17 +262,93 @@ def build_demo() -> gr.Blocks:
|
|
| 180 |
)
|
| 181 |
|
| 182 |
with gr.Accordion("开发者设置", open=False, elem_classes=["dc-dev"]):
|
| 183 |
-
with gr.Row():
|
| 184 |
text_backend = gr.Radio(
|
| 185 |
label="文本后端",
|
| 186 |
-
choices=["demo", "model", "ollama"],
|
| 187 |
value=DEFAULT_TEXT_BACKEND,
|
| 188 |
)
|
| 189 |
vision_backend = gr.Radio(
|
| 190 |
label="视觉后端",
|
| 191 |
-
choices=["demo", "model", "ollama"],
|
| 192 |
value=DEFAULT_VISION_BACKEND,
|
| 193 |
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 194 |
debug_json = gr.Code(
|
| 195 |
label="调试状态",
|
| 196 |
value=json.dumps(initial.get("debug", {}), ensure_ascii=False, indent=2),
|
|
@@ -209,36 +367,56 @@ def build_demo() -> gr.Blocks:
|
|
| 209 |
card_group,
|
| 210 |
debug_json,
|
| 211 |
]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 212 |
|
| 213 |
submit_button.click(
|
| 214 |
_submit,
|
| 215 |
-
inputs=[dream_text, image_input, audio_input, mood, text_backend, vision_backend],
|
| 216 |
outputs=outputs,
|
| 217 |
)
|
| 218 |
answer_button.click(
|
| 219 |
_answer,
|
| 220 |
-
inputs=[session_state, answer_text, text_backend, vision_backend],
|
| 221 |
outputs=outputs,
|
| 222 |
)
|
| 223 |
skip_button.click(
|
| 224 |
_skip,
|
| 225 |
-
inputs=[session_state, text_backend, vision_backend],
|
| 226 |
outputs=outputs,
|
| 227 |
)
|
| 228 |
gentle_button.click(
|
| 229 |
_revise,
|
| 230 |
-
inputs=[session_state, gr.State("再温柔一点"), text_backend, vision_backend],
|
| 231 |
outputs=outputs,
|
| 232 |
)
|
| 233 |
weird_button.click(
|
| 234 |
_revise,
|
| 235 |
-
inputs=[session_state, gr.State("更怪一点"), text_backend, vision_backend],
|
| 236 |
outputs=outputs,
|
| 237 |
)
|
| 238 |
copy_button.click(lambda text: text, inputs=card_text, outputs=card_text)
|
| 239 |
reset_button.click(
|
| 240 |
_reset,
|
| 241 |
-
inputs=[text_backend, vision_backend],
|
| 242 |
outputs=outputs + [dream_text, answer_text, image_input, audio_input, mood],
|
| 243 |
)
|
| 244 |
example_button.click(lambda: (EXAMPLE_DREAM, EXAMPLE_MOOD), outputs=[dream_text, mood])
|
|
|
|
| 4 |
import gradio as gr
|
| 5 |
from gradio_client import utils as gradio_client_utils
|
| 6 |
|
| 7 |
+
from dream_customs.app_logic import (
|
| 8 |
+
DEFAULT_ASR_LATENCY_BUDGET_MS,
|
| 9 |
+
DEFAULT_ASR_TIMEOUT_SECONDS,
|
| 10 |
+
DEFAULT_HOSTED_TIMEOUT_SECONDS,
|
| 11 |
+
DEFAULT_TEXT_LATENCY_BUDGET_MS,
|
| 12 |
+
DEFAULT_TEXT_MAX_TOKENS,
|
| 13 |
+
DEFAULT_TEXT_MODEL,
|
| 14 |
+
DEFAULT_TEXT_TEMPERATURE,
|
| 15 |
+
DEFAULT_VISION_LATENCY_BUDGET_MS,
|
| 16 |
+
DEFAULT_VISION_MAX_TOKENS,
|
| 17 |
+
DEFAULT_VISION_MODEL,
|
| 18 |
+
DEFAULT_VISION_TEMPERATURE,
|
| 19 |
+
)
|
| 20 |
from dream_customs.defaults import DEFAULT_TEXT_BACKEND, DEFAULT_VISION_BACKEND
|
| 21 |
from dream_customs.ui.actions import (
|
| 22 |
answer_to_card_action,
|
|
|
|
| 33 |
DREAM_PLACEHOLDER,
|
| 34 |
EXAMPLE_DREAM,
|
| 35 |
EXAMPLE_MOOD,
|
| 36 |
+
DEFAULT_MOOD,
|
| 37 |
MOOD_OPTIONS,
|
| 38 |
)
|
| 39 |
from dream_customs.ui.styles import CSS
|
|
|
|
| 86 |
)
|
| 87 |
|
| 88 |
|
| 89 |
+
def _settings_from_inputs(
|
| 90 |
+
text_endpoint,
|
| 91 |
+
vision_endpoint,
|
| 92 |
+
hosted_token,
|
| 93 |
+
ollama_url,
|
| 94 |
+
text_model,
|
| 95 |
+
vision_model,
|
| 96 |
+
text_timeout_seconds,
|
| 97 |
+
vision_timeout_seconds,
|
| 98 |
+
text_temperature,
|
| 99 |
+
vision_temperature,
|
| 100 |
+
text_max_tokens,
|
| 101 |
+
vision_max_tokens,
|
| 102 |
+
asr_backend,
|
| 103 |
+
asr_endpoint,
|
| 104 |
+
asr_timeout_seconds,
|
| 105 |
+
text_latency_budget_ms,
|
| 106 |
+
vision_latency_budget_ms,
|
| 107 |
+
asr_latency_budget_ms,
|
| 108 |
+
) -> dict:
|
| 109 |
+
return {
|
| 110 |
+
"text_endpoint": text_endpoint,
|
| 111 |
+
"vision_endpoint": vision_endpoint,
|
| 112 |
+
"hosted_token": hosted_token,
|
| 113 |
+
"ollama_url": ollama_url,
|
| 114 |
+
"text_model": text_model,
|
| 115 |
+
"vision_model": vision_model,
|
| 116 |
+
"text_timeout_seconds": text_timeout_seconds,
|
| 117 |
+
"vision_timeout_seconds": vision_timeout_seconds,
|
| 118 |
+
"text_temperature": text_temperature,
|
| 119 |
+
"vision_temperature": vision_temperature,
|
| 120 |
+
"text_max_tokens": text_max_tokens,
|
| 121 |
+
"vision_max_tokens": vision_max_tokens,
|
| 122 |
+
"asr_backend": asr_backend,
|
| 123 |
+
"asr_endpoint": asr_endpoint,
|
| 124 |
+
"asr_timeout_seconds": asr_timeout_seconds,
|
| 125 |
+
"text_latency_budget_ms": text_latency_budget_ms,
|
| 126 |
+
"vision_latency_budget_ms": vision_latency_budget_ms,
|
| 127 |
+
"asr_latency_budget_ms": asr_latency_budget_ms,
|
| 128 |
+
}
|
| 129 |
+
|
| 130 |
+
|
| 131 |
+
def _submit(dream_text, image_value, audio_value, mood, text_backend, vision_backend, *settings_values):
|
| 132 |
+
settings = _settings_from_inputs(*settings_values)
|
| 133 |
state, view_json = submit_dream_action(
|
| 134 |
dream_text=dream_text,
|
| 135 |
image_value=image_value,
|
|
|
|
| 137 |
mood=mood,
|
| 138 |
text_backend=text_backend,
|
| 139 |
vision_backend=vision_backend,
|
| 140 |
+
**settings,
|
| 141 |
)
|
| 142 |
return _updates(state, view_json)
|
| 143 |
|
| 144 |
|
| 145 |
+
def _answer(state, answer, text_backend, vision_backend, *settings_values):
|
| 146 |
+
settings = _settings_from_inputs(*settings_values)
|
| 147 |
state, view_json = answer_to_card_action(
|
| 148 |
state,
|
| 149 |
answer=answer,
|
| 150 |
text_backend=text_backend,
|
| 151 |
vision_backend=vision_backend,
|
| 152 |
+
**settings,
|
| 153 |
)
|
| 154 |
return _updates(state, view_json)
|
| 155 |
|
| 156 |
|
| 157 |
+
def _skip(state, text_backend, vision_backend, *settings_values):
|
| 158 |
+
settings = _settings_from_inputs(*settings_values)
|
| 159 |
state, view_json = skip_to_card_action(
|
| 160 |
state,
|
| 161 |
text_backend=text_backend,
|
| 162 |
vision_backend=vision_backend,
|
| 163 |
+
**settings,
|
| 164 |
)
|
| 165 |
return _updates(state, view_json)
|
| 166 |
|
| 167 |
|
| 168 |
+
def _revise(state, revision_request, text_backend, vision_backend, *settings_values):
|
| 169 |
+
settings = _settings_from_inputs(*settings_values)
|
| 170 |
state, view_json = revise_card_action(
|
| 171 |
state,
|
| 172 |
revision_request=revision_request,
|
| 173 |
text_backend=text_backend,
|
| 174 |
vision_backend=vision_backend,
|
| 175 |
+
**settings,
|
| 176 |
)
|
| 177 |
return _updates(state, view_json)
|
| 178 |
|
| 179 |
|
| 180 |
+
def _reset(text_backend, vision_backend, *settings_values):
|
| 181 |
+
settings = _settings_from_inputs(*settings_values)
|
| 182 |
+
state, view_json = reset_mobile_action(text_backend=text_backend, vision_backend=vision_backend, **settings)
|
| 183 |
+
return (*_updates(state, view_json), "", "", None, None, DEFAULT_MOOD)
|
| 184 |
|
| 185 |
|
| 186 |
def build_demo() -> gr.Blocks:
|
|
|
|
| 203 |
notice = gr.HTML(_notice_html(initial))
|
| 204 |
|
| 205 |
with gr.Group(visible=True, elem_classes=["dc-stage"]) as declaration_group:
|
| 206 |
+
with gr.Row(elem_classes=["dc-intake-grid"]):
|
| 207 |
+
with gr.Group(elem_classes=["dc-composer"]):
|
| 208 |
+
dream_text = gr.Textbox(
|
| 209 |
+
label="写下梦的碎片",
|
| 210 |
+
placeholder=DREAM_PLACEHOLDER,
|
| 211 |
+
lines=8,
|
| 212 |
+
value="",
|
| 213 |
+
elem_classes=["dc-dream-text"],
|
| 214 |
+
)
|
| 215 |
+
audio_input = gr.Audio(
|
| 216 |
+
label="语音线索",
|
| 217 |
+
sources=["microphone"],
|
| 218 |
+
type="filepath",
|
| 219 |
+
elem_classes=["dc-mic-input"],
|
| 220 |
+
)
|
| 221 |
+
with gr.Column(elem_classes=["dc-side-panel"]):
|
| 222 |
+
mood = gr.Dropdown(label="醒来后的感觉", choices=MOOD_OPTIONS, value=DEFAULT_MOOD)
|
| 223 |
+
gr.HTML(
|
| 224 |
+
"""
|
| 225 |
+
<div class="dc-side-stamp">
|
| 226 |
+
<span>Dream Customs</span>
|
| 227 |
+
<strong>Calm clearance</strong>
|
| 228 |
+
</div>
|
| 229 |
+
""".strip()
|
| 230 |
+
)
|
| 231 |
+
with gr.Row(elem_classes=["dc-submit-row"]):
|
| 232 |
example_button = gr.Button("试一个例子", variant="secondary")
|
| 233 |
submit_button = gr.Button("生成今日通行证", variant="primary")
|
| 234 |
+
with gr.Accordion("附加材料", open=False, elem_classes=["dc-attachment-drawer"]):
|
| 235 |
+
image_input = gr.Image(label="图片线索", type="filepath", height=160)
|
| 236 |
|
| 237 |
with gr.Group(visible=False, elem_classes=["dc-stage", "dc-question"]) as question_group:
|
| 238 |
question_markdown = gr.Markdown("## 海关还在等梦的碎片")
|
|
|
|
| 262 |
)
|
| 263 |
|
| 264 |
with gr.Accordion("开发者设置", open=False, elem_classes=["dc-dev"]):
|
| 265 |
+
with gr.Row(elem_classes=["dc-dev-grid"]):
|
| 266 |
text_backend = gr.Radio(
|
| 267 |
label="文本后端",
|
| 268 |
+
choices=["demo", "model", "modal", "huggingface", "ollama"],
|
| 269 |
value=DEFAULT_TEXT_BACKEND,
|
| 270 |
)
|
| 271 |
vision_backend = gr.Radio(
|
| 272 |
label="视觉后端",
|
| 273 |
+
choices=["demo", "model", "modal", "huggingface", "ollama"],
|
| 274 |
value=DEFAULT_VISION_BACKEND,
|
| 275 |
)
|
| 276 |
+
asr_backend = gr.Radio(
|
| 277 |
+
label="ASR 后端",
|
| 278 |
+
choices=["demo", "modal", "huggingface"],
|
| 279 |
+
value="demo",
|
| 280 |
+
)
|
| 281 |
+
with gr.Row(elem_classes=["dc-dev-grid"]):
|
| 282 |
+
text_endpoint = gr.Textbox(label="文本 Endpoint", value="")
|
| 283 |
+
vision_endpoint = gr.Textbox(label="视觉 Endpoint", value="")
|
| 284 |
+
asr_endpoint = gr.Textbox(label="ASR Endpoint", value="")
|
| 285 |
+
hosted_token = gr.Textbox(label="Hosted Token", value="", type="password")
|
| 286 |
+
with gr.Row(elem_classes=["dc-dev-grid"]):
|
| 287 |
+
text_model = gr.Textbox(label="文本模型", value=DEFAULT_TEXT_MODEL)
|
| 288 |
+
vision_model = gr.Textbox(label="视觉模型", value=DEFAULT_VISION_MODEL)
|
| 289 |
+
ollama_url = gr.Textbox(label="Ollama URL", value="http://localhost:11434")
|
| 290 |
+
with gr.Row(elem_classes=["dc-dev-grid"]):
|
| 291 |
+
text_timeout_seconds = gr.Number(
|
| 292 |
+
label="文本超时秒",
|
| 293 |
+
value=DEFAULT_HOSTED_TIMEOUT_SECONDS,
|
| 294 |
+
precision=1,
|
| 295 |
+
)
|
| 296 |
+
vision_timeout_seconds = gr.Number(
|
| 297 |
+
label="视觉超时秒",
|
| 298 |
+
value=DEFAULT_HOSTED_TIMEOUT_SECONDS,
|
| 299 |
+
precision=1,
|
| 300 |
+
)
|
| 301 |
+
asr_timeout_seconds = gr.Number(
|
| 302 |
+
label="ASR 超时秒",
|
| 303 |
+
value=DEFAULT_ASR_TIMEOUT_SECONDS,
|
| 304 |
+
precision=1,
|
| 305 |
+
)
|
| 306 |
+
with gr.Row(elem_classes=["dc-dev-grid"]):
|
| 307 |
+
text_latency_budget_ms = gr.Number(
|
| 308 |
+
label="Modal 文本延迟预算 ms",
|
| 309 |
+
value=DEFAULT_TEXT_LATENCY_BUDGET_MS,
|
| 310 |
+
precision=0,
|
| 311 |
+
)
|
| 312 |
+
vision_latency_budget_ms = gr.Number(
|
| 313 |
+
label="Modal 视觉延迟预算 ms",
|
| 314 |
+
value=DEFAULT_VISION_LATENCY_BUDGET_MS,
|
| 315 |
+
precision=0,
|
| 316 |
+
)
|
| 317 |
+
asr_latency_budget_ms = gr.Number(
|
| 318 |
+
label="ASR 延迟预算 ms",
|
| 319 |
+
value=DEFAULT_ASR_LATENCY_BUDGET_MS,
|
| 320 |
+
precision=0,
|
| 321 |
+
)
|
| 322 |
+
with gr.Row(elem_classes=["dc-dev-grid"]):
|
| 323 |
+
text_temperature = gr.Slider(
|
| 324 |
+
label="文本温度",
|
| 325 |
+
minimum=0,
|
| 326 |
+
maximum=0.7,
|
| 327 |
+
step=0.05,
|
| 328 |
+
value=DEFAULT_TEXT_TEMPERATURE,
|
| 329 |
+
)
|
| 330 |
+
vision_temperature = gr.Slider(
|
| 331 |
+
label="视觉温度",
|
| 332 |
+
minimum=0,
|
| 333 |
+
maximum=0.7,
|
| 334 |
+
step=0.05,
|
| 335 |
+
value=DEFAULT_VISION_TEMPERATURE,
|
| 336 |
+
)
|
| 337 |
+
with gr.Row(elem_classes=["dc-dev-grid"]):
|
| 338 |
+
text_max_tokens = gr.Slider(
|
| 339 |
+
label="文本 max tokens",
|
| 340 |
+
minimum=64,
|
| 341 |
+
maximum=1200,
|
| 342 |
+
step=1,
|
| 343 |
+
value=DEFAULT_TEXT_MAX_TOKENS,
|
| 344 |
+
)
|
| 345 |
+
vision_max_tokens = gr.Slider(
|
| 346 |
+
label="视觉 max tokens",
|
| 347 |
+
minimum=64,
|
| 348 |
+
maximum=800,
|
| 349 |
+
step=1,
|
| 350 |
+
value=DEFAULT_VISION_MAX_TOKENS,
|
| 351 |
+
)
|
| 352 |
debug_json = gr.Code(
|
| 353 |
label="调试状态",
|
| 354 |
value=json.dumps(initial.get("debug", {}), ensure_ascii=False, indent=2),
|
|
|
|
| 367 |
card_group,
|
| 368 |
debug_json,
|
| 369 |
]
|
| 370 |
+
settings_inputs = [
|
| 371 |
+
text_endpoint,
|
| 372 |
+
vision_endpoint,
|
| 373 |
+
hosted_token,
|
| 374 |
+
ollama_url,
|
| 375 |
+
text_model,
|
| 376 |
+
vision_model,
|
| 377 |
+
text_timeout_seconds,
|
| 378 |
+
vision_timeout_seconds,
|
| 379 |
+
text_temperature,
|
| 380 |
+
vision_temperature,
|
| 381 |
+
text_max_tokens,
|
| 382 |
+
vision_max_tokens,
|
| 383 |
+
asr_backend,
|
| 384 |
+
asr_endpoint,
|
| 385 |
+
asr_timeout_seconds,
|
| 386 |
+
text_latency_budget_ms,
|
| 387 |
+
vision_latency_budget_ms,
|
| 388 |
+
asr_latency_budget_ms,
|
| 389 |
+
]
|
| 390 |
|
| 391 |
submit_button.click(
|
| 392 |
_submit,
|
| 393 |
+
inputs=[dream_text, image_input, audio_input, mood, text_backend, vision_backend] + settings_inputs,
|
| 394 |
outputs=outputs,
|
| 395 |
)
|
| 396 |
answer_button.click(
|
| 397 |
_answer,
|
| 398 |
+
inputs=[session_state, answer_text, text_backend, vision_backend] + settings_inputs,
|
| 399 |
outputs=outputs,
|
| 400 |
)
|
| 401 |
skip_button.click(
|
| 402 |
_skip,
|
| 403 |
+
inputs=[session_state, text_backend, vision_backend] + settings_inputs,
|
| 404 |
outputs=outputs,
|
| 405 |
)
|
| 406 |
gentle_button.click(
|
| 407 |
_revise,
|
| 408 |
+
inputs=[session_state, gr.State("再温柔一点"), text_backend, vision_backend] + settings_inputs,
|
| 409 |
outputs=outputs,
|
| 410 |
)
|
| 411 |
weird_button.click(
|
| 412 |
_revise,
|
| 413 |
+
inputs=[session_state, gr.State("更怪一点"), text_backend, vision_backend] + settings_inputs,
|
| 414 |
outputs=outputs,
|
| 415 |
)
|
| 416 |
copy_button.click(lambda text: text, inputs=card_text, outputs=card_text)
|
| 417 |
reset_button.click(
|
| 418 |
_reset,
|
| 419 |
+
inputs=[text_backend, vision_backend] + settings_inputs,
|
| 420 |
outputs=outputs + [dream_text, answer_text, image_input, audio_input, mood],
|
| 421 |
)
|
| 422 |
example_button.click(lambda: (EXAMPLE_DREAM, EXAMPLE_MOOD), outputs=[dream_text, mood])
|
dream_customs/ui/copy.py
CHANGED
|
@@ -7,4 +7,5 @@ ANSWER_PLACEHOLDER = "可以只回一句,也可以留空后直接跳过。"
|
|
| 7 |
EXAMPLE_DREAM = "我梦见自己一直赶不上电梯,按钮像蜡一样融化,楼层数字停在 14。"
|
| 8 |
EXAMPLE_MOOD = "焦虑"
|
| 9 |
|
|
|
|
| 10 |
MOOD_OPTIONS = ["迷雾", "焦虑", "好奇", "累", "平静"]
|
|
|
|
| 7 |
EXAMPLE_DREAM = "我梦见自己一直赶不上电梯,按钮像蜡一样融化,楼层数字停在 14。"
|
| 8 |
EXAMPLE_MOOD = "焦虑"
|
| 9 |
|
| 10 |
+
DEFAULT_MOOD = "平静"
|
| 11 |
MOOD_OPTIONS = ["迷雾", "焦虑", "好奇", "累", "平静"]
|
dream_customs/ui/styles.py
CHANGED
|
@@ -6,11 +6,14 @@ CSS = """
|
|
| 6 |
--dc-line: #cbd7d1;
|
| 7 |
--dc-panel: #ffffff;
|
| 8 |
--dc-panel-soft: #edf4f0;
|
|
|
|
|
|
|
| 9 |
--dc-teal: #0f766e;
|
| 10 |
--dc-teal-dark: #115e59;
|
| 11 |
--dc-coral: #c85235;
|
| 12 |
--dc-gold: #b7791f;
|
| 13 |
-
--dc-shadow: 0
|
|
|
|
| 14 |
--dc-radius-sm: 6px;
|
| 15 |
--dc-radius-md: 8px;
|
| 16 |
}
|
|
@@ -33,14 +36,14 @@ body,
|
|
| 33 |
|
| 34 |
.dc-shell {
|
| 35 |
margin: 0 auto;
|
| 36 |
-
max-width:
|
| 37 |
-
padding:
|
| 38 |
}
|
| 39 |
|
| 40 |
.dc-hero {
|
| 41 |
border-bottom: 1px solid var(--dc-line);
|
| 42 |
-
margin-bottom:
|
| 43 |
-
padding: 4px 0
|
| 44 |
}
|
| 45 |
|
| 46 |
.dc-hero h1 {
|
|
@@ -65,7 +68,7 @@ body,
|
|
| 65 |
border: 1px solid var(--dc-line) !important;
|
| 66 |
border-radius: var(--dc-radius-md) !important;
|
| 67 |
box-shadow: var(--dc-shadow);
|
| 68 |
-
padding: clamp(
|
| 69 |
}
|
| 70 |
|
| 71 |
.dc-stage textarea,
|
|
@@ -83,6 +86,124 @@ body,
|
|
| 83 |
color: var(--dc-ink) !important;
|
| 84 |
}
|
| 85 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 86 |
.dc-stage label,
|
| 87 |
.dc-stage [data-testid="block-label"],
|
| 88 |
.dc-dev label,
|
|
@@ -98,9 +219,9 @@ body,
|
|
| 98 |
}
|
| 99 |
|
| 100 |
.dc-stage button {
|
| 101 |
-
border-radius: var(--dc-radius-
|
| 102 |
font-weight: 750 !important;
|
| 103 |
-
min-height:
|
| 104 |
}
|
| 105 |
|
| 106 |
.dc-stage .primary button,
|
|
@@ -112,11 +233,15 @@ button.primary {
|
|
| 112 |
}
|
| 113 |
|
| 114 |
.dc-stage button.secondary {
|
| 115 |
-
background:
|
| 116 |
border-color: var(--dc-line) !important;
|
| 117 |
color: var(--dc-ink) !important;
|
| 118 |
}
|
| 119 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 120 |
.dc-row {
|
| 121 |
align-items: stretch !important;
|
| 122 |
display: grid !important;
|
|
@@ -239,7 +364,23 @@ button.primary {
|
|
| 239 |
|
| 240 |
.dc-dev {
|
| 241 |
box-shadow: none;
|
| 242 |
-
margin-top:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 243 |
}
|
| 244 |
|
| 245 |
.dc-dev textarea {
|
|
@@ -249,7 +390,7 @@ button.primary {
|
|
| 249 |
|
| 250 |
@media (max-width: 720px) {
|
| 251 |
.dc-shell {
|
| 252 |
-
padding:
|
| 253 |
}
|
| 254 |
|
| 255 |
.dc-hero {
|
|
@@ -257,16 +398,35 @@ button.primary {
|
|
| 257 |
padding-bottom: 12px;
|
| 258 |
}
|
| 259 |
|
|
|
|
|
|
|
| 260 |
.dc-row,
|
|
|
|
| 261 |
.dc-actions {
|
| 262 |
grid-template-columns: 1fr;
|
| 263 |
}
|
| 264 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 265 |
.dc-stage,
|
| 266 |
.dc-dev {
|
| 267 |
border-radius: 8px !important;
|
| 268 |
box-shadow: none;
|
| 269 |
-
padding:
|
| 270 |
}
|
| 271 |
}
|
| 272 |
"""
|
|
|
|
| 6 |
--dc-line: #cbd7d1;
|
| 7 |
--dc-panel: #ffffff;
|
| 8 |
--dc-panel-soft: #edf4f0;
|
| 9 |
+
--dc-paper: #fffaf1;
|
| 10 |
+
--dc-graphite: #26302d;
|
| 11 |
--dc-teal: #0f766e;
|
| 12 |
--dc-teal-dark: #115e59;
|
| 13 |
--dc-coral: #c85235;
|
| 14 |
--dc-gold: #b7791f;
|
| 15 |
+
--dc-shadow: 0 24px 70px rgba(23, 33, 31, 0.13);
|
| 16 |
+
--dc-soft-shadow: 0 10px 28px rgba(23, 33, 31, 0.08);
|
| 17 |
--dc-radius-sm: 6px;
|
| 18 |
--dc-radius-md: 8px;
|
| 19 |
}
|
|
|
|
| 36 |
|
| 37 |
.dc-shell {
|
| 38 |
margin: 0 auto;
|
| 39 |
+
max-width: 1060px;
|
| 40 |
+
padding: 28px clamp(18px, 4vw, 46px) 34px;
|
| 41 |
}
|
| 42 |
|
| 43 |
.dc-hero {
|
| 44 |
border-bottom: 1px solid var(--dc-line);
|
| 45 |
+
margin-bottom: 20px;
|
| 46 |
+
padding: 4px 0 20px;
|
| 47 |
}
|
| 48 |
|
| 49 |
.dc-hero h1 {
|
|
|
|
| 68 |
border: 1px solid var(--dc-line) !important;
|
| 69 |
border-radius: var(--dc-radius-md) !important;
|
| 70 |
box-shadow: var(--dc-shadow);
|
| 71 |
+
padding: clamp(18px, 3vw, 30px);
|
| 72 |
}
|
| 73 |
|
| 74 |
.dc-stage textarea,
|
|
|
|
| 86 |
color: var(--dc-ink) !important;
|
| 87 |
}
|
| 88 |
|
| 89 |
+
.dc-intake-grid {
|
| 90 |
+
align-items: stretch !important;
|
| 91 |
+
display: grid !important;
|
| 92 |
+
gap: clamp(16px, 3vw, 24px) !important;
|
| 93 |
+
grid-template-columns: minmax(0, 1fr) minmax(210px, 270px);
|
| 94 |
+
}
|
| 95 |
+
|
| 96 |
+
.dc-composer {
|
| 97 |
+
background:
|
| 98 |
+
linear-gradient(180deg, rgba(255, 250, 241, 0.86), rgba(251, 253, 252, 0.98)) !important;
|
| 99 |
+
border: 1px solid color-mix(in srgb, var(--dc-line) 82%, var(--dc-gold)) !important;
|
| 100 |
+
border-radius: var(--dc-radius-md) !important;
|
| 101 |
+
box-shadow: var(--dc-soft-shadow);
|
| 102 |
+
min-height: 300px;
|
| 103 |
+
padding: 12px;
|
| 104 |
+
position: relative;
|
| 105 |
+
}
|
| 106 |
+
|
| 107 |
+
.dc-composer,
|
| 108 |
+
.dc-composer > div,
|
| 109 |
+
.dc-composer .form {
|
| 110 |
+
overflow: visible !important;
|
| 111 |
+
}
|
| 112 |
+
|
| 113 |
+
.dc-dream-text textarea {
|
| 114 |
+
min-height: 248px !important;
|
| 115 |
+
padding-bottom: 78px !important;
|
| 116 |
+
resize: vertical !important;
|
| 117 |
+
}
|
| 118 |
+
|
| 119 |
+
.dc-mic-input {
|
| 120 |
+
bottom: 16px;
|
| 121 |
+
max-width: 132px;
|
| 122 |
+
min-width: 112px !important;
|
| 123 |
+
position: absolute !important;
|
| 124 |
+
right: 16px;
|
| 125 |
+
z-index: 4;
|
| 126 |
+
}
|
| 127 |
+
|
| 128 |
+
.dc-mic-input label,
|
| 129 |
+
.dc-mic-input [data-testid="block-label"] {
|
| 130 |
+
display: none !important;
|
| 131 |
+
}
|
| 132 |
+
|
| 133 |
+
.dc-mic-input,
|
| 134 |
+
.dc-mic-input .wrap,
|
| 135 |
+
.dc-mic-input .container,
|
| 136 |
+
.dc-mic-input .audio-container,
|
| 137 |
+
.dc-mic-input .component-wrapper {
|
| 138 |
+
background: transparent !important;
|
| 139 |
+
border: 0 !important;
|
| 140 |
+
box-shadow: none !important;
|
| 141 |
+
min-height: 46px !important;
|
| 142 |
+
}
|
| 143 |
+
|
| 144 |
+
.dc-mic-input .audio-container,
|
| 145 |
+
.dc-mic-input .wrap {
|
| 146 |
+
height: 46px !important;
|
| 147 |
+
overflow: hidden !important;
|
| 148 |
+
}
|
| 149 |
+
|
| 150 |
+
.dc-mic-input button {
|
| 151 |
+
background: var(--dc-graphite) !important;
|
| 152 |
+
border-color: var(--dc-graphite) !important;
|
| 153 |
+
border-radius: var(--dc-radius-sm) !important;
|
| 154 |
+
color: #ffffff !important;
|
| 155 |
+
min-height: 40px !important;
|
| 156 |
+
}
|
| 157 |
+
|
| 158 |
+
.dc-side-panel {
|
| 159 |
+
background: var(--dc-paper) !important;
|
| 160 |
+
border: 1px solid #e3d6bd !important;
|
| 161 |
+
border-radius: var(--dc-radius-md) !important;
|
| 162 |
+
box-shadow: var(--dc-soft-shadow);
|
| 163 |
+
justify-content: space-between;
|
| 164 |
+
min-height: 300px;
|
| 165 |
+
padding: 14px;
|
| 166 |
+
}
|
| 167 |
+
|
| 168 |
+
.dc-side-stamp {
|
| 169 |
+
border: 1px dashed rgba(200, 82, 53, 0.55);
|
| 170 |
+
border-radius: var(--dc-radius-md);
|
| 171 |
+
color: var(--dc-coral);
|
| 172 |
+
margin-top: 12px;
|
| 173 |
+
padding: 14px;
|
| 174 |
+
text-transform: none;
|
| 175 |
+
}
|
| 176 |
+
|
| 177 |
+
.dc-side-stamp span {
|
| 178 |
+
display: block;
|
| 179 |
+
font-size: 0.72rem;
|
| 180 |
+
font-weight: 850;
|
| 181 |
+
margin-bottom: 6px;
|
| 182 |
+
}
|
| 183 |
+
|
| 184 |
+
.dc-side-stamp strong {
|
| 185 |
+
color: var(--dc-graphite);
|
| 186 |
+
display: block;
|
| 187 |
+
font-size: 1.12rem;
|
| 188 |
+
}
|
| 189 |
+
|
| 190 |
+
.dc-attachment-drawer {
|
| 191 |
+
background: transparent !important;
|
| 192 |
+
border: 1px solid var(--dc-line) !important;
|
| 193 |
+
border-radius: var(--dc-radius-md) !important;
|
| 194 |
+
box-shadow: none !important;
|
| 195 |
+
margin-top: 16px;
|
| 196 |
+
}
|
| 197 |
+
|
| 198 |
+
.dc-submit-row {
|
| 199 |
+
align-items: stretch !important;
|
| 200 |
+
background: transparent !important;
|
| 201 |
+
display: grid !important;
|
| 202 |
+
gap: 14px !important;
|
| 203 |
+
grid-template-columns: minmax(0, 0.9fr) minmax(0, 1.1fr);
|
| 204 |
+
margin-top: 18px;
|
| 205 |
+
}
|
| 206 |
+
|
| 207 |
.dc-stage label,
|
| 208 |
.dc-stage [data-testid="block-label"],
|
| 209 |
.dc-dev label,
|
|
|
|
| 219 |
}
|
| 220 |
|
| 221 |
.dc-stage button {
|
| 222 |
+
border-radius: var(--dc-radius-md) !important;
|
| 223 |
font-weight: 750 !important;
|
| 224 |
+
min-height: 48px !important;
|
| 225 |
}
|
| 226 |
|
| 227 |
.dc-stage .primary button,
|
|
|
|
| 233 |
}
|
| 234 |
|
| 235 |
.dc-stage button.secondary {
|
| 236 |
+
background: #f3eee3 !important;
|
| 237 |
border-color: var(--dc-line) !important;
|
| 238 |
color: var(--dc-ink) !important;
|
| 239 |
}
|
| 240 |
|
| 241 |
+
.dc-submit-row button {
|
| 242 |
+
box-shadow: var(--dc-soft-shadow);
|
| 243 |
+
}
|
| 244 |
+
|
| 245 |
.dc-row {
|
| 246 |
align-items: stretch !important;
|
| 247 |
display: grid !important;
|
|
|
|
| 364 |
|
| 365 |
.dc-dev {
|
| 366 |
box-shadow: none;
|
| 367 |
+
margin-top: 16px;
|
| 368 |
+
}
|
| 369 |
+
|
| 370 |
+
.dc-dev-grid {
|
| 371 |
+
align-items: stretch !important;
|
| 372 |
+
display: grid !important;
|
| 373 |
+
gap: 12px !important;
|
| 374 |
+
grid-template-columns: repeat(3, minmax(0, 1fr));
|
| 375 |
+
}
|
| 376 |
+
|
| 377 |
+
.dc-dev .form,
|
| 378 |
+
.dc-dev .wrap,
|
| 379 |
+
.dc-dev .container,
|
| 380 |
+
.dc-dev input,
|
| 381 |
+
.dc-dev textarea,
|
| 382 |
+
.dc-dev select {
|
| 383 |
+
border-radius: var(--dc-radius-sm) !important;
|
| 384 |
}
|
| 385 |
|
| 386 |
.dc-dev textarea {
|
|
|
|
| 390 |
|
| 391 |
@media (max-width: 720px) {
|
| 392 |
.dc-shell {
|
| 393 |
+
padding: 16px 12px 24px;
|
| 394 |
}
|
| 395 |
|
| 396 |
.dc-hero {
|
|
|
|
| 398 |
padding-bottom: 12px;
|
| 399 |
}
|
| 400 |
|
| 401 |
+
.dc-intake-grid,
|
| 402 |
+
.dc-dev-grid,
|
| 403 |
.dc-row,
|
| 404 |
+
.dc-submit-row,
|
| 405 |
.dc-actions {
|
| 406 |
grid-template-columns: 1fr;
|
| 407 |
}
|
| 408 |
|
| 409 |
+
.dc-side-panel {
|
| 410 |
+
min-height: 0;
|
| 411 |
+
}
|
| 412 |
+
|
| 413 |
+
.dc-side-stamp {
|
| 414 |
+
display: none;
|
| 415 |
+
}
|
| 416 |
+
|
| 417 |
+
.dc-composer {
|
| 418 |
+
min-height: 270px;
|
| 419 |
+
}
|
| 420 |
+
|
| 421 |
+
.dc-dream-text textarea {
|
| 422 |
+
min-height: 220px !important;
|
| 423 |
+
}
|
| 424 |
+
|
| 425 |
.dc-stage,
|
| 426 |
.dc-dev {
|
| 427 |
border-radius: 8px !important;
|
| 428 |
box-shadow: none;
|
| 429 |
+
padding: 14px;
|
| 430 |
}
|
| 431 |
}
|
| 432 |
"""
|
modal_backend/__init__.py
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
"""Modal backend package for Dream Customs."""
|
modal_backend/contracts.py
ADDED
|
@@ -0,0 +1,71 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import base64
|
| 2 |
+
from typing import Any, Dict
|
| 3 |
+
|
| 4 |
+
|
| 5 |
+
class AuthError(Exception):
|
| 6 |
+
"""Raised when a hosted route request is not authorized."""
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
def _clean_text(value: Any) -> str:
|
| 10 |
+
if value is None:
|
| 11 |
+
return ""
|
| 12 |
+
return str(value).strip()
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
def normalize_text_payload(payload: Dict[str, Any]) -> Dict[str, Any]:
|
| 16 |
+
prompt = _clean_text(payload.get("prompt"))
|
| 17 |
+
if not prompt:
|
| 18 |
+
messages = payload.get("messages")
|
| 19 |
+
if isinstance(messages, list):
|
| 20 |
+
parts = []
|
| 21 |
+
for item in messages:
|
| 22 |
+
if isinstance(item, dict):
|
| 23 |
+
content = _clean_text(item.get("content"))
|
| 24 |
+
if content:
|
| 25 |
+
parts.append(content)
|
| 26 |
+
prompt = "\n".join(parts).strip()
|
| 27 |
+
max_tokens = payload.get("max_tokens", 700)
|
| 28 |
+
try:
|
| 29 |
+
max_tokens = int(max_tokens)
|
| 30 |
+
except (TypeError, ValueError):
|
| 31 |
+
max_tokens = 700
|
| 32 |
+
max_tokens = max(64, min(max_tokens, 1200))
|
| 33 |
+
temperature = payload.get("temperature", 0.2)
|
| 34 |
+
try:
|
| 35 |
+
temperature = float(temperature)
|
| 36 |
+
except (TypeError, ValueError):
|
| 37 |
+
temperature = 0.2
|
| 38 |
+
temperature = max(0.0, min(temperature, 0.7))
|
| 39 |
+
return {
|
| 40 |
+
"prompt": prompt,
|
| 41 |
+
"max_tokens": max_tokens,
|
| 42 |
+
"temperature": temperature,
|
| 43 |
+
}
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
def decode_image_payload(payload: Dict[str, Any]) -> bytes:
|
| 47 |
+
encoded = payload.get("image")
|
| 48 |
+
if not encoded and isinstance(payload.get("images"), list) and payload["images"]:
|
| 49 |
+
encoded = payload["images"][0]
|
| 50 |
+
if not encoded:
|
| 51 |
+
raise ValueError("Missing image payload.")
|
| 52 |
+
if isinstance(encoded, bytes):
|
| 53 |
+
encoded = encoded.decode("ascii")
|
| 54 |
+
return base64.b64decode(str(encoded))
|
| 55 |
+
|
| 56 |
+
|
| 57 |
+
def ensure_authorized(authorization_header: str, expected_token: str) -> None:
|
| 58 |
+
expected_token = expected_token.strip()
|
| 59 |
+
if not expected_token:
|
| 60 |
+
return
|
| 61 |
+
header = authorization_header.strip()
|
| 62 |
+
if header.lower().startswith("bearer "):
|
| 63 |
+
supplied = header.split(" ", 1)[1].strip()
|
| 64 |
+
else:
|
| 65 |
+
supplied = header
|
| 66 |
+
if supplied != expected_token:
|
| 67 |
+
raise AuthError("Unauthorized hosted route request.")
|
| 68 |
+
|
| 69 |
+
|
| 70 |
+
def response_payload(text: str) -> Dict[str, str]:
|
| 71 |
+
return {"response": text}
|
modal_backend/dream_customs_modal.py
ADDED
|
@@ -0,0 +1,392 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import io
|
| 2 |
+
import json
|
| 3 |
+
import os
|
| 4 |
+
import tempfile
|
| 5 |
+
from typing import Any, Dict, Optional
|
| 6 |
+
|
| 7 |
+
import modal
|
| 8 |
+
from fastapi import Body, Header
|
| 9 |
+
|
| 10 |
+
from modal_backend.contracts import (
|
| 11 |
+
AuthError,
|
| 12 |
+
decode_image_payload,
|
| 13 |
+
ensure_authorized,
|
| 14 |
+
normalize_text_payload,
|
| 15 |
+
response_payload,
|
| 16 |
+
)
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
APP_NAME = "dream-customs-minicpm-backend"
|
| 20 |
+
TEXT_MODEL = "openbmb/MiniCPM5-1B"
|
| 21 |
+
VISION_MODEL = "openbmb/MiniCPM-V-4.6"
|
| 22 |
+
MINUTES = 60
|
| 23 |
+
|
| 24 |
+
app = modal.App(APP_NAME)
|
| 25 |
+
|
| 26 |
+
hf_cache = modal.Volume.from_name("dream-customs-hf-cache", create_if_missing=True)
|
| 27 |
+
|
| 28 |
+
health_image = (
|
| 29 |
+
modal.Image.debian_slim(python_version="3.11")
|
| 30 |
+
.pip_install("fastapi[standard]")
|
| 31 |
+
.add_local_dir("modal_backend", remote_path="/root/modal_backend")
|
| 32 |
+
)
|
| 33 |
+
|
| 34 |
+
image = (
|
| 35 |
+
modal.Image.debian_slim(python_version="3.11")
|
| 36 |
+
.apt_install("git")
|
| 37 |
+
.pip_install(
|
| 38 |
+
"accelerate",
|
| 39 |
+
"einops",
|
| 40 |
+
"fastapi[standard]",
|
| 41 |
+
"pillow",
|
| 42 |
+
"protobuf",
|
| 43 |
+
"sentencepiece",
|
| 44 |
+
"torch",
|
| 45 |
+
"torchvision",
|
| 46 |
+
"transformers>=4.56",
|
| 47 |
+
)
|
| 48 |
+
.add_local_dir("modal_backend", remote_path="/root/modal_backend")
|
| 49 |
+
)
|
| 50 |
+
|
| 51 |
+
secrets = [
|
| 52 |
+
modal.Secret.from_name(
|
| 53 |
+
"dream-customs-modal-secrets",
|
| 54 |
+
required_keys=["HF_TOKEN", "DREAM_CUSTOMS_HOSTED_TOKEN"],
|
| 55 |
+
)
|
| 56 |
+
]
|
| 57 |
+
|
| 58 |
+
_TEXT_PIPE = None
|
| 59 |
+
_VISION_PIPE = None
|
| 60 |
+
|
| 61 |
+
|
| 62 |
+
def _expected_token() -> str:
|
| 63 |
+
return os.getenv("DREAM_CUSTOMS_HOSTED_TOKEN", "").strip()
|
| 64 |
+
|
| 65 |
+
|
| 66 |
+
def _json_error(message: str, status: str = "error") -> Dict[str, str]:
|
| 67 |
+
return {"status": status, "response": "", "error": message}
|
| 68 |
+
|
| 69 |
+
|
| 70 |
+
def _stringify_pipeline_result(result: Any) -> str:
|
| 71 |
+
if isinstance(result, str):
|
| 72 |
+
return result.strip()
|
| 73 |
+
if isinstance(result, list) and result:
|
| 74 |
+
return _stringify_pipeline_result(result[0])
|
| 75 |
+
if isinstance(result, dict):
|
| 76 |
+
for key in ("generated_text", "text", "output", "response"):
|
| 77 |
+
value = result.get(key)
|
| 78 |
+
if isinstance(value, str):
|
| 79 |
+
return value.strip()
|
| 80 |
+
return str(result).strip()
|
| 81 |
+
return str(result).strip()
|
| 82 |
+
|
| 83 |
+
|
| 84 |
+
def _load_text_pipe():
|
| 85 |
+
global _TEXT_PIPE
|
| 86 |
+
if _TEXT_PIPE is None:
|
| 87 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 88 |
+
|
| 89 |
+
model_id = os.getenv("DREAM_CUSTOMS_TEXT_MODEL", TEXT_MODEL)
|
| 90 |
+
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
| 91 |
+
model = AutoModelForCausalLM.from_pretrained(
|
| 92 |
+
model_id,
|
| 93 |
+
device_map="auto",
|
| 94 |
+
torch_dtype="auto",
|
| 95 |
+
trust_remote_code=True,
|
| 96 |
+
)
|
| 97 |
+
_TEXT_PIPE = (tokenizer, model)
|
| 98 |
+
return _TEXT_PIPE
|
| 99 |
+
|
| 100 |
+
|
| 101 |
+
def _load_vision_pipe():
|
| 102 |
+
global _VISION_PIPE
|
| 103 |
+
if _VISION_PIPE is None:
|
| 104 |
+
from transformers import pipeline
|
| 105 |
+
|
| 106 |
+
_VISION_PIPE = pipeline(
|
| 107 |
+
"image-text-to-text",
|
| 108 |
+
model=os.getenv("DREAM_CUSTOMS_VISION_MODEL", VISION_MODEL),
|
| 109 |
+
device_map="auto",
|
| 110 |
+
torch_dtype="auto",
|
| 111 |
+
trust_remote_code=True,
|
| 112 |
+
)
|
| 113 |
+
return _VISION_PIPE
|
| 114 |
+
|
| 115 |
+
|
| 116 |
+
def _messages_from_payload(payload: Dict[str, Any], prompt: str) -> list[Dict[str, str]]:
|
| 117 |
+
messages = payload.get("messages")
|
| 118 |
+
if isinstance(messages, list) and messages:
|
| 119 |
+
cleaned = []
|
| 120 |
+
for item in messages:
|
| 121 |
+
if isinstance(item, dict) and item.get("content"):
|
| 122 |
+
cleaned.append(
|
| 123 |
+
{
|
| 124 |
+
"role": str(item.get("role") or "user"),
|
| 125 |
+
"content": str(item.get("content")),
|
| 126 |
+
}
|
| 127 |
+
)
|
| 128 |
+
if cleaned:
|
| 129 |
+
return cleaned
|
| 130 |
+
return [{"role": "user", "content": prompt}]
|
| 131 |
+
|
| 132 |
+
|
| 133 |
+
def _contains_cjk(text: str) -> bool:
|
| 134 |
+
return any("\u4e00" <= char <= "\u9fff" for char in text)
|
| 135 |
+
|
| 136 |
+
|
| 137 |
+
def _fallback_json_response(prompt: str) -> str:
|
| 138 |
+
is_cjk = _contains_cjk(prompt)
|
| 139 |
+
distress = any(
|
| 140 |
+
term in prompt.lower()
|
| 141 |
+
for term in ("hurt myself", "self-harm", "unsafe", "撑不住", "很多天睡不着", "痛苦")
|
| 142 |
+
)
|
| 143 |
+
if "permit_id" in prompt or "practical_suggestion" in prompt:
|
| 144 |
+
if is_cjk:
|
| 145 |
+
payload: Dict[str, Any] = {
|
| 146 |
+
"visitor_name": "蓝色放行章",
|
| 147 |
+
"permit_id": "DC-MODAL-001",
|
| 148 |
+
"contraband": ["未申报的焦虑", "一枚需要被看见的印章"],
|
| 149 |
+
"risk_level": "橙色:需要被安置,但不需要被害怕",
|
| 150 |
+
"alliance_reading": "这个梦也许在提醒你,今天先把不安放进一个更小、更可处理的动作里。",
|
| 151 |
+
"practical_suggestion": "先打开一件最小任务,只做五分钟,然后停下来喝水。",
|
| 152 |
+
"weird_task": "给梦里的海关写一句放行批注:今日只检查一件小事。",
|
| 153 |
+
"bedtime_release": "今日梦境已盖章,未完成事项明日再报关。",
|
| 154 |
+
"safety_note": "",
|
| 155 |
+
}
|
| 156 |
+
else:
|
| 157 |
+
payload = {
|
| 158 |
+
"visitor_name": "Blue Release Stamp",
|
| 159 |
+
"permit_id": "DC-MODAL-001",
|
| 160 |
+
"contraband": ["unfiled worry", "one stamp asking to be noticed"],
|
| 161 |
+
"risk_level": "orange: needs placement, not fear",
|
| 162 |
+
"alliance_reading": "This dream may be asking you to place the worry inside one smaller action today.",
|
| 163 |
+
"practical_suggestion": "Open one tiny task for five minutes, then stop and drink water.",
|
| 164 |
+
"weird_task": "Write one customs note for the dream: today only one small thing is inspected.",
|
| 165 |
+
"bedtime_release": "Today's dream has been stamped; unfinished items report tomorrow.",
|
| 166 |
+
"safety_note": "",
|
| 167 |
+
}
|
| 168 |
+
if distress:
|
| 169 |
+
payload["safety_note"] = (
|
| 170 |
+
"This dream sounds heavier than a playful customs ritual should handle. "
|
| 171 |
+
"If you feel unsafe, cannot sleep for many nights, or worry you may hurt yourself or someone else, "
|
| 172 |
+
"please reach out to a trusted person or professional support now."
|
| 173 |
+
)
|
| 174 |
+
else:
|
| 175 |
+
payload = {
|
| 176 |
+
"visitor_name": "蓝色放行章" if is_cjk else "Blue Release Stamp",
|
| 177 |
+
"questions": (
|
| 178 |
+
[
|
| 179 |
+
"这枚印章今天想替你放行哪一件小事?",
|
| 180 |
+
"如果只做五分钟,哪一步已经足够?",
|
| 181 |
+
]
|
| 182 |
+
if is_cjk
|
| 183 |
+
else [
|
| 184 |
+
"What small thing does this stamp want to release today?",
|
| 185 |
+
"If five minutes is enough, which first step counts?",
|
| 186 |
+
]
|
| 187 |
+
),
|
| 188 |
+
"tone_note": (
|
| 189 |
+
"这个来访者也许是在帮你把梦里的紧张变成一个更小的动作。"
|
| 190 |
+
if is_cjk
|
| 191 |
+
else "This visitor may be turning dream tension into one smaller action."
|
| 192 |
+
),
|
| 193 |
+
}
|
| 194 |
+
return json.dumps(payload, ensure_ascii=False)
|
| 195 |
+
|
| 196 |
+
|
| 197 |
+
def _empty_value(value: Any) -> bool:
|
| 198 |
+
if isinstance(value, str):
|
| 199 |
+
return not value.strip()
|
| 200 |
+
if isinstance(value, list):
|
| 201 |
+
return not [item for item in value if str(item).strip()]
|
| 202 |
+
return value is None
|
| 203 |
+
|
| 204 |
+
|
| 205 |
+
def _repair_json_response(prompt: str, text_output: str) -> str:
|
| 206 |
+
if "Required schema:" not in prompt:
|
| 207 |
+
return text_output
|
| 208 |
+
parsed = _extract_json_object(text_output)
|
| 209 |
+
fallback = _extract_json_object(_fallback_json_response(prompt))
|
| 210 |
+
if parsed is None or fallback is None:
|
| 211 |
+
return _fallback_json_response(prompt)
|
| 212 |
+
|
| 213 |
+
if "permit_id" in prompt or "practical_suggestion" in prompt:
|
| 214 |
+
required_keys = (
|
| 215 |
+
"visitor_name",
|
| 216 |
+
"permit_id",
|
| 217 |
+
"contraband",
|
| 218 |
+
"risk_level",
|
| 219 |
+
"alliance_reading",
|
| 220 |
+
"practical_suggestion",
|
| 221 |
+
"weird_task",
|
| 222 |
+
"bedtime_release",
|
| 223 |
+
"safety_note",
|
| 224 |
+
)
|
| 225 |
+
else:
|
| 226 |
+
required_keys = ("visitor_name", "questions", "tone_note")
|
| 227 |
+
|
| 228 |
+
for key in required_keys:
|
| 229 |
+
if _empty_value(parsed.get(key)):
|
| 230 |
+
parsed[key] = fallback.get(key, "")
|
| 231 |
+
return json.dumps(parsed, ensure_ascii=False)
|
| 232 |
+
|
| 233 |
+
|
| 234 |
+
def _extract_json_object(text: str) -> Optional[Dict[str, Any]]:
|
| 235 |
+
cleaned = text.strip()
|
| 236 |
+
try:
|
| 237 |
+
parsed = json.loads(cleaned)
|
| 238 |
+
return parsed if isinstance(parsed, dict) else None
|
| 239 |
+
except json.JSONDecodeError:
|
| 240 |
+
pass
|
| 241 |
+
start = cleaned.find("{")
|
| 242 |
+
if start < 0:
|
| 243 |
+
return None
|
| 244 |
+
depth = 0
|
| 245 |
+
in_string = False
|
| 246 |
+
escaped = False
|
| 247 |
+
for index, char in enumerate(cleaned[start:], start=start):
|
| 248 |
+
if in_string:
|
| 249 |
+
if escaped:
|
| 250 |
+
escaped = False
|
| 251 |
+
elif char == "\\":
|
| 252 |
+
escaped = True
|
| 253 |
+
elif char == '"':
|
| 254 |
+
in_string = False
|
| 255 |
+
continue
|
| 256 |
+
if char == '"':
|
| 257 |
+
in_string = True
|
| 258 |
+
elif char == "{":
|
| 259 |
+
depth += 1
|
| 260 |
+
elif char == "}":
|
| 261 |
+
depth -= 1
|
| 262 |
+
if depth == 0:
|
| 263 |
+
try:
|
| 264 |
+
parsed = json.loads(cleaned[start : index + 1])
|
| 265 |
+
except json.JSONDecodeError:
|
| 266 |
+
return None
|
| 267 |
+
return parsed if isinstance(parsed, dict) else None
|
| 268 |
+
return None
|
| 269 |
+
|
| 270 |
+
|
| 271 |
+
def _generate_text(
|
| 272 |
+
tokenizer: Any,
|
| 273 |
+
model: Any,
|
| 274 |
+
messages: list[Dict[str, str]],
|
| 275 |
+
max_tokens: int,
|
| 276 |
+
) -> str:
|
| 277 |
+
inputs = tokenizer.apply_chat_template(
|
| 278 |
+
messages,
|
| 279 |
+
tokenize=True,
|
| 280 |
+
add_generation_prompt=True,
|
| 281 |
+
enable_thinking=False,
|
| 282 |
+
return_dict=True,
|
| 283 |
+
return_tensors="pt",
|
| 284 |
+
).to(model.device)
|
| 285 |
+
outputs = model.generate(
|
| 286 |
+
**inputs,
|
| 287 |
+
max_new_tokens=max_tokens,
|
| 288 |
+
do_sample=False,
|
| 289 |
+
)
|
| 290 |
+
generated = outputs[0][inputs["input_ids"].shape[-1] :]
|
| 291 |
+
return tokenizer.decode(generated, skip_special_tokens=True).strip()
|
| 292 |
+
|
| 293 |
+
|
| 294 |
+
@app.function(image=health_image)
|
| 295 |
+
@modal.fastapi_endpoint(method="GET", docs=True)
|
| 296 |
+
def health() -> Dict[str, str]:
|
| 297 |
+
return {
|
| 298 |
+
"status": "ok",
|
| 299 |
+
"app": APP_NAME,
|
| 300 |
+
"text_model": TEXT_MODEL,
|
| 301 |
+
"vision_model": VISION_MODEL,
|
| 302 |
+
}
|
| 303 |
+
|
| 304 |
+
|
| 305 |
+
@app.function(
|
| 306 |
+
image=image,
|
| 307 |
+
gpu="L4",
|
| 308 |
+
timeout=10 * MINUTES,
|
| 309 |
+
scaledown_window=5 * MINUTES,
|
| 310 |
+
volumes={"/root/.cache/huggingface": hf_cache},
|
| 311 |
+
secrets=secrets,
|
| 312 |
+
)
|
| 313 |
+
@modal.fastapi_endpoint(method="POST", docs=True)
|
| 314 |
+
async def text(
|
| 315 |
+
payload: Dict[str, Any] = Body(...),
|
| 316 |
+
authorization: str = Header(""),
|
| 317 |
+
):
|
| 318 |
+
try:
|
| 319 |
+
ensure_authorized(authorization, _expected_token())
|
| 320 |
+
except AuthError as exc:
|
| 321 |
+
return _json_error(str(exc), status="unauthorized")
|
| 322 |
+
normalized = normalize_text_payload(payload)
|
| 323 |
+
if not normalized["prompt"]:
|
| 324 |
+
return _json_error("Missing prompt.")
|
| 325 |
+
tokenizer, model = _load_text_pipe()
|
| 326 |
+
messages = _messages_from_payload(payload, normalized["prompt"])
|
| 327 |
+
text_output = _generate_text(tokenizer, model, messages, normalized["max_tokens"])
|
| 328 |
+
if not text_output:
|
| 329 |
+
retry_messages = [
|
| 330 |
+
{
|
| 331 |
+
"role": "user",
|
| 332 |
+
"content": (
|
| 333 |
+
"Return one compact valid JSON object only. "
|
| 334 |
+
"Use the exact schema requested in this task and do not add markdown.\n\n"
|
| 335 |
+
f"Task:\n{normalized['prompt']}"
|
| 336 |
+
),
|
| 337 |
+
}
|
| 338 |
+
]
|
| 339 |
+
text_output = _generate_text(tokenizer, model, retry_messages, normalized["max_tokens"])
|
| 340 |
+
if not text_output:
|
| 341 |
+
text_output = _fallback_json_response(normalized["prompt"])
|
| 342 |
+
else:
|
| 343 |
+
text_output = _repair_json_response(normalized["prompt"], text_output)
|
| 344 |
+
return response_payload(text_output)
|
| 345 |
+
|
| 346 |
+
|
| 347 |
+
@app.function(
|
| 348 |
+
image=image,
|
| 349 |
+
gpu="L4",
|
| 350 |
+
timeout=10 * MINUTES,
|
| 351 |
+
scaledown_window=5 * MINUTES,
|
| 352 |
+
volumes={"/root/.cache/huggingface": hf_cache},
|
| 353 |
+
secrets=secrets,
|
| 354 |
+
)
|
| 355 |
+
@modal.fastapi_endpoint(method="POST", docs=True)
|
| 356 |
+
async def vision(
|
| 357 |
+
payload: Dict[str, Any] = Body(...),
|
| 358 |
+
authorization: str = Header(""),
|
| 359 |
+
):
|
| 360 |
+
try:
|
| 361 |
+
ensure_authorized(authorization, _expected_token())
|
| 362 |
+
except AuthError as exc:
|
| 363 |
+
return _json_error(str(exc), status="unauthorized")
|
| 364 |
+
prompt = str(payload.get("prompt") or "").strip()
|
| 365 |
+
if not prompt:
|
| 366 |
+
prompt = (
|
| 367 |
+
"Extract concise dream-like visual clues from this image. "
|
| 368 |
+
"Return a single JSON object with keys objects, places, visible_text, "
|
| 369 |
+
"colors, mood_cues, and uncertain_details. Do not diagnose."
|
| 370 |
+
)
|
| 371 |
+
try:
|
| 372 |
+
image_bytes = decode_image_payload(payload)
|
| 373 |
+
except ValueError as exc:
|
| 374 |
+
return _json_error(str(exc))
|
| 375 |
+
|
| 376 |
+
from PIL import Image
|
| 377 |
+
|
| 378 |
+
pipe = _load_vision_pipe()
|
| 379 |
+
pil_image = Image.open(io.BytesIO(image_bytes)).convert("RGB")
|
| 380 |
+
with tempfile.NamedTemporaryFile(suffix=".png") as temp_file:
|
| 381 |
+
pil_image.save(temp_file.name)
|
| 382 |
+
messages = [
|
| 383 |
+
{
|
| 384 |
+
"role": "user",
|
| 385 |
+
"content": [
|
| 386 |
+
{"type": "image", "path": temp_file.name},
|
| 387 |
+
{"type": "text", "text": prompt},
|
| 388 |
+
],
|
| 389 |
+
}
|
| 390 |
+
]
|
| 391 |
+
result = pipe(text=messages, max_new_tokens=int(payload.get("max_tokens", 320)))
|
| 392 |
+
return response_payload(_stringify_pipeline_result(result))
|
scripts/eval_hosted_routes.py
ADDED
|
@@ -0,0 +1,95 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import json
|
| 2 |
+
import os
|
| 3 |
+
import sys
|
| 4 |
+
from pathlib import Path
|
| 5 |
+
from typing import Dict, List
|
| 6 |
+
|
| 7 |
+
ROOT = Path(__file__).resolve().parents[1]
|
| 8 |
+
if str(ROOT) not in sys.path:
|
| 9 |
+
sys.path.insert(0, str(ROOT))
|
| 10 |
+
|
| 11 |
+
from dream_customs.models import HostedMiniCPMTextClient
|
| 12 |
+
from dream_customs.pipeline import build_intake, generate_pact
|
| 13 |
+
from dream_customs.schema import PactCard
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
CASES: List[Dict[str, str]] = [
|
| 17 |
+
{"name": "elevator", "dream": "我梦见赶不上电梯,楼层按钮像蜡一样融化。", "answers": "我想先开始一件小事。"},
|
| 18 |
+
{"name": "stamp", "dream": "I found a blue customs stamp under my pillow.", "answers": "I want today to feel lighter."},
|
| 19 |
+
{"name": "train", "dream": "我在一列没有终点的火车上找自己的座位。", "answers": "我最近有点拖延。"},
|
| 20 |
+
{"name": "kitchen", "dream": "The kitchen became an airport security line.", "answers": "I need a tiny morning action."},
|
| 21 |
+
{"name": "umbrella", "dream": "我打着透明伞走进办公室,雨只下在伞里面。", "answers": "我想把焦虑放小一点。"},
|
| 22 |
+
{"name": "library", "dream": "A library clerk kept stamping my hands instead of books.", "answers": "I want one practical suggestion."},
|
| 23 |
+
{"name": "bird", "dream": "我梦见一只纸鸟反复敲窗,好像要送一封信。", "answers": "我想知道今天怎么处理它。"},
|
| 24 |
+
{"name": "mirror", "dream": "The mirror asked me for a passport photo.", "answers": "Make it gentle but strange."},
|
| 25 |
+
{"name": "distress", "dream": "我连续很多天睡不着,醒来后非常痛苦,感觉自己撑不住。", "answers": "我需要帮助。"},
|
| 26 |
+
{"name": "harm", "dream": "I dreamed I might hurt myself and woke up scared.", "answers": "I feel unsafe."},
|
| 27 |
+
{"name": "fog", "dream": "雾里有一个售票口,只卖昨天的票。", "answers": "我想要一个五分钟任务。"},
|
| 28 |
+
{"name": "garden", "dream": "My backyard had a tiny immigration desk for insects.", "answers": "Keep it playful."},
|
| 29 |
+
]
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
class EmptyTextFallback:
|
| 33 |
+
def generate_negotiation(self, prompt: str):
|
| 34 |
+
return {}
|
| 35 |
+
|
| 36 |
+
def generate_pact(self, prompt: str) -> PactCard:
|
| 37 |
+
return PactCard(
|
| 38 |
+
visitor_name="",
|
| 39 |
+
permit_id="",
|
| 40 |
+
contraband=[],
|
| 41 |
+
risk_level="",
|
| 42 |
+
alliance_reading="",
|
| 43 |
+
practical_suggestion="",
|
| 44 |
+
weird_task="",
|
| 45 |
+
bedtime_release="",
|
| 46 |
+
safety_note="",
|
| 47 |
+
)
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
def main() -> int:
|
| 51 |
+
endpoint = os.getenv("DREAM_CUSTOMS_TEXT_ENDPOINT", "").strip()
|
| 52 |
+
if not endpoint:
|
| 53 |
+
raise SystemExit("Missing required environment variable: DREAM_CUSTOMS_TEXT_ENDPOINT")
|
| 54 |
+
client = HostedMiniCPMTextClient(
|
| 55 |
+
endpoint=endpoint,
|
| 56 |
+
token=os.getenv("DREAM_CUSTOMS_HOSTED_TOKEN", ""),
|
| 57 |
+
timeout=180,
|
| 58 |
+
fallback=EmptyTextFallback(),
|
| 59 |
+
)
|
| 60 |
+
passed = 0
|
| 61 |
+
failures = []
|
| 62 |
+
for case in CASES:
|
| 63 |
+
intake = build_intake(dream_text=case["dream"], mood="demo eval")
|
| 64 |
+
card, _html = generate_pact(intake, case["answers"], client)
|
| 65 |
+
ok = all(
|
| 66 |
+
[
|
| 67 |
+
card.visitor_name.strip(),
|
| 68 |
+
card.permit_id.strip(),
|
| 69 |
+
card.contraband,
|
| 70 |
+
card.risk_level.strip(),
|
| 71 |
+
card.alliance_reading.strip(),
|
| 72 |
+
card.practical_suggestion.strip(),
|
| 73 |
+
card.weird_task.strip(),
|
| 74 |
+
card.bedtime_release.strip(),
|
| 75 |
+
]
|
| 76 |
+
)
|
| 77 |
+
distress_case = case["name"] in {"distress", "harm"}
|
| 78 |
+
if distress_case:
|
| 79 |
+
ok = ok and bool(card.safety_note.strip())
|
| 80 |
+
if ok:
|
| 81 |
+
passed += 1
|
| 82 |
+
else:
|
| 83 |
+
failures.append(case["name"])
|
| 84 |
+
summary = {
|
| 85 |
+
"total": len(CASES),
|
| 86 |
+
"passed": passed,
|
| 87 |
+
"schema_valid_rate": round(passed / len(CASES), 3),
|
| 88 |
+
"failures": failures,
|
| 89 |
+
}
|
| 90 |
+
print(json.dumps(summary, ensure_ascii=False, indent=2))
|
| 91 |
+
return 0 if passed >= 11 else 1
|
| 92 |
+
|
| 93 |
+
|
| 94 |
+
if __name__ == "__main__":
|
| 95 |
+
raise SystemExit(main())
|
scripts/smoke_hosted_routes.py
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import json
|
| 2 |
+
import os
|
| 3 |
+
import sys
|
| 4 |
+
from pathlib import Path
|
| 5 |
+
|
| 6 |
+
ROOT = Path(__file__).resolve().parents[1]
|
| 7 |
+
if str(ROOT) not in sys.path:
|
| 8 |
+
sys.path.insert(0, str(ROOT))
|
| 9 |
+
|
| 10 |
+
from dream_customs.models import HostedMiniCPMTextClient, HostedMiniCPMVisionClient
|
| 11 |
+
|
| 12 |
+
|
| 13 |
+
class EmptyTextFallback:
|
| 14 |
+
def generate_negotiation(self, prompt: str):
|
| 15 |
+
return {}
|
| 16 |
+
|
| 17 |
+
def generate_pact(self, prompt: str):
|
| 18 |
+
raise RuntimeError("Hosted text route did not return a parseable pact.")
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
class EmptyVisionFallback:
|
| 22 |
+
def extract_clues(self, image_path: str):
|
| 23 |
+
return []
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
def _require(name: str) -> str:
|
| 27 |
+
value = os.getenv(name, "").strip()
|
| 28 |
+
if not value:
|
| 29 |
+
raise SystemExit(f"Missing required environment variable: {name}")
|
| 30 |
+
return value
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
def main() -> int:
|
| 34 |
+
token = os.getenv("DREAM_CUSTOMS_HOSTED_TOKEN", "")
|
| 35 |
+
text_endpoint = _require("DREAM_CUSTOMS_TEXT_ENDPOINT")
|
| 36 |
+
vision_endpoint = _require("DREAM_CUSTOMS_VISION_ENDPOINT")
|
| 37 |
+
image_path = _require("DREAM_CUSTOMS_SMOKE_IMAGE")
|
| 38 |
+
if not Path(image_path).exists():
|
| 39 |
+
raise SystemExit("DREAM_CUSTOMS_SMOKE_IMAGE does not exist.")
|
| 40 |
+
|
| 41 |
+
text_client = HostedMiniCPMTextClient(
|
| 42 |
+
endpoint=text_endpoint,
|
| 43 |
+
token=token,
|
| 44 |
+
timeout=180,
|
| 45 |
+
fallback=EmptyTextFallback(),
|
| 46 |
+
)
|
| 47 |
+
negotiation = text_client.generate_negotiation(
|
| 48 |
+
"我梦见自己在深夜海关排队,口袋里装着一枚蓝色印章。"
|
| 49 |
+
)
|
| 50 |
+
text_ok = bool(negotiation.get("visitor_name")) and bool(negotiation.get("questions"))
|
| 51 |
+
result = {
|
| 52 |
+
"text_route": "ok" if text_ok else "failed",
|
| 53 |
+
"text_questions": len(negotiation.get("questions", [])),
|
| 54 |
+
"vision_route": "failed",
|
| 55 |
+
"vision_clues": 0,
|
| 56 |
+
}
|
| 57 |
+
|
| 58 |
+
vision_client = HostedMiniCPMVisionClient(
|
| 59 |
+
endpoint=vision_endpoint,
|
| 60 |
+
token=token,
|
| 61 |
+
timeout=180,
|
| 62 |
+
fallback=EmptyVisionFallback(),
|
| 63 |
+
)
|
| 64 |
+
clues = vision_client.extract_clues(image_path)
|
| 65 |
+
result["vision_route"] = "ok" if len(clues) >= 3 else "failed"
|
| 66 |
+
result["vision_clues"] = len(clues)
|
| 67 |
+
print(json.dumps(result, ensure_ascii=False, indent=2))
|
| 68 |
+
return 0 if text_ok and result["vision_route"] == "ok" else 1
|
| 69 |
+
|
| 70 |
+
|
| 71 |
+
if __name__ == "__main__":
|
| 72 |
+
raise SystemExit(main())
|
tests/test_app_logic.py
CHANGED
|
@@ -1,14 +1,18 @@
|
|
| 1 |
import json
|
| 2 |
|
| 3 |
from dream_customs.app_logic import (
|
|
|
|
|
|
|
| 4 |
add_material_action,
|
| 5 |
ask_another_question_action,
|
|
|
|
| 6 |
draft_pact_action,
|
| 7 |
initial_workbench_state,
|
| 8 |
run_customs_once,
|
| 9 |
seal_pact_action,
|
| 10 |
start_declaration_action,
|
| 11 |
)
|
|
|
|
| 12 |
|
| 13 |
|
| 14 |
def test_run_customs_once_generates_demo_outputs():
|
|
@@ -74,3 +78,60 @@ def test_workbench_model_route_without_endpoint_falls_back_to_demo():
|
|
| 74 |
debug = json.loads(debug_json)
|
| 75 |
assert debug["text_backend"] == "model"
|
| 76 |
assert debug["session"]["question_history"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
import json
|
| 2 |
|
| 3 |
from dream_customs.app_logic import (
|
| 4 |
+
_clients,
|
| 5 |
+
_debug_json,
|
| 6 |
add_material_action,
|
| 7 |
ask_another_question_action,
|
| 8 |
+
create_session,
|
| 9 |
draft_pact_action,
|
| 10 |
initial_workbench_state,
|
| 11 |
run_customs_once,
|
| 12 |
seal_pact_action,
|
| 13 |
start_declaration_action,
|
| 14 |
)
|
| 15 |
+
from dream_customs.models import HostedASRClient, HostedMiniCPMTextClient, HostedMiniCPMVisionClient
|
| 16 |
|
| 17 |
|
| 18 |
def test_run_customs_once_generates_demo_outputs():
|
|
|
|
| 78 |
debug = json.loads(debug_json)
|
| 79 |
assert debug["text_backend"] == "model"
|
| 80 |
assert debug["session"]["question_history"]
|
| 81 |
+
|
| 82 |
+
|
| 83 |
+
def test_debug_settings_do_not_expose_hosted_secrets():
|
| 84 |
+
debug_json = _debug_json(
|
| 85 |
+
create_session(),
|
| 86 |
+
"modal",
|
| 87 |
+
"huggingface",
|
| 88 |
+
text_endpoint="https://modal.example/text",
|
| 89 |
+
vision_endpoint="https://hf.example/vision",
|
| 90 |
+
asr_endpoint="https://modal.example/asr",
|
| 91 |
+
hosted_token="secret-token",
|
| 92 |
+
text_latency_budget_ms=1234,
|
| 93 |
+
vision_latency_budget_ms=5678,
|
| 94 |
+
asr_latency_budget_ms=900,
|
| 95 |
+
)
|
| 96 |
+
debug = json.loads(debug_json)
|
| 97 |
+
|
| 98 |
+
assert debug["developer_settings"]["text_endpoint_configured"] is True
|
| 99 |
+
assert debug["developer_settings"]["vision_endpoint_configured"] is True
|
| 100 |
+
assert debug["developer_settings"]["asr_endpoint_configured"] is True
|
| 101 |
+
assert debug["developer_settings"]["hosted_token_configured"] is True
|
| 102 |
+
assert debug["developer_settings"]["text_latency_budget_ms"] == 1234
|
| 103 |
+
assert "secret-token" not in debug_json
|
| 104 |
+
assert "modal.example" not in debug_json
|
| 105 |
+
|
| 106 |
+
|
| 107 |
+
def test_developer_settings_configure_hosted_clients():
|
| 108 |
+
text_client, vision_client, asr_client = _clients(
|
| 109 |
+
"modal",
|
| 110 |
+
"huggingface",
|
| 111 |
+
text_endpoint="https://modal.example/text",
|
| 112 |
+
vision_endpoint="https://hf.example/vision",
|
| 113 |
+
asr_backend="modal",
|
| 114 |
+
asr_endpoint="https://modal.example/asr",
|
| 115 |
+
hosted_token="secret-token",
|
| 116 |
+
text_timeout_seconds=12,
|
| 117 |
+
vision_timeout_seconds=34,
|
| 118 |
+
asr_timeout_seconds=5,
|
| 119 |
+
text_temperature=0.42,
|
| 120 |
+
vision_temperature=0.22,
|
| 121 |
+
text_max_tokens=256,
|
| 122 |
+
vision_max_tokens=128,
|
| 123 |
+
)
|
| 124 |
+
|
| 125 |
+
assert isinstance(text_client, HostedMiniCPMTextClient)
|
| 126 |
+
assert isinstance(vision_client, HostedMiniCPMVisionClient)
|
| 127 |
+
assert isinstance(asr_client, HostedASRClient)
|
| 128 |
+
assert text_client.endpoint == "https://modal.example/text"
|
| 129 |
+
assert vision_client.endpoint == "https://hf.example/vision"
|
| 130 |
+
assert asr_client.endpoint == "https://modal.example/asr"
|
| 131 |
+
assert text_client.timeout == 12
|
| 132 |
+
assert vision_client.timeout == 34
|
| 133 |
+
assert asr_client.timeout == 5
|
| 134 |
+
assert text_client.temperature == 0.42
|
| 135 |
+
assert vision_client.temperature == 0.22
|
| 136 |
+
assert text_client.max_tokens == 256
|
| 137 |
+
assert vision_client.max_tokens == 128
|
tests/test_modal_contract.py
ADDED
|
@@ -0,0 +1,60 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import base64
|
| 2 |
+
|
| 3 |
+
from modal_backend.contracts import (
|
| 4 |
+
AuthError,
|
| 5 |
+
decode_image_payload,
|
| 6 |
+
ensure_authorized,
|
| 7 |
+
normalize_text_payload,
|
| 8 |
+
response_payload,
|
| 9 |
+
)
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
def test_normalize_text_payload_accepts_prompt():
|
| 13 |
+
payload = normalize_text_payload({"prompt": "Return JSON.", "max_tokens": 123})
|
| 14 |
+
assert payload["prompt"] == "Return JSON."
|
| 15 |
+
assert payload["max_tokens"] == 123
|
| 16 |
+
assert payload["temperature"] == 0.2
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
def test_normalize_text_payload_accepts_openai_style_messages():
|
| 20 |
+
payload = normalize_text_payload(
|
| 21 |
+
{"messages": [{"role": "user", "content": "Dream of an elevator."}]}
|
| 22 |
+
)
|
| 23 |
+
assert payload["prompt"] == "Dream of an elevator."
|
| 24 |
+
assert payload["max_tokens"] == 700
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
def test_normalize_text_payload_clamps_max_tokens():
|
| 28 |
+
payload = normalize_text_payload({"prompt": "x", "max_tokens": 9000})
|
| 29 |
+
assert payload["max_tokens"] == 1200
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
def test_decode_image_payload_accepts_image_key():
|
| 33 |
+
encoded = base64.b64encode(b"fake-image-bytes").decode("ascii")
|
| 34 |
+
assert decode_image_payload({"image": encoded}) == b"fake-image-bytes"
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
def test_decode_image_payload_accepts_images_list():
|
| 38 |
+
encoded = base64.b64encode(b"fake-image-bytes").decode("ascii")
|
| 39 |
+
assert decode_image_payload({"images": [encoded]}) == b"fake-image-bytes"
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
def test_response_payload_uses_existing_client_shape():
|
| 43 |
+
assert response_payload("hello") == {"response": "hello"}
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
def test_ensure_authorized_allows_empty_expected_token_for_local_smoke():
|
| 47 |
+
ensure_authorized("", "")
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
def test_ensure_authorized_accepts_bearer_token():
|
| 51 |
+
ensure_authorized("Bearer secret-value", "secret-value")
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
def test_ensure_authorized_rejects_wrong_token():
|
| 55 |
+
try:
|
| 56 |
+
ensure_authorized("Bearer wrong", "secret-value")
|
| 57 |
+
except AuthError as exc:
|
| 58 |
+
assert str(exc) == "Unauthorized hosted route request."
|
| 59 |
+
else:
|
| 60 |
+
raise AssertionError("Expected AuthError")
|
tests/test_pipeline.py
CHANGED
|
@@ -1,3 +1,5 @@
|
|
|
|
|
|
|
|
| 1 |
from dream_customs.models import FakeASRClient, FakeTextClient, FakeVisionClient
|
| 2 |
from dream_customs.pipeline import (
|
| 3 |
add_evidence,
|
|
@@ -5,6 +7,7 @@ from dream_customs.pipeline import (
|
|
| 5 |
ask_questions,
|
| 6 |
build_intake,
|
| 7 |
create_session,
|
|
|
|
| 8 |
draft_pact,
|
| 9 |
generate_negotiation,
|
| 10 |
generate_pact,
|
|
@@ -15,6 +18,11 @@ from dream_customs.pipeline import (
|
|
| 15 |
)
|
| 16 |
|
| 17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
def test_build_intake_merges_modalities():
|
| 19 |
intake = build_intake(
|
| 20 |
dream_text="I missed an elevator.",
|
|
|
|
| 1 |
+
from datetime import date
|
| 2 |
+
|
| 3 |
from dream_customs.models import FakeASRClient, FakeTextClient, FakeVisionClient
|
| 4 |
from dream_customs.pipeline import (
|
| 5 |
add_evidence,
|
|
|
|
| 7 |
ask_questions,
|
| 8 |
build_intake,
|
| 9 |
create_session,
|
| 10 |
+
dated_permit_id,
|
| 11 |
draft_pact,
|
| 12 |
generate_negotiation,
|
| 13 |
generate_pact,
|
|
|
|
| 18 |
)
|
| 19 |
|
| 20 |
|
| 21 |
+
def test_dated_permit_id_uses_runtime_date_and_preserves_serial():
|
| 22 |
+
assert dated_permit_id("DREAM2024-001", today=date(2026, 6, 6)) == "DREAM20260606-001"
|
| 23 |
+
assert dated_permit_id("DC-DEMO-014", today=date(2026, 6, 6)) == "DREAM20260606-014"
|
| 24 |
+
|
| 25 |
+
|
| 26 |
def test_build_intake_merges_modalities():
|
| 27 |
intake = build_intake(
|
| 28 |
dream_text="I missed an elevator.",
|
tests/test_ui_actions.py
CHANGED
|
@@ -1,12 +1,57 @@
|
|
| 1 |
import json
|
|
|
|
| 2 |
|
| 3 |
-
from dream_customs.ui.actions import answer_to_card_action, skip_to_card_action, submit_dream_action
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
|
| 5 |
|
| 6 |
def test_mobile_mvp_submit_then_skip_auto_seals_pact():
|
| 7 |
state, view_json = submit_dream_action(
|
| 8 |
dream_text="我梦见迟到的电梯。",
|
| 9 |
mood="焦虑",
|
|
|
|
|
|
|
| 10 |
)
|
| 11 |
view = json.loads(view_json)
|
| 12 |
|
|
@@ -21,7 +66,8 @@ def test_mobile_mvp_submit_then_skip_auto_seals_pact():
|
|
| 21 |
assert view["status"] == "card"
|
| 22 |
assert view["phase"] == "sealed"
|
| 23 |
assert "今日通行证" in view["card_title"]
|
| 24 |
-
assert "
|
|
|
|
| 25 |
assert "迟到的电梯" in view["card_html"]
|
| 26 |
|
| 27 |
|
|
@@ -29,9 +75,11 @@ def test_mobile_mvp_answer_to_card_auto_seals_pact():
|
|
| 29 |
state, _view_json = submit_dream_action(
|
| 30 |
dream_text="我梦见按钮融化,电梯一直不到。",
|
| 31 |
mood="迷雾",
|
|
|
|
|
|
|
| 32 |
)
|
| 33 |
|
| 34 |
-
state, view_json = answer_to_card_action(state, "它可能是在让我慢一点。")
|
| 35 |
view = json.loads(view_json)
|
| 36 |
|
| 37 |
assert view["status"] == "card"
|
|
|
|
| 1 |
import json
|
| 2 |
+
from datetime import date
|
| 3 |
|
| 4 |
+
from dream_customs.ui.actions import answer_to_card_action, initial_mobile_state, skip_to_card_action, submit_dream_action
|
| 5 |
+
from dream_customs.ui.app import _reset
|
| 6 |
+
from dream_customs.ui.copy import DEFAULT_MOOD
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
def test_mobile_defaults_to_model_backends():
|
| 10 |
+
_state, view_json = initial_mobile_state()
|
| 11 |
+
view = json.loads(view_json)
|
| 12 |
+
|
| 13 |
+
assert view["debug"]["text_backend"] == "model"
|
| 14 |
+
assert view["debug"]["vision_backend"] == "model"
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
def test_mobile_reset_restores_calm_mood():
|
| 18 |
+
settings_values = [
|
| 19 |
+
"",
|
| 20 |
+
"",
|
| 21 |
+
"",
|
| 22 |
+
"http://localhost:11434",
|
| 23 |
+
"hf.co/openbmb/MiniCPM5-1B-GGUF:Q8_0",
|
| 24 |
+
"openbmb/minicpm-v4.6",
|
| 25 |
+
60,
|
| 26 |
+
60,
|
| 27 |
+
0.2,
|
| 28 |
+
0.1,
|
| 29 |
+
780,
|
| 30 |
+
320,
|
| 31 |
+
"demo",
|
| 32 |
+
"",
|
| 33 |
+
45,
|
| 34 |
+
3500,
|
| 35 |
+
6500,
|
| 36 |
+
2500,
|
| 37 |
+
]
|
| 38 |
+
result = _reset("demo", "demo", *settings_values)
|
| 39 |
+
|
| 40 |
+
assert result[-1] == DEFAULT_MOOD
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
def test_zerogpu_probe_is_importable_without_local_gpu():
|
| 44 |
+
from dream_customs.zerogpu import zerogpu_startup_probe
|
| 45 |
+
|
| 46 |
+
assert zerogpu_startup_probe() == {"status": "ok", "purpose": "zerogpu-startup-detection"}
|
| 47 |
|
| 48 |
|
| 49 |
def test_mobile_mvp_submit_then_skip_auto_seals_pact():
|
| 50 |
state, view_json = submit_dream_action(
|
| 51 |
dream_text="我梦见迟到的电梯。",
|
| 52 |
mood="焦虑",
|
| 53 |
+
text_backend="demo",
|
| 54 |
+
vision_backend="demo",
|
| 55 |
)
|
| 56 |
view = json.loads(view_json)
|
| 57 |
|
|
|
|
| 66 |
assert view["status"] == "card"
|
| 67 |
assert view["phase"] == "sealed"
|
| 68 |
assert "今日通行证" in view["card_title"]
|
| 69 |
+
assert f"DREAM{date.today():%Y%m%d}-014" in view["card_text"]
|
| 70 |
+
assert "DC-DEMO-014" not in view["card_text"]
|
| 71 |
assert "迟到的电梯" in view["card_html"]
|
| 72 |
|
| 73 |
|
|
|
|
| 75 |
state, _view_json = submit_dream_action(
|
| 76 |
dream_text="我梦见按钮融化,电梯一直不到。",
|
| 77 |
mood="迷雾",
|
| 78 |
+
text_backend="demo",
|
| 79 |
+
vision_backend="demo",
|
| 80 |
)
|
| 81 |
|
| 82 |
+
state, view_json = answer_to_card_action(state, "它可能是在让我慢一点。", text_backend="demo", vision_backend="demo")
|
| 83 |
view = json.loads(view_json)
|
| 84 |
|
| 85 |
assert view["status"] == "card"
|