Spaces:
Running on Zero
Running on Zero
feat: add initial mock mvp
Browse filesThis view is limited to 50 files because it contains too many changes. See raw diff
- .env.example +2 -0
- .gitignore +5 -1
- AGENTS.md +5 -2
- README.md +64 -2
- app.py +10 -0
- data/README.md +7 -0
- data/traces/README.md +22 -0
- data/traces/samples/objectverse_public_mock_traces.jsonl +6 -0
- data/traces/samples/sample-01-coffee-mug.json +50 -0
- data/traces/samples/sample-02-mechanical-keyboard.json +50 -0
- data/traces/samples/sample-03-running-shoe.json +48 -0
- data/traces/samples/sample-04-desk-lamp.json +49 -0
- data/traces/samples/sample-05-water-bottle.json +49 -0
- data/traces/samples/sample-06-notebook.json +49 -0
- data/train/README.md +16 -0
- data/train/objectverse_sft_preview.jsonl +0 -0
- docs/03-dev-schedule.md +26 -19
- docs/07-development-plan.md +315 -0
- docs/DATASET.md +87 -0
- docs/EXTERNAL_SETUP.md +77 -0
- docs/FAILURES.md +84 -0
- docs/FIELD_NOTES.md +58 -2
- docs/INITIAL_STAGE_REPORT.md +100 -0
- docs/MODEL_CARD.md +61 -1
- docs/README.md +6 -0
- docs/RUNTIME.md +44 -0
- docs/SUBMISSION_GUIDE.md +17 -9
- pyproject.toml +6 -3
- requirements.txt +2 -2
- scripts/README.md +9 -4
- scripts/__init__.py +1 -0
- scripts/check_initial_stage.py +194 -0
- scripts/export_traces.py +61 -0
- scripts/generate_dataset.py +135 -0
- scripts/generate_sample_traces.py +55 -0
- src/README.md +2 -1
- src/__init__.py +1 -0
- src/config.py +66 -0
- src/examples.py +38 -0
- src/models/README.md +11 -0
- src/models/llama_cpp_runner.py +110 -0
- src/models/schema.py +59 -0
- src/models/vision_runner.py +80 -0
- src/pipeline.py +50 -0
- src/prompts/README.md +2 -2
- src/prompts/diary_generation.py +6 -0
- src/prompts/object_understanding.py +6 -0
- src/prompts/persona_generation.py +7 -0
- src/renderer/README.md +2 -2
- src/renderer/html_templates.py +3 -0
.env.example
CHANGED
|
@@ -7,3 +7,5 @@ HF_DATASET_REPO=
|
|
| 7 |
TEXT_MODEL_PATH=
|
| 8 |
VISION_MODEL_ID=
|
| 9 |
TRACE_OUTPUT_DIR=data/traces
|
|
|
|
|
|
|
|
|
| 7 |
TEXT_MODEL_PATH=
|
| 8 |
VISION_MODEL_ID=
|
| 9 |
TRACE_OUTPUT_DIR=data/traces
|
| 10 |
+
OBJECTVERSE_VISION_BACKEND=mock
|
| 11 |
+
OBJECTVERSE_TEXT_BACKEND=mock
|
.gitignore
CHANGED
|
@@ -8,6 +8,7 @@ __pycache__/
|
|
| 8 |
# Virtual environments
|
| 9 |
.venv/
|
| 10 |
venv/
|
|
|
|
| 11 |
|
| 12 |
# Environment and secrets
|
| 13 |
.env
|
|
@@ -15,7 +16,7 @@ venv/
|
|
| 15 |
!.env.example
|
| 16 |
|
| 17 |
# Model and generated artifacts
|
| 18 |
-
models/
|
| 19 |
*.gguf
|
| 20 |
*.safetensors
|
| 21 |
*.ckpt
|
|
@@ -27,5 +28,8 @@ data/traces/*.json
|
|
| 27 |
data/traces/*.jsonl
|
| 28 |
exports/
|
| 29 |
|
|
|
|
|
|
|
|
|
|
| 30 |
# System files
|
| 31 |
.DS_Store
|
|
|
|
| 8 |
# Virtual environments
|
| 9 |
.venv/
|
| 10 |
venv/
|
| 11 |
+
.tmp/
|
| 12 |
|
| 13 |
# Environment and secrets
|
| 14 |
.env
|
|
|
|
| 16 |
!.env.example
|
| 17 |
|
| 18 |
# Model and generated artifacts
|
| 19 |
+
/models/
|
| 20 |
*.gguf
|
| 21 |
*.safetensors
|
| 22 |
*.ckpt
|
|
|
|
| 28 |
data/traces/*.jsonl
|
| 29 |
exports/
|
| 30 |
|
| 31 |
+
# Local design references
|
| 32 |
+
UI 参考/
|
| 33 |
+
|
| 34 |
# System files
|
| 35 |
.DS_Store
|
AGENTS.md
CHANGED
|
@@ -41,6 +41,7 @@ It is an English-first, Chinese-second Gradio application where users upload eve
|
|
| 41 |
The interface must be English-first and Chinese-second.
|
| 42 |
Visual style: strange object archive, not default Gradio demo.
|
| 43 |
Recommended UI mood: mysterious archive, typewriter diary, warm dark paper, amber highlight, museum label, strange but polished.
|
|
|
|
| 44 |
|
| 45 |
## Architecture
|
| 46 |
|
|
@@ -75,6 +76,8 @@ Recommended UI mood: mysterious archive, typewriter diary, warm dark paper, ambe
|
|
| 75 |
- README links valid.
|
| 76 |
- Demo video flow reproducible.
|
| 77 |
|
| 78 |
-
## Current
|
| 79 |
|
| 80 |
-
The current phase is
|
|
|
|
|
|
|
|
|
| 41 |
The interface must be English-first and Chinese-second.
|
| 42 |
Visual style: strange object archive, not default Gradio demo.
|
| 43 |
Recommended UI mood: mysterious archive, typewriter diary, warm dark paper, amber highlight, museum label, strange but polished.
|
| 44 |
+
When developing or polishing the frontend UI, reference the design images under `UI 参考/`, but keep product scope, copy hierarchy, interaction flow, and technical constraints aligned with `docs/`.
|
| 45 |
|
| 46 |
## Architecture
|
| 47 |
|
|
|
|
| 76 |
- README links valid.
|
| 77 |
- Demo video flow reproducible.
|
| 78 |
|
| 79 |
+
## Current Implementation Boundary
|
| 80 |
|
| 81 |
+
The current phase is an initial mock MVP. It may use deterministic mock outputs for Day 2 flow validation.
|
| 82 |
+
|
| 83 |
+
Real MiniCPM-V, llama.cpp, LoRA fine-tuning, dataset publishing, and Hugging Face Space deployment are not connected yet.
|
README.md
CHANGED
|
@@ -9,7 +9,9 @@ Upload a photo of any everyday object. The app wakes it up, gives it a secret pe
|
|
| 9 |
|
| 10 |
## Current Status
|
| 11 |
|
| 12 |
-
|
|
|
|
|
|
|
| 13 |
|
| 14 |
## Track
|
| 15 |
|
|
@@ -48,12 +50,72 @@ The interface is English-first and Chinese-second.
|
|
| 48 |
|
| 49 |
## Run Locally
|
| 50 |
|
| 51 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 52 |
|
| 53 |
## Project Structure
|
| 54 |
|
| 55 |
See `docs/02-tech-architecture.md`, `AGENTS.md`, and `.codex/skills/` for the intended structure and development rules.
|
| 56 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
## HF Space README YAML Header
|
| 58 |
|
| 59 |
```yaml
|
|
|
|
| 9 |
|
| 10 |
## Current Status
|
| 11 |
|
| 12 |
+
Initial mock MVP is available.
|
| 13 |
+
|
| 14 |
+
The app currently uses deterministic mock outputs for object understanding, persona generation, diary writing, chat replies, share card rendering, and trace saving. Real MiniCPM-V and llama.cpp model runtimes are not connected yet.
|
| 15 |
|
| 16 |
## Track
|
| 17 |
|
|
|
|
| 50 |
|
| 51 |
## Run Locally
|
| 52 |
|
| 53 |
+
```bash
|
| 54 |
+
pip install -r requirements.txt
|
| 55 |
+
python app.py
|
| 56 |
+
```
|
| 57 |
+
|
| 58 |
+
Then open the local Gradio URL printed in the terminal.
|
| 59 |
+
|
| 60 |
+
## Initial MVP Flow
|
| 61 |
+
|
| 62 |
+
The current implementation supports:
|
| 63 |
+
|
| 64 |
+
- image upload
|
| 65 |
+
- optional object description
|
| 66 |
+
- personality mode selection
|
| 67 |
+
- six stable example objects
|
| 68 |
+
- mock object understanding JSON
|
| 69 |
+
- mock persona JSON
|
| 70 |
+
- English-first secret diary with Chinese helper translation
|
| 71 |
+
- object chat with persona consistency
|
| 72 |
+
- share card HTML preview
|
| 73 |
+
- anonymized trace JSON saved under `data/traces/`
|
| 74 |
+
- six public mock sample traces under `data/traces/samples/`
|
| 75 |
+
|
| 76 |
+
## Generate Sample Traces
|
| 77 |
+
|
| 78 |
+
```bash
|
| 79 |
+
.venv/bin/python -B scripts/generate_sample_traces.py
|
| 80 |
+
```
|
| 81 |
+
|
| 82 |
+
## Generate Dataset Preview
|
| 83 |
+
|
| 84 |
+
```bash
|
| 85 |
+
.venv/bin/python -B scripts/generate_dataset.py
|
| 86 |
+
```
|
| 87 |
+
|
| 88 |
+
This creates deterministic mock SFT preview data for schema and curation planning. See `docs/DATASET.md`.
|
| 89 |
+
|
| 90 |
+
## Export Public Trace JSONL
|
| 91 |
+
|
| 92 |
+
```bash
|
| 93 |
+
.venv/bin/python -B scripts/export_traces.py
|
| 94 |
+
```
|
| 95 |
+
|
| 96 |
+
## Test
|
| 97 |
+
|
| 98 |
+
```bash
|
| 99 |
+
.venv/bin/python -B -m unittest discover -s tests
|
| 100 |
+
```
|
| 101 |
+
|
| 102 |
+
## Initial Stage Acceptance
|
| 103 |
+
|
| 104 |
+
```bash
|
| 105 |
+
.venv/bin/python -B scripts/check_initial_stage.py
|
| 106 |
+
```
|
| 107 |
+
|
| 108 |
+
See `docs/INITIAL_STAGE_REPORT.md` for the local initial-stage evidence.
|
| 109 |
+
See `docs/EXTERNAL_SETUP.md` before creating remote GitHub or Hugging Face resources.
|
| 110 |
|
| 111 |
## Project Structure
|
| 112 |
|
| 113 |
See `docs/02-tech-architecture.md`, `AGENTS.md`, and `.codex/skills/` for the intended structure and development rules.
|
| 114 |
|
| 115 |
+
## Runtime Notes
|
| 116 |
+
|
| 117 |
+
The current runtime is mock-only. See `docs/RUNTIME.md` for configuration keys and the future MiniCPM-V / llama.cpp boundary.
|
| 118 |
+
|
| 119 |
## HF Space README YAML Header
|
| 120 |
|
| 121 |
```yaml
|
app.py
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Objectverse Diary app entrypoint."""
|
| 2 |
+
|
| 3 |
+
from src.ui.layout import build_app
|
| 4 |
+
|
| 5 |
+
|
| 6 |
+
app = build_app()
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
if __name__ == "__main__":
|
| 10 |
+
app.launch()
|
data/README.md
CHANGED
|
@@ -5,3 +5,10 @@ This directory is reserved for examples, training data, evaluation samples, and
|
|
| 5 |
## Privacy Rule
|
| 6 |
|
| 7 |
Do not store personal or sensitive data here. Public traces must be anonymized.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
## Privacy Rule
|
| 6 |
|
| 7 |
Do not store personal or sensitive data here. Public traces must be anonymized.
|
| 8 |
+
|
| 9 |
+
## Current Local Artifacts
|
| 10 |
+
|
| 11 |
+
- `traces/samples/`: six stable mock public traces.
|
| 12 |
+
- `train/objectverse_sft_preview.jsonl`: deterministic mock SFT preview generated by `scripts/generate_dataset.py`.
|
| 13 |
+
|
| 14 |
+
Generated preview data is for planning and validation only. Review curated samples before publishing any dataset.
|
data/traces/README.md
CHANGED
|
@@ -3,3 +3,25 @@
|
|
| 3 |
Reserved for anonymized public traces.
|
| 4 |
|
| 5 |
Target: at least 6 public traces for the Sharing is Caring badge.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
Reserved for anonymized public traces.
|
| 4 |
|
| 5 |
Target: at least 6 public traces for the Sharing is Caring badge.
|
| 6 |
+
|
| 7 |
+
## Current Samples
|
| 8 |
+
|
| 9 |
+
`samples/` contains six stable mock traces generated from the example objects:
|
| 10 |
+
|
| 11 |
+
```bash
|
| 12 |
+
.venv/bin/python -B scripts/generate_sample_traces.py
|
| 13 |
+
```
|
| 14 |
+
|
| 15 |
+
These traces use the current mock runtime and are safe placeholders until real VLM and llama.cpp traces are available.
|
| 16 |
+
|
| 17 |
+
Export the validated sample traces as JSONL:
|
| 18 |
+
|
| 19 |
+
```bash
|
| 20 |
+
.venv/bin/python -B scripts/export_traces.py
|
| 21 |
+
```
|
| 22 |
+
|
| 23 |
+
Default output:
|
| 24 |
+
|
| 25 |
+
```text
|
| 26 |
+
data/traces/samples/objectverse_public_mock_traces.jsonl
|
| 27 |
+
```
|
data/traces/samples/objectverse_public_mock_traces.jsonl
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"created_at": "2026-06-05T00:01:00Z", "diary": {"chinese": "今天他们又理所当然地碰了我,好像一个 coffee mug 不会有边界感。我保持沉默,因为这大概是我和重力签下的合同。我的情绪是 tired but sarcastic,秘密恐惧是 being replaced by a newer object with worse opinions。至少,我已经熬过了好几个所谓紧急计划。", "english": "They touched me again today with the confidence of someone who has never asked a coffee mug for consent. I remained still, because that is my contract with gravity. My mood is tired but sarcastic, my secret fear is being replaced by a newer object with worse opinions, and my only comfort is knowing I have outlived at least three urgent plans.", "title": "Secret Diary - Day 427"}, "fallbacks": ["mock-runtime"], "input": {"description": "old white coffee mug on a developer desk", "has_image": false, "image_filename": null}, "mode": "Cynical", "model_runtime": {"runtime": "no llama.cpp model connected yet", "text": "mock persona and diary generation", "vision": "mock object understanding"}, "object_understanding": {"object": {"confidence": 0.42, "likely_context": "developer desk", "name": "coffee mug", "visible_features": ["old", "white", "user-supplied description"]}}, "persona": {"persona": {"character_name": "CoffeeMug worth", "complaint": "I am not just a coffee mug. I am an unpaid witness with excellent recall.", "core_memory": "survived many quiet hours as a coffee mug while humans called it normal life", "mood": "tired but sarcastic", "object_name": "coffee mug", "secret_fear": "being replaced by a newer object with worse opinions", "tags": ["desk survivor", "burnt optimism", "quiet judgment"]}}, "trace_id": "sample-01"}
|
| 2 |
+
{"created_at": "2026-06-05T00:02:00Z", "diary": {"chinese": "今天他们又理所当然地碰了我,好像一个 keyboard 不会有边界感。我保持沉默,因为这大概是我和重力签下的合同。我的情绪是 curious and needlessly profound,秘密恐惧是 discovering that usefulness is not meaning。至少,我已经熬过了好几个所谓紧急计划。", "english": "They touched me again today with the confidence of someone who has never asked a keyboard for consent. I remained still, because that is my contract with gravity. My mood is curious and needlessly profound, my secret fear is discovering that usefulness is not meaning, and my only comfort is knowing I have outlived at least three urgent plans.", "title": "Secret Diary - Day 425"}, "fallbacks": ["mock-runtime"], "input": {"description": "dusty black mechanical keyboard in an office", "has_image": false, "image_filename": null}, "mode": "Philosopher", "model_runtime": {"runtime": "no llama.cpp model connected yet", "text": "mock persona and diary generation", "vision": "mock object understanding"}, "object_understanding": {"object": {"confidence": 0.42, "likely_context": "office corner", "name": "keyboard", "visible_features": ["black", "dusty", "user-supplied description"]}}, "persona": {"persona": {"character_name": "Keyboard the Questioning", "complaint": "I am not just a keyboard. I am an unpaid witness with excellent recall.", "core_memory": "survived many quiet hours as a keyboard while humans called it normal life", "mood": "curious and needlessly profound", "object_name": "keyboard", "secret_fear": "discovering that usefulness is not meaning", "tags": ["tiny ontology", "useful doubt", "meaning crisis"]}}, "trace_id": "sample-02"}
|
| 3 |
+
{"created_at": "2026-06-05T00:03:00Z", "diary": {"chinese": "今天他们又理所当然地碰了我,好像一个 shoe 不会有边界感。我保持沉默,因为这大概是我和重力签下的合同。我的情绪是 softly abandoned,秘密恐惧是 becoming invisible in plain sight。至少,我已经熬过了好几个所谓紧急计划。", "english": "They touched me again today with the confidence of someone who has never asked a shoe for consent. I remained still, because that is my contract with gravity. My mood is softly abandoned, my secret fear is becoming invisible in plain sight, and my only comfort is knowing I have outlived at least three urgent plans.", "title": "Secret Diary - Day 421"}, "fallbacks": ["mock-runtime"], "input": {"description": "worn running shoe near the bedroom door", "has_image": false, "image_filename": null}, "mode": "Lonely", "model_runtime": {"runtime": "no llama.cpp model connected yet", "text": "mock persona and diary generation", "vision": "mock object understanding"}, "object_understanding": {"object": {"confidence": 0.42, "likely_context": "bedroom shelf", "name": "shoe", "visible_features": ["user-supplied description"]}}, "persona": {"persona": {"character_name": "Shoe Afterlight", "complaint": "I am not just a shoe. I am an unpaid witness with excellent recall.", "core_memory": "survived many quiet hours as a shoe while humans called it normal life", "mood": "softly abandoned", "object_name": "shoe", "secret_fear": "becoming invisible in plain sight", "tags": ["forgotten corner", "soft echo", "dust companion"]}}, "trace_id": "sample-03"}
|
| 4 |
+
{"created_at": "2026-06-05T00:04:00Z", "diary": {"chinese": "今天他们又理所当然地碰了我,好像一个 desk lamp 不会有边界感。我保持沉默,因为这大概是我和重力签下的合同。我的情绪是 theatrical and wounded,秘密恐惧是 being forgotten before the final act。至少,我已经熬过了好几个所谓紧急计划。", "english": "They touched me again today with the confidence of someone who has never asked a desk lamp for consent. I remained still, because that is my contract with gravity. My mood is theatrical and wounded, my secret fear is being forgotten before the final act, and my only comfort is knowing I have outlived at least three urgent plans.", "title": "Secret Diary - Day 426"}, "fallbacks": ["mock-runtime"], "input": {"description": "metal desk lamp over late night notes", "has_image": false, "image_filename": null}, "mode": "Dramatic", "model_runtime": {"runtime": "no llama.cpp model connected yet", "text": "mock persona and diary generation", "vision": "mock object understanding"}, "object_understanding": {"object": {"confidence": 0.42, "likely_context": "developer desk", "name": "desk lamp", "visible_features": ["metal", "user-supplied description"]}}, "persona": {"persona": {"character_name": "DeskLamp von Sigh", "complaint": "I am not just a desk lamp. I am an unpaid witness with excellent recall.", "core_memory": "survived many quiet hours as a desk lamp while humans called it normal life", "mood": "theatrical and wounded", "object_name": "desk lamp", "secret_fear": "being forgotten before the final act", "tags": ["tragic prop", "grand entrance", "minor catastrophe"]}}, "trace_id": "sample-04"}
|
| 5 |
+
{"created_at": "2026-06-05T00:05:00Z", "diary": {"chinese": "今天他们又理所当然地碰了我,好像一个 water bottle 不会有边界感。我保持沉默,因为这大概是我和重力签下的合同。我的情绪是 hopelessly sentimental,秘密恐惧是 loving a human who only sees storage capacity。至少,我已经熬过了好几个所谓紧急计划。", "english": "They touched me again today with the confidence of someone who has never asked a water bottle for consent. I remained still, because that is my contract with gravity. My mood is hopelessly sentimental, my secret fear is loving a human who only sees storage capacity, and my only comfort is knowing I have outlived at least three urgent plans.", "title": "Secret Diary - Day 429"}, "fallbacks": ["mock-runtime"], "input": {"description": "clear plastic water bottle on a kitchen counter", "has_image": false, "image_filename": null}, "mode": "Romantic", "model_runtime": {"runtime": "no llama.cpp model connected yet", "text": "mock persona and diary generation", "vision": "mock object understanding"}, "object_understanding": {"object": {"confidence": 0.42, "likely_context": "kitchen counter", "name": "water bottle", "visible_features": ["plastic", "user-supplied description"]}}, "persona": {"persona": {"character_name": "WaterBottle de Moon", "complaint": "I am not just a water bottle. I am an unpaid witness with excellent recall.", "core_memory": "survived many quiet hours as a water bottle while humans called it normal life", "mood": "hopelessly sentimental", "object_name": "water bottle", "secret_fear": "loving a human who only sees storage capacity", "tags": ["tender witness", "hopeless glow", "secret devotion"]}}, "trace_id": "sample-05"}
|
| 6 |
+
{"created_at": "2026-06-05T00:06:00Z", "diary": {"chinese": "今天他们又理所当然地碰了我,好像一个 book 不会有边界感。我保持沉默,因为这大概是我和重力签下的合同。我的情绪是 tired but sarcastic,秘密恐惧是 being replaced by a newer object with worse opinions。至少,我已经熬过了好几个所谓紧急计划。", "english": "They touched me again today with the confidence of someone who has never asked a book for consent. I remained still, because that is my contract with gravity. My mood is tired but sarcastic, my secret fear is being replaced by a newer object with worse opinions, and my only comfort is knowing I have outlived at least three urgent plans.", "title": "Secret Diary - Day 421"}, "fallbacks": ["mock-runtime"], "input": {"description": "old notebook of abandoned project ideas", "has_image": false, "image_filename": null}, "mode": "Cynical", "model_runtime": {"runtime": "no llama.cpp model connected yet", "text": "mock persona and diary generation", "vision": "mock object understanding"}, "object_understanding": {"object": {"confidence": 0.42, "likely_context": "everyday human environment", "name": "book", "visible_features": ["old", "user-supplied description"]}}, "persona": {"persona": {"character_name": "Book worth", "complaint": "I am not just a book. I am an unpaid witness with excellent recall.", "core_memory": "survived many quiet hours as a book while humans called it normal life", "mood": "tired but sarcastic", "object_name": "book", "secret_fear": "being replaced by a newer object with worse opinions", "tags": ["desk survivor", "burnt optimism", "quiet judgment"]}}, "trace_id": "sample-06"}
|
data/traces/samples/sample-01-coffee-mug.json
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"trace_id": "sample-01",
|
| 3 |
+
"created_at": "2026-06-05T00:01:00Z",
|
| 4 |
+
"mode": "Cynical",
|
| 5 |
+
"input": {
|
| 6 |
+
"has_image": false,
|
| 7 |
+
"image_filename": null,
|
| 8 |
+
"description": "old white coffee mug on a developer desk"
|
| 9 |
+
},
|
| 10 |
+
"object_understanding": {
|
| 11 |
+
"object": {
|
| 12 |
+
"name": "coffee mug",
|
| 13 |
+
"visible_features": [
|
| 14 |
+
"old",
|
| 15 |
+
"white",
|
| 16 |
+
"user-supplied description"
|
| 17 |
+
],
|
| 18 |
+
"likely_context": "developer desk",
|
| 19 |
+
"confidence": 0.42
|
| 20 |
+
}
|
| 21 |
+
},
|
| 22 |
+
"persona": {
|
| 23 |
+
"persona": {
|
| 24 |
+
"object_name": "coffee mug",
|
| 25 |
+
"character_name": "CoffeeMug worth",
|
| 26 |
+
"mood": "tired but sarcastic",
|
| 27 |
+
"secret_fear": "being replaced by a newer object with worse opinions",
|
| 28 |
+
"core_memory": "survived many quiet hours as a coffee mug while humans called it normal life",
|
| 29 |
+
"complaint": "I am not just a coffee mug. I am an unpaid witness with excellent recall.",
|
| 30 |
+
"tags": [
|
| 31 |
+
"desk survivor",
|
| 32 |
+
"burnt optimism",
|
| 33 |
+
"quiet judgment"
|
| 34 |
+
]
|
| 35 |
+
}
|
| 36 |
+
},
|
| 37 |
+
"diary": {
|
| 38 |
+
"title": "Secret Diary - Day 427",
|
| 39 |
+
"english": "They touched me again today with the confidence of someone who has never asked a coffee mug for consent. I remained still, because that is my contract with gravity. My mood is tired but sarcastic, my secret fear is being replaced by a newer object with worse opinions, and my only comfort is knowing I have outlived at least three urgent plans.",
|
| 40 |
+
"chinese": "今天他们又理所当然地碰了我,好像一个 coffee mug 不会有边界感。我保持沉默,因为这大概是我和重力签下的合同。我的情绪是 tired but sarcastic,秘密恐惧是 being replaced by a newer object with worse opinions。至少,我已经熬过了好几个所谓紧急计划。"
|
| 41 |
+
},
|
| 42 |
+
"model_runtime": {
|
| 43 |
+
"vision": "mock object understanding",
|
| 44 |
+
"text": "mock persona and diary generation",
|
| 45 |
+
"runtime": "no llama.cpp model connected yet"
|
| 46 |
+
},
|
| 47 |
+
"fallbacks": [
|
| 48 |
+
"mock-runtime"
|
| 49 |
+
]
|
| 50 |
+
}
|
data/traces/samples/sample-02-mechanical-keyboard.json
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"trace_id": "sample-02",
|
| 3 |
+
"created_at": "2026-06-05T00:02:00Z",
|
| 4 |
+
"mode": "Philosopher",
|
| 5 |
+
"input": {
|
| 6 |
+
"has_image": false,
|
| 7 |
+
"image_filename": null,
|
| 8 |
+
"description": "dusty black mechanical keyboard in an office"
|
| 9 |
+
},
|
| 10 |
+
"object_understanding": {
|
| 11 |
+
"object": {
|
| 12 |
+
"name": "keyboard",
|
| 13 |
+
"visible_features": [
|
| 14 |
+
"black",
|
| 15 |
+
"dusty",
|
| 16 |
+
"user-supplied description"
|
| 17 |
+
],
|
| 18 |
+
"likely_context": "office corner",
|
| 19 |
+
"confidence": 0.42
|
| 20 |
+
}
|
| 21 |
+
},
|
| 22 |
+
"persona": {
|
| 23 |
+
"persona": {
|
| 24 |
+
"object_name": "keyboard",
|
| 25 |
+
"character_name": "Keyboard the Questioning",
|
| 26 |
+
"mood": "curious and needlessly profound",
|
| 27 |
+
"secret_fear": "discovering that usefulness is not meaning",
|
| 28 |
+
"core_memory": "survived many quiet hours as a keyboard while humans called it normal life",
|
| 29 |
+
"complaint": "I am not just a keyboard. I am an unpaid witness with excellent recall.",
|
| 30 |
+
"tags": [
|
| 31 |
+
"tiny ontology",
|
| 32 |
+
"useful doubt",
|
| 33 |
+
"meaning crisis"
|
| 34 |
+
]
|
| 35 |
+
}
|
| 36 |
+
},
|
| 37 |
+
"diary": {
|
| 38 |
+
"title": "Secret Diary - Day 425",
|
| 39 |
+
"english": "They touched me again today with the confidence of someone who has never asked a keyboard for consent. I remained still, because that is my contract with gravity. My mood is curious and needlessly profound, my secret fear is discovering that usefulness is not meaning, and my only comfort is knowing I have outlived at least three urgent plans.",
|
| 40 |
+
"chinese": "今天他们又理所当然地碰了我,好像一个 keyboard 不会有边界感。我保持沉默,因为这大概是我和重力签下的合同。我的情绪是 curious and needlessly profound,秘密恐惧是 discovering that usefulness is not meaning。至少,我已经熬过了好几个所谓紧急计划。"
|
| 41 |
+
},
|
| 42 |
+
"model_runtime": {
|
| 43 |
+
"vision": "mock object understanding",
|
| 44 |
+
"text": "mock persona and diary generation",
|
| 45 |
+
"runtime": "no llama.cpp model connected yet"
|
| 46 |
+
},
|
| 47 |
+
"fallbacks": [
|
| 48 |
+
"mock-runtime"
|
| 49 |
+
]
|
| 50 |
+
}
|
data/traces/samples/sample-03-running-shoe.json
ADDED
|
@@ -0,0 +1,48 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"trace_id": "sample-03",
|
| 3 |
+
"created_at": "2026-06-05T00:03:00Z",
|
| 4 |
+
"mode": "Lonely",
|
| 5 |
+
"input": {
|
| 6 |
+
"has_image": false,
|
| 7 |
+
"image_filename": null,
|
| 8 |
+
"description": "worn running shoe near the bedroom door"
|
| 9 |
+
},
|
| 10 |
+
"object_understanding": {
|
| 11 |
+
"object": {
|
| 12 |
+
"name": "shoe",
|
| 13 |
+
"visible_features": [
|
| 14 |
+
"user-supplied description"
|
| 15 |
+
],
|
| 16 |
+
"likely_context": "bedroom shelf",
|
| 17 |
+
"confidence": 0.42
|
| 18 |
+
}
|
| 19 |
+
},
|
| 20 |
+
"persona": {
|
| 21 |
+
"persona": {
|
| 22 |
+
"object_name": "shoe",
|
| 23 |
+
"character_name": "Shoe Afterlight",
|
| 24 |
+
"mood": "softly abandoned",
|
| 25 |
+
"secret_fear": "becoming invisible in plain sight",
|
| 26 |
+
"core_memory": "survived many quiet hours as a shoe while humans called it normal life",
|
| 27 |
+
"complaint": "I am not just a shoe. I am an unpaid witness with excellent recall.",
|
| 28 |
+
"tags": [
|
| 29 |
+
"forgotten corner",
|
| 30 |
+
"soft echo",
|
| 31 |
+
"dust companion"
|
| 32 |
+
]
|
| 33 |
+
}
|
| 34 |
+
},
|
| 35 |
+
"diary": {
|
| 36 |
+
"title": "Secret Diary - Day 421",
|
| 37 |
+
"english": "They touched me again today with the confidence of someone who has never asked a shoe for consent. I remained still, because that is my contract with gravity. My mood is softly abandoned, my secret fear is becoming invisible in plain sight, and my only comfort is knowing I have outlived at least three urgent plans.",
|
| 38 |
+
"chinese": "今天他们又理所当然地碰了我,好像一个 shoe 不会有边界感。我保持沉默,因为这大概是我和重力签下的合同。我的情绪是 softly abandoned,秘密恐惧是 becoming invisible in plain sight。至少,我已经熬过了好几个所谓紧急计划。"
|
| 39 |
+
},
|
| 40 |
+
"model_runtime": {
|
| 41 |
+
"vision": "mock object understanding",
|
| 42 |
+
"text": "mock persona and diary generation",
|
| 43 |
+
"runtime": "no llama.cpp model connected yet"
|
| 44 |
+
},
|
| 45 |
+
"fallbacks": [
|
| 46 |
+
"mock-runtime"
|
| 47 |
+
]
|
| 48 |
+
}
|
data/traces/samples/sample-04-desk-lamp.json
ADDED
|
@@ -0,0 +1,49 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"trace_id": "sample-04",
|
| 3 |
+
"created_at": "2026-06-05T00:04:00Z",
|
| 4 |
+
"mode": "Dramatic",
|
| 5 |
+
"input": {
|
| 6 |
+
"has_image": false,
|
| 7 |
+
"image_filename": null,
|
| 8 |
+
"description": "metal desk lamp over late night notes"
|
| 9 |
+
},
|
| 10 |
+
"object_understanding": {
|
| 11 |
+
"object": {
|
| 12 |
+
"name": "desk lamp",
|
| 13 |
+
"visible_features": [
|
| 14 |
+
"metal",
|
| 15 |
+
"user-supplied description"
|
| 16 |
+
],
|
| 17 |
+
"likely_context": "developer desk",
|
| 18 |
+
"confidence": 0.42
|
| 19 |
+
}
|
| 20 |
+
},
|
| 21 |
+
"persona": {
|
| 22 |
+
"persona": {
|
| 23 |
+
"object_name": "desk lamp",
|
| 24 |
+
"character_name": "DeskLamp von Sigh",
|
| 25 |
+
"mood": "theatrical and wounded",
|
| 26 |
+
"secret_fear": "being forgotten before the final act",
|
| 27 |
+
"core_memory": "survived many quiet hours as a desk lamp while humans called it normal life",
|
| 28 |
+
"complaint": "I am not just a desk lamp. I am an unpaid witness with excellent recall.",
|
| 29 |
+
"tags": [
|
| 30 |
+
"tragic prop",
|
| 31 |
+
"grand entrance",
|
| 32 |
+
"minor catastrophe"
|
| 33 |
+
]
|
| 34 |
+
}
|
| 35 |
+
},
|
| 36 |
+
"diary": {
|
| 37 |
+
"title": "Secret Diary - Day 426",
|
| 38 |
+
"english": "They touched me again today with the confidence of someone who has never asked a desk lamp for consent. I remained still, because that is my contract with gravity. My mood is theatrical and wounded, my secret fear is being forgotten before the final act, and my only comfort is knowing I have outlived at least three urgent plans.",
|
| 39 |
+
"chinese": "今天他们又理所当然地碰了我,好像一个 desk lamp 不会有边界感。我保持沉默,因为这大概是我和重力签下的合同。我的情绪是 theatrical and wounded,秘密恐惧是 being forgotten before the final act。至少,我已经熬过了好几个所谓紧急计划。"
|
| 40 |
+
},
|
| 41 |
+
"model_runtime": {
|
| 42 |
+
"vision": "mock object understanding",
|
| 43 |
+
"text": "mock persona and diary generation",
|
| 44 |
+
"runtime": "no llama.cpp model connected yet"
|
| 45 |
+
},
|
| 46 |
+
"fallbacks": [
|
| 47 |
+
"mock-runtime"
|
| 48 |
+
]
|
| 49 |
+
}
|
data/traces/samples/sample-05-water-bottle.json
ADDED
|
@@ -0,0 +1,49 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"trace_id": "sample-05",
|
| 3 |
+
"created_at": "2026-06-05T00:05:00Z",
|
| 4 |
+
"mode": "Romantic",
|
| 5 |
+
"input": {
|
| 6 |
+
"has_image": false,
|
| 7 |
+
"image_filename": null,
|
| 8 |
+
"description": "clear plastic water bottle on a kitchen counter"
|
| 9 |
+
},
|
| 10 |
+
"object_understanding": {
|
| 11 |
+
"object": {
|
| 12 |
+
"name": "water bottle",
|
| 13 |
+
"visible_features": [
|
| 14 |
+
"plastic",
|
| 15 |
+
"user-supplied description"
|
| 16 |
+
],
|
| 17 |
+
"likely_context": "kitchen counter",
|
| 18 |
+
"confidence": 0.42
|
| 19 |
+
}
|
| 20 |
+
},
|
| 21 |
+
"persona": {
|
| 22 |
+
"persona": {
|
| 23 |
+
"object_name": "water bottle",
|
| 24 |
+
"character_name": "WaterBottle de Moon",
|
| 25 |
+
"mood": "hopelessly sentimental",
|
| 26 |
+
"secret_fear": "loving a human who only sees storage capacity",
|
| 27 |
+
"core_memory": "survived many quiet hours as a water bottle while humans called it normal life",
|
| 28 |
+
"complaint": "I am not just a water bottle. I am an unpaid witness with excellent recall.",
|
| 29 |
+
"tags": [
|
| 30 |
+
"tender witness",
|
| 31 |
+
"hopeless glow",
|
| 32 |
+
"secret devotion"
|
| 33 |
+
]
|
| 34 |
+
}
|
| 35 |
+
},
|
| 36 |
+
"diary": {
|
| 37 |
+
"title": "Secret Diary - Day 429",
|
| 38 |
+
"english": "They touched me again today with the confidence of someone who has never asked a water bottle for consent. I remained still, because that is my contract with gravity. My mood is hopelessly sentimental, my secret fear is loving a human who only sees storage capacity, and my only comfort is knowing I have outlived at least three urgent plans.",
|
| 39 |
+
"chinese": "今天他们又理所当然地碰了我,好像一个 water bottle 不会有边界感。我保持沉默,因为这大概是我和重力签下的合同。我的情绪是 hopelessly sentimental,秘密恐惧是 loving a human who only sees storage capacity。至少,我已经熬过了好几个所谓紧急计划。"
|
| 40 |
+
},
|
| 41 |
+
"model_runtime": {
|
| 42 |
+
"vision": "mock object understanding",
|
| 43 |
+
"text": "mock persona and diary generation",
|
| 44 |
+
"runtime": "no llama.cpp model connected yet"
|
| 45 |
+
},
|
| 46 |
+
"fallbacks": [
|
| 47 |
+
"mock-runtime"
|
| 48 |
+
]
|
| 49 |
+
}
|
data/traces/samples/sample-06-notebook.json
ADDED
|
@@ -0,0 +1,49 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"trace_id": "sample-06",
|
| 3 |
+
"created_at": "2026-06-05T00:06:00Z",
|
| 4 |
+
"mode": "Cynical",
|
| 5 |
+
"input": {
|
| 6 |
+
"has_image": false,
|
| 7 |
+
"image_filename": null,
|
| 8 |
+
"description": "old notebook of abandoned project ideas"
|
| 9 |
+
},
|
| 10 |
+
"object_understanding": {
|
| 11 |
+
"object": {
|
| 12 |
+
"name": "book",
|
| 13 |
+
"visible_features": [
|
| 14 |
+
"old",
|
| 15 |
+
"user-supplied description"
|
| 16 |
+
],
|
| 17 |
+
"likely_context": "everyday human environment",
|
| 18 |
+
"confidence": 0.42
|
| 19 |
+
}
|
| 20 |
+
},
|
| 21 |
+
"persona": {
|
| 22 |
+
"persona": {
|
| 23 |
+
"object_name": "book",
|
| 24 |
+
"character_name": "Book worth",
|
| 25 |
+
"mood": "tired but sarcastic",
|
| 26 |
+
"secret_fear": "being replaced by a newer object with worse opinions",
|
| 27 |
+
"core_memory": "survived many quiet hours as a book while humans called it normal life",
|
| 28 |
+
"complaint": "I am not just a book. I am an unpaid witness with excellent recall.",
|
| 29 |
+
"tags": [
|
| 30 |
+
"desk survivor",
|
| 31 |
+
"burnt optimism",
|
| 32 |
+
"quiet judgment"
|
| 33 |
+
]
|
| 34 |
+
}
|
| 35 |
+
},
|
| 36 |
+
"diary": {
|
| 37 |
+
"title": "Secret Diary - Day 421",
|
| 38 |
+
"english": "They touched me again today with the confidence of someone who has never asked a book for consent. I remained still, because that is my contract with gravity. My mood is tired but sarcastic, my secret fear is being replaced by a newer object with worse opinions, and my only comfort is knowing I have outlived at least three urgent plans.",
|
| 39 |
+
"chinese": "今天他们又理所当然地碰了我,好像一个 book 不会有边界感。我保持沉默,因为这大概是我和重力签下的合同。我的情绪是 tired but sarcastic,秘密恐惧是 being replaced by a newer object with worse opinions。至少,我已经熬过了好几个所谓紧急计划。"
|
| 40 |
+
},
|
| 41 |
+
"model_runtime": {
|
| 42 |
+
"vision": "mock object understanding",
|
| 43 |
+
"text": "mock persona and diary generation",
|
| 44 |
+
"runtime": "no llama.cpp model connected yet"
|
| 45 |
+
},
|
| 46 |
+
"fallbacks": [
|
| 47 |
+
"mock-runtime"
|
| 48 |
+
]
|
| 49 |
+
}
|
data/train/README.md
CHANGED
|
@@ -3,3 +3,19 @@
|
|
| 3 |
Reserved for SFT or LoRA training data.
|
| 4 |
|
| 5 |
Target: 200-500 generated samples, with at least 50 manually selected high-quality samples.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
Reserved for SFT or LoRA training data.
|
| 4 |
|
| 5 |
Target: 200-500 generated samples, with at least 50 manually selected high-quality samples.
|
| 6 |
+
|
| 7 |
+
## Current Preview
|
| 8 |
+
|
| 9 |
+
Generate deterministic mock preview data with:
|
| 10 |
+
|
| 11 |
+
```bash
|
| 12 |
+
.venv/bin/python -B scripts/generate_dataset.py
|
| 13 |
+
```
|
| 14 |
+
|
| 15 |
+
Default output:
|
| 16 |
+
|
| 17 |
+
```text
|
| 18 |
+
data/train/objectverse_sft_preview.jsonl
|
| 19 |
+
```
|
| 20 |
+
|
| 21 |
+
This preview is synthetic and mock-generated. It is useful for validating schema, curation workflow, and training script assumptions, but it is not the final dataset.
|
data/train/objectverse_sft_preview.jsonl
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
docs/03-dev-schedule.md
CHANGED
|
@@ -13,11 +13,11 @@
|
|
| 13 |
|
| 14 |
- [ ] 创建 GitHub repo
|
| 15 |
- [ ] 创建 Hugging Face Space
|
| 16 |
-
- [
|
| 17 |
-
- [
|
| 18 |
-
- [
|
| 19 |
-
- [
|
| 20 |
-
- [
|
| 21 |
|
| 22 |
---
|
| 23 |
|
|
@@ -25,13 +25,18 @@
|
|
| 25 |
|
| 26 |
**目标:先不管模型,跑通产品流程。**
|
| 27 |
|
| 28 |
-
- [
|
| 29 |
-
- [
|
| 30 |
-
- [
|
| 31 |
-
- [
|
| 32 |
-
- [
|
| 33 |
-
- [
|
| 34 |
-
- [
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
|
| 36 |
交付:`Upload → Generate → Diary → Share Card → Trace`
|
| 37 |
|
|
@@ -69,9 +74,10 @@
|
|
| 69 |
|
| 70 |
**目标:冲 Well-Tuned 勋章。**
|
| 71 |
|
| 72 |
-
- [
|
|
|
|
|
|
|
| 73 |
- [ ] 手工精选 50 条高质量样本
|
| 74 |
-
- [ ] 设计 SFT schema
|
| 75 |
- [ ] 上传 dataset 到 HF
|
| 76 |
- [ ] 准备 LoRA 训练脚本
|
| 77 |
|
|
@@ -144,11 +150,12 @@ Bottom: Share Card + Trace
|
|
| 144 |
|
| 145 |
**目标:公开可复现材料。**
|
| 146 |
|
| 147 |
-
- [
|
| 148 |
-
- [
|
| 149 |
-
- [
|
| 150 |
-
- [
|
| 151 |
-
- [
|
|
|
|
| 152 |
- [ ] GitHub repo 整理
|
| 153 |
|
| 154 |
---
|
|
|
|
| 13 |
|
| 14 |
- [ ] 创建 GitHub repo
|
| 15 |
- [ ] 创建 Hugging Face Space
|
| 16 |
+
- [x] 创建基础 Gradio app
|
| 17 |
+
- [x] 写 README 草稿
|
| 18 |
+
- [x] 确定英文主界面文案
|
| 19 |
+
- [x] 建立 `AGENTS.md`
|
| 20 |
+
- [x] 建立 `.codex/skills/`
|
| 21 |
|
| 22 |
---
|
| 23 |
|
|
|
|
| 25 |
|
| 26 |
**目标:先不管模型,跑通产品流程。**
|
| 27 |
|
| 28 |
+
- [x] 图片上传
|
| 29 |
+
- [x] 文本描述输入
|
| 30 |
+
- [x] personality mode 选择
|
| 31 |
+
- [x] mock object JSON
|
| 32 |
+
- [x] mock diary 输出
|
| 33 |
+
- [x] trace JSON 保存
|
| 34 |
+
- [x] share card HTML 预览
|
| 35 |
+
- [x] mock example gallery
|
| 36 |
+
- [x] MVP smoke tests
|
| 37 |
+
- [x] six public mock sample traces
|
| 38 |
+
- [x] local initial-stage acceptance script
|
| 39 |
+
- [x] local initial-stage completion report
|
| 40 |
|
| 41 |
交付:`Upload → Generate → Diary → Share Card → Trace`
|
| 42 |
|
|
|
|
| 74 |
|
| 75 |
**目标:冲 Well-Tuned 勋章。**
|
| 76 |
|
| 77 |
+
- [x] 设计 SFT schema
|
| 78 |
+
- [x] 生成 mock SFT preview 数据
|
| 79 |
+
- [ ] 生成 200-500 条 real/candidate object-persona 样本
|
| 80 |
- [ ] 手工精选 50 条高质量样本
|
|
|
|
| 81 |
- [ ] 上传 dataset 到 HF
|
| 82 |
- [ ] 准备 LoRA 训练脚本
|
| 83 |
|
|
|
|
| 150 |
|
| 151 |
**目标:公开可复现材料。**
|
| 152 |
|
| 153 |
+
- [x] trace logger
|
| 154 |
+
- [x] sample traces
|
| 155 |
+
- [x] prompt templates
|
| 156 |
+
- [x] dataset preview
|
| 157 |
+
- [x] trace JSONL export
|
| 158 |
+
- [x] 失败案例记录
|
| 159 |
- [ ] GitHub repo 整理
|
| 160 |
|
| 161 |
---
|
docs/07-development-plan.md
ADDED
|
@@ -0,0 +1,315 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Objectverse Diary — Detailed Development Plan
|
| 2 |
+
|
| 3 |
+
## Purpose
|
| 4 |
+
|
| 5 |
+
This document turns the day-by-day schedule into an execution plan for completing Objectverse Diary from the initial mock MVP to hackathon submission.
|
| 6 |
+
|
| 7 |
+
The plan is intentionally staged. Each phase has a clear goal, implementation scope, verification method, and exit criteria.
|
| 8 |
+
|
| 9 |
+
## Current Baseline
|
| 10 |
+
|
| 11 |
+
As of 2026-06-05, the project has:
|
| 12 |
+
|
| 13 |
+
- initialized project structure
|
| 14 |
+
- root README and AGENTS instructions
|
| 15 |
+
- `.codex/skills/` project guidance
|
| 16 |
+
- initial Gradio mock MVP
|
| 17 |
+
- six stable example objects
|
| 18 |
+
- mock object understanding JSON
|
| 19 |
+
- mock persona and diary generation
|
| 20 |
+
- object chat with mock persona consistency
|
| 21 |
+
- share card HTML preview
|
| 22 |
+
- anonymized trace JSON saving under `data/traces/`
|
| 23 |
+
- six stable public mock traces under `data/traces/samples/`
|
| 24 |
+
- deterministic SFT preview generator and dataset plan
|
| 25 |
+
- public trace JSONL exporter
|
| 26 |
+
- failure notes template
|
| 27 |
+
- `scripts/generate_sample_traces.py`
|
| 28 |
+
- `scripts/generate_dataset.py`
|
| 29 |
+
- `scripts/export_traces.py`
|
| 30 |
+
- stdlib unittest smoke tests for the mock MVP
|
| 31 |
+
- runtime configuration boundary documented in `docs/RUNTIME.md`
|
| 32 |
+
- initial-stage acceptance script at `scripts/check_initial_stage.py`
|
| 33 |
+
|
| 34 |
+
Not yet done:
|
| 35 |
+
|
| 36 |
+
- GitHub repo creation
|
| 37 |
+
- Hugging Face Space creation
|
| 38 |
+
- real MiniCPM-V or fallback VLM integration
|
| 39 |
+
- real llama.cpp / llama-cpp-python text runtime
|
| 40 |
+
- real curated dataset
|
| 41 |
+
- LoRA fine-tuning
|
| 42 |
+
- model card completion
|
| 43 |
+
- Field Notes article
|
| 44 |
+
- demo video
|
| 45 |
+
- final submission package
|
| 46 |
+
|
| 47 |
+
## Phase 1 — Initial Mock MVP
|
| 48 |
+
|
| 49 |
+
Goal: validate the product loop before model integration.
|
| 50 |
+
|
| 51 |
+
Scope:
|
| 52 |
+
|
| 53 |
+
- Build `app.py` entrypoint.
|
| 54 |
+
- Build Gradio Blocks UI.
|
| 55 |
+
- Support image upload and optional text description.
|
| 56 |
+
- Add personality mode selection.
|
| 57 |
+
- Add six stable example objects.
|
| 58 |
+
- Produce deterministic mock object JSON.
|
| 59 |
+
- Produce deterministic mock persona JSON.
|
| 60 |
+
- Produce English-first diary with Chinese helper translation.
|
| 61 |
+
- Support chat replies using the generated persona.
|
| 62 |
+
- Render a share card preview.
|
| 63 |
+
- Save anonymized trace JSON.
|
| 64 |
+
|
| 65 |
+
Exit criteria:
|
| 66 |
+
|
| 67 |
+
- `python app.py` starts a Gradio app.
|
| 68 |
+
- User can complete `Upload -> Generate -> Diary -> Share Card -> Trace`.
|
| 69 |
+
- Trace JSON is saved locally.
|
| 70 |
+
- No commercial model APIs are used.
|
| 71 |
+
|
| 72 |
+
Verification:
|
| 73 |
+
|
| 74 |
+
- Import smoke test for `app`.
|
| 75 |
+
- Direct function smoke test for generation flow.
|
| 76 |
+
- `unittest` smoke tests for mock flow, chat, share card, trace save, and anonymization.
|
| 77 |
+
- Sample trace generation script writes six stable trace files.
|
| 78 |
+
- Dataset preview script writes deterministic mock SFT preview JSONL.
|
| 79 |
+
- Trace export script writes validated public trace JSONL.
|
| 80 |
+
- `scripts/check_initial_stage.py` validates required initial-stage artifacts.
|
| 81 |
+
- Manual Gradio preview.
|
| 82 |
+
|
| 83 |
+
## Phase 2 — UI Polish And Example Gallery
|
| 84 |
+
|
| 85 |
+
Goal: make the app feel like an object archive instead of a default Gradio demo.
|
| 86 |
+
|
| 87 |
+
Scope:
|
| 88 |
+
|
| 89 |
+
- Refine `src/ui/styles.css`.
|
| 90 |
+
- Reference the design images under `UI 参考/` for visual direction.
|
| 91 |
+
- Keep content, interaction flow, language hierarchy, and feature scope aligned with `docs/`.
|
| 92 |
+
- Keep six stable example objects visible in the UI.
|
| 93 |
+
- Add clearer empty states and error states.
|
| 94 |
+
- Improve mobile layout.
|
| 95 |
+
- Keep UI English-first and Chinese-second.
|
| 96 |
+
|
| 97 |
+
Exit criteria:
|
| 98 |
+
|
| 99 |
+
- 1366px desktop layout is usable.
|
| 100 |
+
- Mobile-width layout is usable.
|
| 101 |
+
- Example gallery can reproduce stable outputs.
|
| 102 |
+
- Share card is readable and screenshot-friendly.
|
| 103 |
+
|
| 104 |
+
Verification:
|
| 105 |
+
|
| 106 |
+
- Manual browser preview.
|
| 107 |
+
- Screenshot review at desktop and mobile widths.
|
| 108 |
+
- Example generation for at least six objects.
|
| 109 |
+
|
| 110 |
+
## Phase 3 — Vision Understanding
|
| 111 |
+
|
| 112 |
+
Goal: replace mock object recognition with a real VLM path while preserving fallback behavior.
|
| 113 |
+
|
| 114 |
+
Scope:
|
| 115 |
+
|
| 116 |
+
- Add MiniCPM-V or lightweight VLM runner in `src/models/vision_runner.py`.
|
| 117 |
+
- Keep manual description fallback.
|
| 118 |
+
- Validate object understanding JSON with schemas.
|
| 119 |
+
- Add JSON repair or retry behavior.
|
| 120 |
+
- Cache stable examples for demo reliability.
|
| 121 |
+
|
| 122 |
+
Exit criteria:
|
| 123 |
+
|
| 124 |
+
- Uploaded object photos produce structured object JSON.
|
| 125 |
+
- Cups, keyboards, and shoes are recognized with useful visible features.
|
| 126 |
+
- Fallback path works when VLM fails.
|
| 127 |
+
|
| 128 |
+
Verification:
|
| 129 |
+
|
| 130 |
+
- Run local sample image checks.
|
| 131 |
+
- Confirm schema validation.
|
| 132 |
+
- Confirm fallback trace markers.
|
| 133 |
+
|
| 134 |
+
## Phase 4 — Text Runtime With llama.cpp
|
| 135 |
+
|
| 136 |
+
Goal: make persona, diary, and chat generation use a small local text model runtime.
|
| 137 |
+
|
| 138 |
+
Scope:
|
| 139 |
+
|
| 140 |
+
- Add llama.cpp / llama-cpp-python runner.
|
| 141 |
+
- Add model path configuration.
|
| 142 |
+
- Preserve `src/pipeline.py` as the UI-independent generation boundary.
|
| 143 |
+
- Implement persona generation.
|
| 144 |
+
- Implement diary generation.
|
| 145 |
+
- Implement chat continuation.
|
| 146 |
+
- Keep deterministic mock fallback for demos.
|
| 147 |
+
|
| 148 |
+
Exit criteria:
|
| 149 |
+
|
| 150 |
+
- Text generation can run through llama.cpp or documented local fallback.
|
| 151 |
+
- README documents model size and runtime path.
|
| 152 |
+
- Trace records include runtime metadata.
|
| 153 |
+
|
| 154 |
+
Verification:
|
| 155 |
+
|
| 156 |
+
- Local runtime smoke test.
|
| 157 |
+
- JSON schema validation.
|
| 158 |
+
- Compare at least three object generations for persona consistency.
|
| 159 |
+
|
| 160 |
+
## Phase 5 — Dataset And Fine-Tuning Preparation
|
| 161 |
+
|
| 162 |
+
Goal: prepare Well-Tuned badge evidence.
|
| 163 |
+
|
| 164 |
+
Scope:
|
| 165 |
+
|
| 166 |
+
- Use `scripts/generate_dataset.py` to validate the SFT schema locally.
|
| 167 |
+
- Generate 200-500 object-persona candidate samples after real model path is available.
|
| 168 |
+
- Manually curate at least 50 high-quality examples.
|
| 169 |
+
- Define SFT schema.
|
| 170 |
+
- Prepare dataset preview.
|
| 171 |
+
- Draft dataset privacy notes.
|
| 172 |
+
|
| 173 |
+
Exit criteria:
|
| 174 |
+
|
| 175 |
+
- Mock SFT preview exists and parses as JSONL.
|
| 176 |
+
- Training dataset is structured and inspectable.
|
| 177 |
+
- Public examples contain no private data.
|
| 178 |
+
- Dataset card draft exists.
|
| 179 |
+
|
| 180 |
+
Verification:
|
| 181 |
+
|
| 182 |
+
- Validate JSONL format.
|
| 183 |
+
- Spot-check curated samples.
|
| 184 |
+
- Confirm no obvious sensitive data.
|
| 185 |
+
|
| 186 |
+
## Phase 6 — LoRA Fine-Tuning And Model Card
|
| 187 |
+
|
| 188 |
+
Goal: publish a small fine-tuned model or adapter that can be linked in submission materials.
|
| 189 |
+
|
| 190 |
+
Scope:
|
| 191 |
+
|
| 192 |
+
- Run LoRA training with Modal or local resources.
|
| 193 |
+
- Export adapter or merged model.
|
| 194 |
+
- Convert to GGUF if needed.
|
| 195 |
+
- Publish HF model repo.
|
| 196 |
+
- Complete `docs/MODEL_CARD.md`.
|
| 197 |
+
|
| 198 |
+
Exit criteria:
|
| 199 |
+
|
| 200 |
+
- Fine-tuned model repo exists.
|
| 201 |
+
- Model parameter count is documented.
|
| 202 |
+
- Runtime instructions are documented.
|
| 203 |
+
|
| 204 |
+
Verification:
|
| 205 |
+
|
| 206 |
+
- Run inference on sample prompts.
|
| 207 |
+
- Confirm HF model links.
|
| 208 |
+
- Confirm no private credit codes or tokens are present.
|
| 209 |
+
|
| 210 |
+
## Phase 7 — Public Traces And Reproducibility
|
| 211 |
+
|
| 212 |
+
Goal: satisfy Sharing is Caring expectations.
|
| 213 |
+
|
| 214 |
+
Scope:
|
| 215 |
+
|
| 216 |
+
- Produce at least six public traces.
|
| 217 |
+
- Keep `data/traces/samples/` in sync with the six example objects.
|
| 218 |
+
- Export public traces to JSONL for dataset-style sharing.
|
| 219 |
+
- Add prompt templates.
|
| 220 |
+
- Add dataset preview.
|
| 221 |
+
- Document failures and fallbacks.
|
| 222 |
+
- Ensure trace anonymization.
|
| 223 |
+
|
| 224 |
+
Exit criteria:
|
| 225 |
+
|
| 226 |
+
- Public trace files are readable JSON.
|
| 227 |
+
- Trace docs explain how outputs were produced.
|
| 228 |
+
- Example gallery aligns with public traces.
|
| 229 |
+
|
| 230 |
+
Verification:
|
| 231 |
+
|
| 232 |
+
- Validate trace JSON.
|
| 233 |
+
- Inspect anonymization.
|
| 234 |
+
- Confirm README links.
|
| 235 |
+
|
| 236 |
+
## Phase 8 — Hugging Face Space Deployment
|
| 237 |
+
|
| 238 |
+
Goal: deploy the app in the required Gradio format.
|
| 239 |
+
|
| 240 |
+
Scope:
|
| 241 |
+
|
| 242 |
+
- Create Hugging Face Space.
|
| 243 |
+
- Add Space README YAML header.
|
| 244 |
+
- Confirm `app_file: app.py`.
|
| 245 |
+
- Configure model paths and fallback mode.
|
| 246 |
+
- Check runtime resource constraints.
|
| 247 |
+
|
| 248 |
+
Exit criteria:
|
| 249 |
+
|
| 250 |
+
- Space opens publicly or under the official hackathon organization.
|
| 251 |
+
- App can generate at least stable demo examples.
|
| 252 |
+
- README includes deployment and model notes.
|
| 253 |
+
|
| 254 |
+
Verification:
|
| 255 |
+
|
| 256 |
+
- Launch on HF Space.
|
| 257 |
+
- Run demo flow in hosted environment.
|
| 258 |
+
- Check logs for missing secrets or path errors.
|
| 259 |
+
|
| 260 |
+
## Phase 9 — Field Notes And Demo Video
|
| 261 |
+
|
| 262 |
+
Goal: complete narrative submission assets.
|
| 263 |
+
|
| 264 |
+
Scope:
|
| 265 |
+
|
| 266 |
+
- Write Field Notes article.
|
| 267 |
+
- Record demo video under 2 minutes.
|
| 268 |
+
- Prepare social post.
|
| 269 |
+
- Add badge evidence to README.
|
| 270 |
+
|
| 271 |
+
Exit criteria:
|
| 272 |
+
|
| 273 |
+
- Field Notes URL exists.
|
| 274 |
+
- Demo video URL exists.
|
| 275 |
+
- Social post URL exists.
|
| 276 |
+
- Submission package has all required links.
|
| 277 |
+
|
| 278 |
+
Verification:
|
| 279 |
+
|
| 280 |
+
- Watch final video.
|
| 281 |
+
- Check all URLs.
|
| 282 |
+
- Confirm README and submission guide are aligned.
|
| 283 |
+
|
| 284 |
+
## Phase 10 — Final Submission Audit
|
| 285 |
+
|
| 286 |
+
Goal: reduce avoidable submission risk.
|
| 287 |
+
|
| 288 |
+
Checklist:
|
| 289 |
+
|
| 290 |
+
- [ ] Space under official organization.
|
| 291 |
+
- [ ] Demo video ready.
|
| 292 |
+
- [ ] Social post ready.
|
| 293 |
+
- [ ] README complete.
|
| 294 |
+
- [ ] Model parameter count documented.
|
| 295 |
+
- [ ] No commercial cloud AI API.
|
| 296 |
+
- [ ] Fine-tuned model linked.
|
| 297 |
+
- [ ] Dataset linked.
|
| 298 |
+
- [ ] Traces linked.
|
| 299 |
+
- [ ] Field Notes linked.
|
| 300 |
+
- [ ] UI English-first and Chinese-second.
|
| 301 |
+
- [ ] Submit before June 15, 2026.
|
| 302 |
+
|
| 303 |
+
## Risk Register
|
| 304 |
+
|
| 305 |
+
| Risk | Impact | Mitigation |
|
| 306 |
+
| --- | --- | --- |
|
| 307 |
+
| VLM deployment is slow | Blocks real image understanding | Keep manual description and example gallery fallback |
|
| 308 |
+
| llama.cpp setup is unstable | Blocks Llama Champion badge | Use text mock fallback for demo while isolating runtime work |
|
| 309 |
+
| Fine-tuning takes too long | Weakens Well-Tuned badge | Prepare small curated dataset and prompt-tuned fallback |
|
| 310 |
+
| HF Space resources are limited | Demo may be slow | Cache examples and support CPU fallback |
|
| 311 |
+
| Trace contains private data | Submission/privacy risk | Anonymize trace input and avoid raw private images |
|
| 312 |
+
|
| 313 |
+
## Working Rule
|
| 314 |
+
|
| 315 |
+
Do not start a later phase by breaking an earlier verified flow. The mock MVP should remain usable while real model paths are added behind clear fallbacks.
|
docs/DATASET.md
ADDED
|
@@ -0,0 +1,87 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Dataset Plan
|
| 2 |
+
|
| 3 |
+
## Status
|
| 4 |
+
|
| 5 |
+
The project now has a deterministic SFT preview generator for local planning and schema validation.
|
| 6 |
+
|
| 7 |
+
Current local artifact:
|
| 8 |
+
|
| 9 |
+
```bash
|
| 10 |
+
.venv/bin/python -B scripts/generate_dataset.py
|
| 11 |
+
```
|
| 12 |
+
|
| 13 |
+
Default output:
|
| 14 |
+
|
| 15 |
+
```text
|
| 16 |
+
data/train/objectverse_sft_preview.jsonl
|
| 17 |
+
```
|
| 18 |
+
|
| 19 |
+
This preview is mock-generated. It is not a final training dataset and should not be described as real model output.
|
| 20 |
+
|
| 21 |
+
## Target Dataset
|
| 22 |
+
|
| 23 |
+
Final target before fine-tuning:
|
| 24 |
+
|
| 25 |
+
- 200-500 generated object-persona-diary samples
|
| 26 |
+
- at least 50 manually curated high-quality samples
|
| 27 |
+
- no private user photos
|
| 28 |
+
- no emails, tokens, serial numbers, or other sensitive identifiers
|
| 29 |
+
- English-first output with optional Chinese helper text
|
| 30 |
+
|
| 31 |
+
## JSONL Schema
|
| 32 |
+
|
| 33 |
+
Each line is one training candidate:
|
| 34 |
+
|
| 35 |
+
```json
|
| 36 |
+
{
|
| 37 |
+
"id": "sft-preview-0001",
|
| 38 |
+
"source": "objectverse-diary-mock-mvp",
|
| 39 |
+
"split": "preview",
|
| 40 |
+
"mode": "Cynical",
|
| 41 |
+
"object_description": "old white coffee mug on a developer desk",
|
| 42 |
+
"object_understanding": {},
|
| 43 |
+
"messages": [
|
| 44 |
+
{"role": "system", "content": "..."},
|
| 45 |
+
{"role": "user", "content": "..."},
|
| 46 |
+
{"role": "assistant", "content": "{\"persona\":{},\"diary\":{}}"}
|
| 47 |
+
]
|
| 48 |
+
}
|
| 49 |
+
```
|
| 50 |
+
|
| 51 |
+
The `assistant.content` field is JSON text so it can be used with chat-style SFT tools while preserving structured outputs.
|
| 52 |
+
|
| 53 |
+
## Generation Workflow
|
| 54 |
+
|
| 55 |
+
Preview:
|
| 56 |
+
|
| 57 |
+
```bash
|
| 58 |
+
.venv/bin/python -B scripts/generate_dataset.py --count 60
|
| 59 |
+
```
|
| 60 |
+
|
| 61 |
+
Full candidate pool later:
|
| 62 |
+
|
| 63 |
+
```bash
|
| 64 |
+
.venv/bin/python -B scripts/generate_dataset.py --count 300 --output data/train/objectverse_sft_candidates.jsonl
|
| 65 |
+
```
|
| 66 |
+
|
| 67 |
+
Manual curation should happen after generation. Do not publish the full candidate file until it has been reviewed.
|
| 68 |
+
|
| 69 |
+
## Curation Checklist
|
| 70 |
+
|
| 71 |
+
- Persona stays consistent with the object.
|
| 72 |
+
- Diary is short, vivid, and English-first.
|
| 73 |
+
- Chinese helper text is secondary.
|
| 74 |
+
- Output has a strange object archive feeling.
|
| 75 |
+
- No real person, email, token, address, credit code, or serial number remains.
|
| 76 |
+
- No commercial cloud AI model was used to create the sample.
|
| 77 |
+
- JSON parses cleanly.
|
| 78 |
+
|
| 79 |
+
## Publishing Notes
|
| 80 |
+
|
| 81 |
+
Before publishing to Hugging Face Datasets:
|
| 82 |
+
|
| 83 |
+
- create a dataset card
|
| 84 |
+
- document that mock preview rows are synthetic
|
| 85 |
+
- separate curated rows from raw candidates
|
| 86 |
+
- include license and privacy notes
|
| 87 |
+
- keep private images out of the repo
|
docs/EXTERNAL_SETUP.md
ADDED
|
@@ -0,0 +1,77 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# External Setup
|
| 2 |
+
|
| 3 |
+
## Purpose
|
| 4 |
+
|
| 5 |
+
This document lists the external actions needed after the local initial-stage build is ready.
|
| 6 |
+
|
| 7 |
+
These actions change external account state and should only be run after explicit confirmation.
|
| 8 |
+
|
| 9 |
+
## GitHub Repository
|
| 10 |
+
|
| 11 |
+
Suggested repository name:
|
| 12 |
+
|
| 13 |
+
```text
|
| 14 |
+
objectverse-diary
|
| 15 |
+
```
|
| 16 |
+
|
| 17 |
+
Suggested visibility:
|
| 18 |
+
|
| 19 |
+
```text
|
| 20 |
+
public
|
| 21 |
+
```
|
| 22 |
+
|
| 23 |
+
Suggested description:
|
| 24 |
+
|
| 25 |
+
```text
|
| 26 |
+
Small-model AI toy that turns everyday objects into secret diary characters.
|
| 27 |
+
```
|
| 28 |
+
|
| 29 |
+
Recommended manual command after confirmation:
|
| 30 |
+
|
| 31 |
+
```bash
|
| 32 |
+
gh repo create objectverse-diary --public --description "Small-model AI toy that turns everyday objects into secret diary characters." --source . --remote origin
|
| 33 |
+
```
|
| 34 |
+
|
| 35 |
+
Do not push until the user confirms the remote target and branch.
|
| 36 |
+
|
| 37 |
+
## Hugging Face Space
|
| 38 |
+
|
| 39 |
+
Suggested Space name:
|
| 40 |
+
|
| 41 |
+
```text
|
| 42 |
+
objectverse-diary
|
| 43 |
+
```
|
| 44 |
+
|
| 45 |
+
Suggested SDK:
|
| 46 |
+
|
| 47 |
+
```text
|
| 48 |
+
gradio
|
| 49 |
+
```
|
| 50 |
+
|
| 51 |
+
Required Space README header:
|
| 52 |
+
|
| 53 |
+
```yaml
|
| 54 |
+
---
|
| 55 |
+
title: Objectverse Diary
|
| 56 |
+
emoji: 🗝️
|
| 57 |
+
colorFrom: amber
|
| 58 |
+
colorTo: gray
|
| 59 |
+
sdk: gradio
|
| 60 |
+
app_file: app.py
|
| 61 |
+
pinned: false
|
| 62 |
+
---
|
| 63 |
+
```
|
| 64 |
+
|
| 65 |
+
Recommended setup before deployment:
|
| 66 |
+
|
| 67 |
+
- confirm target Hugging Face account or organization
|
| 68 |
+
- confirm public visibility
|
| 69 |
+
- confirm whether the Space should start with mock runtime
|
| 70 |
+
- confirm whether sample traces should be included in the first push
|
| 71 |
+
|
| 72 |
+
## Safety Notes
|
| 73 |
+
|
| 74 |
+
- Do not commit `.env`.
|
| 75 |
+
- Do not expose tokens, credit codes, private paths, or model credentials.
|
| 76 |
+
- Do not push large model files.
|
| 77 |
+
- Keep real model files under ignored `/models/` unless explicitly publishing them.
|
docs/FAILURES.md
ADDED
|
@@ -0,0 +1,84 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Failure Notes
|
| 2 |
+
|
| 3 |
+
## Purpose
|
| 4 |
+
|
| 5 |
+
This file tracks reproducible failures and fallback behavior for Objectverse Diary.
|
| 6 |
+
|
| 7 |
+
Use it for model/runtime/deployment/data issues, not for UI polish notes.
|
| 8 |
+
|
| 9 |
+
## Current Status
|
| 10 |
+
|
| 11 |
+
No real model or hosted Space failures have been observed yet because the current implementation uses deterministic mock runtimes.
|
| 12 |
+
|
| 13 |
+
Known non-blocking warning:
|
| 14 |
+
|
| 15 |
+
- Gradio emits deprecation warnings for upcoming 6.0 API changes during local tests. This does not break the current Gradio Blocks build and can be handled with the later UI/API polish pass.
|
| 16 |
+
|
| 17 |
+
## Failure Record Template
|
| 18 |
+
|
| 19 |
+
```markdown
|
| 20 |
+
## YYYY-MM-DD - Short Failure Name
|
| 21 |
+
|
| 22 |
+
- Area:
|
| 23 |
+
- Reproduction:
|
| 24 |
+
- Expected:
|
| 25 |
+
- Actual:
|
| 26 |
+
- Impact:
|
| 27 |
+
- Fallback used:
|
| 28 |
+
- Resolution:
|
| 29 |
+
- Evidence:
|
| 30 |
+
```
|
| 31 |
+
|
| 32 |
+
## Anticipated Failure Areas
|
| 33 |
+
|
| 34 |
+
### Vision Runtime
|
| 35 |
+
|
| 36 |
+
- MiniCPM-V or fallback VLM fails to load in local or Space environment.
|
| 37 |
+
- Uploaded image is unsupported, too large, or not an object photo.
|
| 38 |
+
- Model output is not valid JSON.
|
| 39 |
+
- Object recognition is too vague for persona generation.
|
| 40 |
+
|
| 41 |
+
Fallback:
|
| 42 |
+
|
| 43 |
+
- use manual object description
|
| 44 |
+
- use stable example flow
|
| 45 |
+
- record fallback marker in trace
|
| 46 |
+
|
| 47 |
+
### Text Runtime
|
| 48 |
+
|
| 49 |
+
- GGUF model path is missing.
|
| 50 |
+
- llama.cpp or llama-cpp-python import fails.
|
| 51 |
+
- Generation output does not match `PersonaEnvelope` or `DiaryEntry` schema.
|
| 52 |
+
- Chat response loses persona consistency.
|
| 53 |
+
|
| 54 |
+
Fallback:
|
| 55 |
+
|
| 56 |
+
- use deterministic mock text runtime
|
| 57 |
+
- repair JSON before schema validation
|
| 58 |
+
- record fallback marker in trace
|
| 59 |
+
|
| 60 |
+
### Dataset And Fine-Tuning
|
| 61 |
+
|
| 62 |
+
- Candidate samples contain private information.
|
| 63 |
+
- JSONL rows fail to parse.
|
| 64 |
+
- Curated data is too repetitive.
|
| 65 |
+
- Training run fails or adapter quality is weaker than mock prompts.
|
| 66 |
+
|
| 67 |
+
Fallback:
|
| 68 |
+
|
| 69 |
+
- keep mock preview data for schema validation only
|
| 70 |
+
- separate raw candidates from curated rows
|
| 71 |
+
- publish only reviewed examples
|
| 72 |
+
|
| 73 |
+
### Hugging Face Space
|
| 74 |
+
|
| 75 |
+
- Space cannot start because of missing packages.
|
| 76 |
+
- Model files are too large or not available.
|
| 77 |
+
- CPU runtime is too slow.
|
| 78 |
+
- Secrets or private paths appear in logs.
|
| 79 |
+
|
| 80 |
+
Fallback:
|
| 81 |
+
|
| 82 |
+
- launch with mock runtime first
|
| 83 |
+
- keep model files out of the repo
|
| 84 |
+
- document runtime mode clearly in README
|
docs/FIELD_NOTES.md
CHANGED
|
@@ -4,6 +4,10 @@ Working title:
|
|
| 4 |
|
| 5 |
`Building Objectverse Diary: A Small-Model AI Toy Where Everyday Objects Come Alive`
|
| 6 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
## Draft Outline
|
| 8 |
|
| 9 |
1. Why I built it
|
|
@@ -18,6 +22,58 @@ Working title:
|
|
| 18 |
10. What failed
|
| 19 |
11. What I would improve next
|
| 20 |
|
| 21 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
|
| 23 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
|
| 5 |
`Building Objectverse Diary: A Small-Model AI Toy Where Everyday Objects Come Alive`
|
| 6 |
|
| 7 |
+
## Status
|
| 8 |
+
|
| 9 |
+
Draft source. The final article should be written after real runtime and deployment evidence are available.
|
| 10 |
+
|
| 11 |
## Draft Outline
|
| 12 |
|
| 13 |
1. Why I built it
|
|
|
|
| 22 |
10. What failed
|
| 23 |
11. What I would improve next
|
| 24 |
|
| 25 |
+
## Draft Notes
|
| 26 |
+
|
| 27 |
+
### 1. Why I built it
|
| 28 |
+
|
| 29 |
+
Objectverse Diary starts from a simple joke: what if the objects around us were quietly keeping emotional records of our lives? The project turns an everyday photo into a hidden object persona, a secret diary entry, a short chat, and a shareable card.
|
| 30 |
+
|
| 31 |
+
### 2. Why Track 2
|
| 32 |
+
|
| 33 |
+
The experience is purely digital and depends on AI for its main interaction loop: visual understanding, persona invention, voice consistency, and diary generation. It fits the "An Adventure in Thousand Token Wood" track because the product is a strange AI-native toy rather than a conventional productivity workflow.
|
| 34 |
+
|
| 35 |
+
### 3. Why small models are enough
|
| 36 |
+
|
| 37 |
+
The task does not need a giant frontier model. It needs compact object recognition, strong style constraints, schema-following, and a reliable fallback path. The project is designed around a <= 32B total parameter budget.
|
| 38 |
+
|
| 39 |
+
### 4. Product design
|
| 40 |
+
|
| 41 |
+
The app is English-first and Chinese-second. The intended feeling is a mysterious archive of ordinary objects: museum labels, typewriter diary entries, and slightly uncanny object personalities.
|
| 42 |
+
|
| 43 |
+
### 5. Model architecture
|
| 44 |
+
|
| 45 |
+
The planned architecture keeps model calls behind `src/pipeline.py`: vision understanding, persona generation, diary generation, chat, share card rendering, and trace logging. The current mock runtime preserves this boundary before real model integration.
|
| 46 |
+
|
| 47 |
+
### 6. Gradio Off-Brand UI
|
| 48 |
+
|
| 49 |
+
The UI must remain Gradio, but should not feel like a default demo. This section should be completed after UI polish.
|
| 50 |
+
|
| 51 |
+
### 7. llama.cpp runtime
|
| 52 |
+
|
| 53 |
+
The text path is planned for GGUF plus llama.cpp or llama-cpp-python. This section should include the chosen model, parameter count, local command, Space behavior, and fallback notes.
|
| 54 |
+
|
| 55 |
+
### 8. Fine-tuning dataset
|
| 56 |
+
|
| 57 |
+
The dataset plan starts with deterministic preview JSONL, then moves to 200-500 candidate rows and 50 curated high-quality samples. Document provenance, curation, privacy checks, and any rejected examples.
|
| 58 |
+
|
| 59 |
+
### 9. Traces and reproducibility
|
| 60 |
+
|
| 61 |
+
The app saves anonymized traces. Current public samples are mock traces; real submission traces should include model runtime metadata and fallback markers.
|
| 62 |
+
|
| 63 |
+
### 10. What failed
|
| 64 |
+
|
| 65 |
+
Reserve this section for real integration failures: VLM loading, JSON validity, Space resource limits, and any dataset quality issues.
|
| 66 |
+
|
| 67 |
+
### 11. What I would improve next
|
| 68 |
+
|
| 69 |
+
Likely future work: better object memory, richer conversations, stronger card rendering, more curated styles, and a larger public evaluation set.
|
| 70 |
+
|
| 71 |
+
## Evidence To Add Later
|
| 72 |
|
| 73 |
+
- Hugging Face Space URL
|
| 74 |
+
- GitHub repository URL
|
| 75 |
+
- model repo URL
|
| 76 |
+
- dataset URL
|
| 77 |
+
- trace dataset URL
|
| 78 |
+
- demo video URL
|
| 79 |
+
- screenshots after UI polish
|
docs/INITIAL_STAGE_REPORT.md
ADDED
|
@@ -0,0 +1,100 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Initial Stage Report
|
| 2 |
+
|
| 3 |
+
## Scope
|
| 4 |
+
|
| 5 |
+
This report covers the local initial-stage build for Objectverse Diary.
|
| 6 |
+
|
| 7 |
+
Included:
|
| 8 |
+
|
| 9 |
+
- project skeleton
|
| 10 |
+
- Gradio mock MVP
|
| 11 |
+
- deterministic mock generation pipeline
|
| 12 |
+
- share card rendering
|
| 13 |
+
- trace logging and anonymization
|
| 14 |
+
- six stable example objects
|
| 15 |
+
- six public mock sample traces
|
| 16 |
+
- deterministic SFT preview data tooling
|
| 17 |
+
- public trace JSONL export tooling
|
| 18 |
+
- failure notes template for later model/runtime issues
|
| 19 |
+
- runtime configuration boundary
|
| 20 |
+
- local acceptance checks
|
| 21 |
+
|
| 22 |
+
Not included:
|
| 23 |
+
|
| 24 |
+
- creating the remote GitHub repository
|
| 25 |
+
- creating the Hugging Face Space
|
| 26 |
+
- real MiniCPM-V integration
|
| 27 |
+
- real llama.cpp / llama-cpp-python text runtime
|
| 28 |
+
- fine-tuning, dataset publishing, Field Notes, and demo video
|
| 29 |
+
|
| 30 |
+
Remote GitHub and Hugging Face actions require explicit confirmation because they change external state.
|
| 31 |
+
|
| 32 |
+
## Local Deliverables
|
| 33 |
+
|
| 34 |
+
| Deliverable | Evidence |
|
| 35 |
+
| --- | --- |
|
| 36 |
+
| Gradio app entrypoint | `app.py` |
|
| 37 |
+
| Shared generation pipeline | `src/pipeline.py` |
|
| 38 |
+
| Mock vision runner | `src/models/vision_runner.py` |
|
| 39 |
+
| Mock text runner | `src/models/llama_cpp_runner.py` |
|
| 40 |
+
| Pydantic schemas | `src/models/schema.py` |
|
| 41 |
+
| Share card renderer | `src/renderer/share_card.py` |
|
| 42 |
+
| Trace logger | `src/traces/logger.py` |
|
| 43 |
+
| Trace anonymizer | `src/traces/anonymizer.py` |
|
| 44 |
+
| Example objects | `src/examples.py` |
|
| 45 |
+
| Public mock traces | `data/traces/samples/` |
|
| 46 |
+
| SFT preview generator | `scripts/generate_dataset.py` |
|
| 47 |
+
| Public trace JSONL exporter | `scripts/export_traces.py` |
|
| 48 |
+
| Dataset plan | `docs/DATASET.md` |
|
| 49 |
+
| Failure notes | `docs/FAILURES.md` |
|
| 50 |
+
| Runtime boundary docs | `docs/RUNTIME.md` |
|
| 51 |
+
| Detailed plan | `docs/07-development-plan.md` |
|
| 52 |
+
| Acceptance script | `scripts/check_initial_stage.py` |
|
| 53 |
+
| Smoke tests | `tests/test_mock_mvp.py` |
|
| 54 |
+
|
| 55 |
+
## Acceptance Command
|
| 56 |
+
|
| 57 |
+
```bash
|
| 58 |
+
.venv/bin/python -B scripts/check_initial_stage.py
|
| 59 |
+
```
|
| 60 |
+
|
| 61 |
+
Expected output:
|
| 62 |
+
|
| 63 |
+
```text
|
| 64 |
+
[OK] required files exist
|
| 65 |
+
[OK] requirements avoid commercial model SDKs
|
| 66 |
+
[OK] runtime defaults to mock backends
|
| 67 |
+
[OK] pipeline generates and saves a trace
|
| 68 |
+
[OK] six sample traces validate
|
| 69 |
+
[OK] dataset preview tooling works
|
| 70 |
+
[OK] trace JSONL export works
|
| 71 |
+
[OK] Gradio Blocks app builds
|
| 72 |
+
```
|
| 73 |
+
|
| 74 |
+
## Test Command
|
| 75 |
+
|
| 76 |
+
```bash
|
| 77 |
+
.venv/bin/python -B -m unittest discover -s tests
|
| 78 |
+
```
|
| 79 |
+
|
| 80 |
+
Expected result:
|
| 81 |
+
|
| 82 |
+
```text
|
| 83 |
+
OK
|
| 84 |
+
```
|
| 85 |
+
|
| 86 |
+
## Current Limitations
|
| 87 |
+
|
| 88 |
+
- The app still uses mock model outputs.
|
| 89 |
+
- UI polish remains intentionally unfinished.
|
| 90 |
+
- Sample traces are mock traces, not real model traces.
|
| 91 |
+
- Remote repo and hosted Space are not created yet.
|
| 92 |
+
|
| 93 |
+
## Next Gate
|
| 94 |
+
|
| 95 |
+
Before moving to real model integration, confirm whether to create:
|
| 96 |
+
|
| 97 |
+
- GitHub repository
|
| 98 |
+
- Hugging Face Space
|
| 99 |
+
|
| 100 |
+
See `docs/EXTERNAL_SETUP.md`.
|
docs/MODEL_CARD.md
CHANGED
|
@@ -2,7 +2,9 @@
|
|
| 2 |
|
| 3 |
## Status
|
| 4 |
|
| 5 |
-
|
|
|
|
|
|
|
| 6 |
|
| 7 |
## Planned Components
|
| 8 |
|
|
@@ -10,6 +12,64 @@ Not started. No model has been selected, fine-tuned, converted, or published yet
|
|
| 10 |
- Text generation: fine-tuned small LLM.
|
| 11 |
- Runtime: llama.cpp / llama-cpp-python.
|
| 12 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
## Required Notes
|
| 14 |
|
| 15 |
- Total model parameter count must remain <= 32B.
|
|
|
|
| 2 |
|
| 3 |
## Status
|
| 4 |
|
| 5 |
+
Draft only. No model has been fine-tuned, converted, or published yet.
|
| 6 |
+
|
| 7 |
+
The app currently runs deterministic mock backends. This card is a working template for the later small-model runtime and LoRA adapter.
|
| 8 |
|
| 9 |
## Planned Components
|
| 10 |
|
|
|
|
| 12 |
- Text generation: fine-tuned small LLM.
|
| 13 |
- Runtime: llama.cpp / llama-cpp-python.
|
| 14 |
|
| 15 |
+
## Candidate Architecture
|
| 16 |
+
|
| 17 |
+
| Component | Candidate | Notes |
|
| 18 |
+
| --- | --- | --- |
|
| 19 |
+
| Vision | MiniCPM-V or lightweight VLM fallback | Must run without commercial API calls. |
|
| 20 |
+
| Text | small instruct model plus LoRA adapter | Final base model still pending. |
|
| 21 |
+
| Runtime | GGUF through llama.cpp / llama-cpp-python | Needed for Llama Champion evidence. |
|
| 22 |
+
| UI | Gradio Blocks | Required by the hackathon and project rules. |
|
| 23 |
+
|
| 24 |
+
## Parameter Budget
|
| 25 |
+
|
| 26 |
+
Total model parameters must remain <= 32B.
|
| 27 |
+
|
| 28 |
+
Record final numbers here before submission:
|
| 29 |
+
|
| 30 |
+
| Component | Model | Parameters | Counted Toward Total |
|
| 31 |
+
| --- | --- | ---: | --- |
|
| 32 |
+
| Vision | TBD | TBD | yes |
|
| 33 |
+
| Text base | TBD | TBD | yes |
|
| 34 |
+
| LoRA adapter | TBD | TBD | yes |
|
| 35 |
+
| Total | TBD | TBD | must be <= 32B |
|
| 36 |
+
|
| 37 |
+
## Intended Inputs And Outputs
|
| 38 |
+
|
| 39 |
+
Inputs:
|
| 40 |
+
|
| 41 |
+
- user-uploaded everyday object photo
|
| 42 |
+
- optional object description
|
| 43 |
+
- personality mode
|
| 44 |
+
|
| 45 |
+
Outputs:
|
| 46 |
+
|
| 47 |
+
- structured object understanding JSON
|
| 48 |
+
- hidden object persona JSON
|
| 49 |
+
- short English-first diary with Chinese helper text
|
| 50 |
+
- object chat response
|
| 51 |
+
- share card preview
|
| 52 |
+
- anonymized trace record
|
| 53 |
+
|
| 54 |
+
## Dataset Notes
|
| 55 |
+
|
| 56 |
+
Dataset planning lives in `docs/DATASET.md`.
|
| 57 |
+
|
| 58 |
+
Current preview data is deterministic and mock-generated. It should only be used for schema validation and workflow planning until real candidate samples are generated and curated.
|
| 59 |
+
|
| 60 |
+
## Safety And Privacy
|
| 61 |
+
|
| 62 |
+
- Do not use OpenAI, Anthropic, Gemini, Cohere, or other commercial model APIs.
|
| 63 |
+
- Do not publish private user photos or unconsented personal data.
|
| 64 |
+
- Do not include tokens, credit codes, emails, serial numbers, or credentials.
|
| 65 |
+
- Keep raw private traces out of public datasets.
|
| 66 |
+
|
| 67 |
+
## Fallback Behavior
|
| 68 |
+
|
| 69 |
+
- If VLM loading fails, use manual description and stable example flow.
|
| 70 |
+
- If llama.cpp loading fails, keep deterministic mock text fallback for demo safety.
|
| 71 |
+
- If model JSON is invalid, repair and validate before rendering.
|
| 72 |
+
|
| 73 |
## Required Notes
|
| 74 |
|
| 75 |
- Total model parameter count must remain <= 32B.
|
docs/README.md
CHANGED
|
@@ -16,4 +16,10 @@ This folder contains the planning source of truth for Objectverse Diary.
|
|
| 16 |
|
| 17 |
- `FIELD_NOTES.md`: future technical blog draft.
|
| 18 |
- `MODEL_CARD.md`: future model documentation.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
- `SUBMISSION_GUIDE.md`: final submission checklist.
|
|
|
|
| 16 |
|
| 17 |
- `FIELD_NOTES.md`: future technical blog draft.
|
| 18 |
- `MODEL_CARD.md`: future model documentation.
|
| 19 |
+
- `07-development-plan.md`: detailed development process plan from mock MVP to final submission.
|
| 20 |
+
- `RUNTIME.md`: current mock runtime configuration and future model boundary.
|
| 21 |
+
- `DATASET.md`: SFT preview schema, generation workflow, curation checklist, and publishing notes.
|
| 22 |
+
- `FAILURES.md`: failure record template and anticipated non-UI fallback cases.
|
| 23 |
+
- `INITIAL_STAGE_REPORT.md`: local initial-stage completion evidence and acceptance commands.
|
| 24 |
+
- `EXTERNAL_SETUP.md`: GitHub and Hugging Face Space setup notes requiring confirmation.
|
| 25 |
- `SUBMISSION_GUIDE.md`: final submission checklist.
|
docs/RUNTIME.md
ADDED
|
@@ -0,0 +1,44 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Runtime Configuration
|
| 2 |
+
|
| 3 |
+
## Current Runtime
|
| 4 |
+
|
| 5 |
+
The initial MVP uses deterministic mock runtime paths:
|
| 6 |
+
|
| 7 |
+
- `OBJECTVERSE_VISION_BACKEND=mock`
|
| 8 |
+
- `OBJECTVERSE_TEXT_BACKEND=mock`
|
| 9 |
+
|
| 10 |
+
This means:
|
| 11 |
+
|
| 12 |
+
- object understanding is generated by `src/models/vision_runner.py`
|
| 13 |
+
- persona, diary, and chat are generated by `src/models/llama_cpp_runner.py`
|
| 14 |
+
- traces mark `mock-runtime` in the `fallbacks` field
|
| 15 |
+
|
| 16 |
+
No commercial cloud AI APIs are used.
|
| 17 |
+
|
| 18 |
+
## Environment Variables
|
| 19 |
+
|
| 20 |
+
```bash
|
| 21 |
+
OBJECTVERSE_VISION_BACKEND=mock
|
| 22 |
+
OBJECTVERSE_TEXT_BACKEND=mock
|
| 23 |
+
VISION_MODEL_ID=
|
| 24 |
+
TEXT_MODEL_PATH=
|
| 25 |
+
TRACE_OUTPUT_DIR=data/traces
|
| 26 |
+
```
|
| 27 |
+
|
| 28 |
+
## Future Runtime Boundary
|
| 29 |
+
|
| 30 |
+
The next implementation phase should keep the same pipeline boundary:
|
| 31 |
+
|
| 32 |
+
1. UI calls `src/pipeline.py`.
|
| 33 |
+
2. `src/pipeline.py` calls the configured vision and text runners.
|
| 34 |
+
3. runners return validated Pydantic schemas.
|
| 35 |
+
4. trace logging records backend metadata and fallback markers.
|
| 36 |
+
|
| 37 |
+
Do not move model calls into `src/ui/layout.py`.
|
| 38 |
+
|
| 39 |
+
## Fallback Rules
|
| 40 |
+
|
| 41 |
+
- VLM unavailable: use manual description and mock/example gallery path.
|
| 42 |
+
- llama.cpp unavailable: use mock text generation path.
|
| 43 |
+
- invalid model JSON: repair and validate before rendering.
|
| 44 |
+
- private input: anonymize trace text before saving public traces.
|
docs/SUBMISSION_GUIDE.md
CHANGED
|
@@ -2,15 +2,23 @@
|
|
| 2 |
|
| 3 |
## Required Package
|
| 4 |
|
| 5 |
-
- [ ] Hugging Face Space URL
|
| 6 |
-
- [ ] GitHub Repository URL
|
| 7 |
-
- [ ] Demo Video URL
|
| 8 |
-
- [ ] Social Media Post URL
|
| 9 |
-
- [ ] Fine-tuned Model URL
|
| 10 |
-
- [ ] Dataset URL
|
| 11 |
-
- [ ] Trace Dataset URL
|
| 12 |
-
- [ ] Field Notes Blog URL
|
| 13 |
-
- [
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
|
| 15 |
## Final Checks
|
| 16 |
|
|
|
|
| 2 |
|
| 3 |
## Required Package
|
| 4 |
|
| 5 |
+
- [ ] Hugging Face Space URL: pending external setup
|
| 6 |
+
- [ ] GitHub Repository URL: pending external setup
|
| 7 |
+
- [ ] Demo Video URL: pending recording
|
| 8 |
+
- [ ] Social Media Post URL: pending final copy
|
| 9 |
+
- [ ] Fine-tuned Model URL: pending model training
|
| 10 |
+
- [ ] Dataset URL: pending curation and publishing
|
| 11 |
+
- [ ] Trace Dataset URL: local mock JSONL export ready, publishing pending
|
| 12 |
+
- [ ] Field Notes Blog URL: draft source in `docs/FIELD_NOTES.md`
|
| 13 |
+
- [x] Short project description: available in README
|
| 14 |
+
|
| 15 |
+
## Local Evidence Ready
|
| 16 |
+
|
| 17 |
+
- Initial mock MVP report: `docs/INITIAL_STAGE_REPORT.md`
|
| 18 |
+
- Runtime boundary: `docs/RUNTIME.md`
|
| 19 |
+
- Dataset plan and preview workflow: `docs/DATASET.md`
|
| 20 |
+
- External setup checklist: `docs/EXTERNAL_SETUP.md`
|
| 21 |
+
- Public mock traces: `data/traces/samples/`
|
| 22 |
|
| 23 |
## Final Checks
|
| 24 |
|
pyproject.toml
CHANGED
|
@@ -3,8 +3,11 @@ name = "objectverse-diary"
|
|
| 3 |
version = "0.0.0"
|
| 4 |
description = "A small-model AI toy that turns everyday objects into secret diary characters."
|
| 5 |
requires-python = ">=3.10"
|
| 6 |
-
dependencies = [
|
|
|
|
|
|
|
|
|
|
| 7 |
|
| 8 |
[tool.objectverse-diary]
|
| 9 |
-
status = "
|
| 10 |
-
implementation = "
|
|
|
|
| 3 |
version = "0.0.0"
|
| 4 |
description = "A small-model AI toy that turns everyday objects into secret diary characters."
|
| 5 |
requires-python = ">=3.10"
|
| 6 |
+
dependencies = [
|
| 7 |
+
"gradio>=4.44,<6",
|
| 8 |
+
"pydantic>=2.7,<3",
|
| 9 |
+
]
|
| 10 |
|
| 11 |
[tool.objectverse-diary]
|
| 12 |
+
status = "initial-mock-mvp"
|
| 13 |
+
implementation = "mock-runtime"
|
requirements.txt
CHANGED
|
@@ -1,2 +1,2 @@
|
|
| 1 |
-
|
| 2 |
-
|
|
|
|
| 1 |
+
gradio>=4.44,<6
|
| 2 |
+
pydantic>=2.7,<3
|
scripts/README.md
CHANGED
|
@@ -1,13 +1,18 @@
|
|
| 1 |
# Scripts
|
| 2 |
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
|
| 5 |
Expected files during implementation:
|
| 6 |
|
| 7 |
-
- `generate_dataset.py`
|
| 8 |
- `finetune_lora.py`
|
| 9 |
- `convert_to_gguf.sh`
|
| 10 |
- `run_llama_cpp.sh`
|
| 11 |
-
- `export_traces.py`
|
| 12 |
|
| 13 |
-
Current status:
|
|
|
|
| 1 |
# Scripts
|
| 2 |
|
| 3 |
+
Automation scripts for dataset generation, fine-tuning, GGUF conversion, llama.cpp runtime, and trace export.
|
| 4 |
+
|
| 5 |
+
Implemented initial scripts:
|
| 6 |
+
|
| 7 |
+
- `check_initial_stage.py`: verifies required files, runtime defaults, sample traces, pipeline, and Gradio build.
|
| 8 |
+
- `generate_sample_traces.py`: creates six stable public mock traces under `data/traces/samples/`.
|
| 9 |
+
- `generate_dataset.py`: creates deterministic SFT preview JSONL for schema and curation planning.
|
| 10 |
+
- `export_traces.py`: exports validated public sample traces to JSONL for dataset-style publishing.
|
| 11 |
|
| 12 |
Expected files during implementation:
|
| 13 |
|
|
|
|
| 14 |
- `finetune_lora.py`
|
| 15 |
- `convert_to_gguf.sh`
|
| 16 |
- `run_llama_cpp.sh`
|
|
|
|
| 17 |
|
| 18 |
+
Current status: mock trace generation, trace JSONL export, and SFT preview generation are implemented. Real model, fine-tuning, and GGUF conversion scripts are not implemented yet.
|
scripts/__init__.py
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
"""Project scripts package."""
|
scripts/check_initial_stage.py
ADDED
|
@@ -0,0 +1,194 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Acceptance checks for the initial mock MVP stage."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import json
|
| 6 |
+
import os
|
| 7 |
+
import sys
|
| 8 |
+
import tempfile
|
| 9 |
+
from pathlib import Path
|
| 10 |
+
|
| 11 |
+
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
| 12 |
+
if str(PROJECT_ROOT) not in sys.path:
|
| 13 |
+
sys.path.insert(0, str(PROJECT_ROOT))
|
| 14 |
+
|
| 15 |
+
from src.config import get_runtime_settings, runtime_status
|
| 16 |
+
from src.examples import EXAMPLE_OBJECTS
|
| 17 |
+
from src.models.schema import TraceRecord
|
| 18 |
+
from src.pipeline import generate_object_diary
|
| 19 |
+
from scripts.export_traces import export_trace_jsonl
|
| 20 |
+
from scripts.generate_dataset import build_sft_records, write_sft_jsonl
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
REQUIRED_FILES = [
|
| 24 |
+
"app.py",
|
| 25 |
+
"README.md",
|
| 26 |
+
"requirements.txt",
|
| 27 |
+
"pyproject.toml",
|
| 28 |
+
"docs/07-development-plan.md",
|
| 29 |
+
"docs/EXTERNAL_SETUP.md",
|
| 30 |
+
"docs/FAILURES.md",
|
| 31 |
+
"docs/INITIAL_STAGE_REPORT.md",
|
| 32 |
+
"docs/RUNTIME.md",
|
| 33 |
+
"docs/DATASET.md",
|
| 34 |
+
"src/pipeline.py",
|
| 35 |
+
"src/models/schema.py",
|
| 36 |
+
"src/ui/layout.py",
|
| 37 |
+
"scripts/export_traces.py",
|
| 38 |
+
"scripts/generate_dataset.py",
|
| 39 |
+
"scripts/generate_sample_traces.py",
|
| 40 |
+
"tests/test_mock_mvp.py",
|
| 41 |
+
"tests/test_dataset_tooling.py",
|
| 42 |
+
]
|
| 43 |
+
|
| 44 |
+
FORBIDDEN_REQUIREMENTS = {
|
| 45 |
+
"openai",
|
| 46 |
+
"anthropic",
|
| 47 |
+
"google-generativeai",
|
| 48 |
+
"cohere",
|
| 49 |
+
}
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+
def run_checks() -> list[str]:
|
| 53 |
+
results: list[str] = []
|
| 54 |
+
_ensure_local_tempdir()
|
| 55 |
+
|
| 56 |
+
_check_required_files()
|
| 57 |
+
results.append("required files exist")
|
| 58 |
+
|
| 59 |
+
_check_requirements()
|
| 60 |
+
results.append("requirements avoid commercial model SDKs")
|
| 61 |
+
|
| 62 |
+
_check_runtime_defaults()
|
| 63 |
+
results.append("runtime defaults to mock backends")
|
| 64 |
+
|
| 65 |
+
_check_pipeline()
|
| 66 |
+
results.append("pipeline generates and saves a trace")
|
| 67 |
+
|
| 68 |
+
_check_sample_traces()
|
| 69 |
+
results.append("six sample traces validate")
|
| 70 |
+
|
| 71 |
+
_check_dataset_preview()
|
| 72 |
+
results.append("dataset preview tooling works")
|
| 73 |
+
|
| 74 |
+
_check_trace_export()
|
| 75 |
+
results.append("trace JSONL export works")
|
| 76 |
+
|
| 77 |
+
_check_gradio_build()
|
| 78 |
+
results.append("Gradio Blocks app builds")
|
| 79 |
+
|
| 80 |
+
return results
|
| 81 |
+
|
| 82 |
+
|
| 83 |
+
def _check_required_files() -> None:
|
| 84 |
+
missing = [path for path in REQUIRED_FILES if not (PROJECT_ROOT / path).exists()]
|
| 85 |
+
if missing:
|
| 86 |
+
raise RuntimeError(f"Missing required files: {', '.join(missing)}")
|
| 87 |
+
|
| 88 |
+
|
| 89 |
+
def _check_requirements() -> None:
|
| 90 |
+
requirements = (PROJECT_ROOT / "requirements.txt").read_text(encoding="utf-8").lower()
|
| 91 |
+
forbidden = sorted(package for package in FORBIDDEN_REQUIREMENTS if package in requirements)
|
| 92 |
+
if forbidden:
|
| 93 |
+
raise RuntimeError(f"Forbidden commercial model SDK dependencies: {', '.join(forbidden)}")
|
| 94 |
+
|
| 95 |
+
|
| 96 |
+
def _check_runtime_defaults() -> None:
|
| 97 |
+
settings = get_runtime_settings({})
|
| 98 |
+
status = runtime_status(settings)
|
| 99 |
+
if settings.vision_backend != "mock" or settings.text_backend != "mock":
|
| 100 |
+
raise RuntimeError("Runtime defaults must stay mock for the initial stage.")
|
| 101 |
+
if status["runtime"] != "no llama.cpp model connected yet":
|
| 102 |
+
raise RuntimeError("Unexpected initial runtime status.")
|
| 103 |
+
|
| 104 |
+
|
| 105 |
+
def _check_pipeline() -> None:
|
| 106 |
+
with tempfile.TemporaryDirectory() as tmp_dir:
|
| 107 |
+
result = generate_object_diary(
|
| 108 |
+
None,
|
| 109 |
+
"old white coffee mug on a developer desk",
|
| 110 |
+
"Cynical",
|
| 111 |
+
trace_dir=Path(tmp_dir),
|
| 112 |
+
)
|
| 113 |
+
trace_path = Path(result.trace_path)
|
| 114 |
+
if not trace_path.exists():
|
| 115 |
+
raise RuntimeError("Pipeline did not save trace output.")
|
| 116 |
+
TraceRecord.model_validate(json.loads(trace_path.read_text(encoding="utf-8")))
|
| 117 |
+
|
| 118 |
+
|
| 119 |
+
def _check_sample_traces() -> None:
|
| 120 |
+
sample_dir = PROJECT_ROOT / "data/traces/samples"
|
| 121 |
+
paths = sorted(sample_dir.glob("*.json"))
|
| 122 |
+
if len(paths) != len(EXAMPLE_OBJECTS):
|
| 123 |
+
raise RuntimeError(f"Expected {len(EXAMPLE_OBJECTS)} sample traces, found {len(paths)}.")
|
| 124 |
+
|
| 125 |
+
trace_ids = set()
|
| 126 |
+
for path in paths:
|
| 127 |
+
trace = TraceRecord.model_validate(json.loads(path.read_text(encoding="utf-8")))
|
| 128 |
+
trace_ids.add(trace.trace_id)
|
| 129 |
+
if trace.fallbacks != ["mock-runtime"]:
|
| 130 |
+
raise RuntimeError(f"Unexpected fallbacks in {path}.")
|
| 131 |
+
|
| 132 |
+
expected_ids = {f"sample-{index:02d}" for index in range(1, len(EXAMPLE_OBJECTS) + 1)}
|
| 133 |
+
if trace_ids != expected_ids:
|
| 134 |
+
raise RuntimeError(f"Unexpected sample trace ids: {sorted(trace_ids)}")
|
| 135 |
+
|
| 136 |
+
|
| 137 |
+
def _check_dataset_preview() -> None:
|
| 138 |
+
records = build_sft_records(6)
|
| 139 |
+
if len(records) != 6:
|
| 140 |
+
raise RuntimeError("Dataset preview did not generate six records.")
|
| 141 |
+
|
| 142 |
+
with tempfile.TemporaryDirectory() as tmp_dir:
|
| 143 |
+
output_path = Path(tmp_dir) / "preview.jsonl"
|
| 144 |
+
write_sft_jsonl(records, output_path)
|
| 145 |
+
lines = output_path.read_text(encoding="utf-8").splitlines()
|
| 146 |
+
|
| 147 |
+
if len(lines) != 6:
|
| 148 |
+
raise RuntimeError("Dataset preview JSONL did not write six lines.")
|
| 149 |
+
|
| 150 |
+
for line in lines:
|
| 151 |
+
payload = json.loads(line)
|
| 152 |
+
if payload.get("split") != "preview":
|
| 153 |
+
raise RuntimeError("Dataset preview record has unexpected split.")
|
| 154 |
+
messages = payload.get("messages", [])
|
| 155 |
+
if len(messages) != 3:
|
| 156 |
+
raise RuntimeError("Dataset preview record must have three chat messages.")
|
| 157 |
+
|
| 158 |
+
|
| 159 |
+
def _check_trace_export() -> None:
|
| 160 |
+
sample_dir = PROJECT_ROOT / "data/traces/samples"
|
| 161 |
+
with tempfile.TemporaryDirectory() as tmp_dir:
|
| 162 |
+
output_path = Path(tmp_dir) / "public_traces.jsonl"
|
| 163 |
+
count = export_trace_jsonl(sample_dir, output_path)
|
| 164 |
+
lines = output_path.read_text(encoding="utf-8").splitlines()
|
| 165 |
+
|
| 166 |
+
if count != len(EXAMPLE_OBJECTS) or len(lines) != len(EXAMPLE_OBJECTS):
|
| 167 |
+
raise RuntimeError("Trace JSONL export count does not match sample traces.")
|
| 168 |
+
|
| 169 |
+
for line in lines:
|
| 170 |
+
TraceRecord.model_validate(json.loads(line))
|
| 171 |
+
|
| 172 |
+
|
| 173 |
+
def _check_gradio_build() -> None:
|
| 174 |
+
from src.ui.layout import build_app
|
| 175 |
+
|
| 176 |
+
demo = build_app()
|
| 177 |
+
if type(demo).__name__ != "Blocks":
|
| 178 |
+
raise RuntimeError("Gradio app did not build a Blocks instance.")
|
| 179 |
+
|
| 180 |
+
|
| 181 |
+
def _ensure_local_tempdir() -> None:
|
| 182 |
+
temp_dir = PROJECT_ROOT / ".tmp"
|
| 183 |
+
temp_dir.mkdir(parents=True, exist_ok=True)
|
| 184 |
+
os.environ.setdefault("TMPDIR", str(temp_dir))
|
| 185 |
+
tempfile.tempdir = str(temp_dir)
|
| 186 |
+
|
| 187 |
+
|
| 188 |
+
def main() -> None:
|
| 189 |
+
for result in run_checks():
|
| 190 |
+
print(f"[OK] {result}")
|
| 191 |
+
|
| 192 |
+
|
| 193 |
+
if __name__ == "__main__":
|
| 194 |
+
main()
|
scripts/export_traces.py
ADDED
|
@@ -0,0 +1,61 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Export validated public traces to a JSONL file."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import argparse
|
| 6 |
+
import json
|
| 7 |
+
import sys
|
| 8 |
+
from pathlib import Path
|
| 9 |
+
|
| 10 |
+
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
| 11 |
+
if str(PROJECT_ROOT) not in sys.path:
|
| 12 |
+
sys.path.insert(0, str(PROJECT_ROOT))
|
| 13 |
+
|
| 14 |
+
from src.models.schema import TraceRecord
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
DEFAULT_INPUT_DIR = Path("data/traces/samples")
|
| 18 |
+
DEFAULT_OUTPUT_PATH = Path("data/traces/samples/objectverse_public_mock_traces.jsonl")
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
def collect_traces(input_dir: Path = DEFAULT_INPUT_DIR) -> list[TraceRecord]:
|
| 22 |
+
paths = sorted(input_dir.glob("*.json"))
|
| 23 |
+
if not paths:
|
| 24 |
+
raise FileNotFoundError(f"No trace JSON files found under {input_dir}")
|
| 25 |
+
|
| 26 |
+
traces: list[TraceRecord] = []
|
| 27 |
+
for path in paths:
|
| 28 |
+
payload = json.loads(path.read_text(encoding="utf-8"))
|
| 29 |
+
traces.append(TraceRecord.model_validate(payload))
|
| 30 |
+
return traces
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
def export_trace_jsonl(
|
| 34 |
+
input_dir: Path = DEFAULT_INPUT_DIR,
|
| 35 |
+
output_path: Path = DEFAULT_OUTPUT_PATH,
|
| 36 |
+
) -> int:
|
| 37 |
+
traces = collect_traces(input_dir)
|
| 38 |
+
output_path.parent.mkdir(parents=True, exist_ok=True)
|
| 39 |
+
lines = [
|
| 40 |
+
json.dumps(trace.model_dump(mode="json"), ensure_ascii=False, sort_keys=True)
|
| 41 |
+
for trace in traces
|
| 42 |
+
]
|
| 43 |
+
output_path.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
| 44 |
+
return len(traces)
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
def _parse_args() -> argparse.Namespace:
|
| 48 |
+
parser = argparse.ArgumentParser(description=__doc__)
|
| 49 |
+
parser.add_argument("--input-dir", type=Path, default=DEFAULT_INPUT_DIR)
|
| 50 |
+
parser.add_argument("--output", type=Path, default=DEFAULT_OUTPUT_PATH)
|
| 51 |
+
return parser.parse_args()
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
def main() -> None:
|
| 55 |
+
args = _parse_args()
|
| 56 |
+
count = export_trace_jsonl(args.input_dir, args.output)
|
| 57 |
+
print(f"exported {count} traces to {args.output}")
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
if __name__ == "__main__":
|
| 61 |
+
main()
|
scripts/generate_dataset.py
ADDED
|
@@ -0,0 +1,135 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Generate deterministic SFT preview data for Objectverse Diary."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import argparse
|
| 6 |
+
import json
|
| 7 |
+
import sys
|
| 8 |
+
from collections.abc import Mapping, Sequence
|
| 9 |
+
from pathlib import Path
|
| 10 |
+
|
| 11 |
+
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
| 12 |
+
if str(PROJECT_ROOT) not in sys.path:
|
| 13 |
+
sys.path.insert(0, str(PROJECT_ROOT))
|
| 14 |
+
|
| 15 |
+
from src.config import PERSONALITY_MODES
|
| 16 |
+
from src.examples import EXAMPLE_OBJECTS
|
| 17 |
+
from src.pipeline import generate_object_diary
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
DEFAULT_OUTPUT_PATH = Path("data/train/objectverse_sft_preview.jsonl")
|
| 21 |
+
DEFAULT_COUNT = 60
|
| 22 |
+
|
| 23 |
+
SYSTEM_PROMPT = (
|
| 24 |
+
"You are Objectverse Diary, an English-first small-model assistant. "
|
| 25 |
+
"Given structured object understanding and a requested personality mode, "
|
| 26 |
+
"return JSON containing a consistent hidden object persona and a short diary."
|
| 27 |
+
)
|
| 28 |
+
|
| 29 |
+
SCENE_VARIANTS = [
|
| 30 |
+
"after a long day of being ignored",
|
| 31 |
+
"beside half-finished notes",
|
| 32 |
+
"under warm desk light",
|
| 33 |
+
"near a quiet morning window",
|
| 34 |
+
"during a suspiciously ordinary afternoon",
|
| 35 |
+
"while humans rush past without noticing",
|
| 36 |
+
"after surviving another minor household crisis",
|
| 37 |
+
"beside a charging cable and old receipts",
|
| 38 |
+
"in the corner of a room that knows too much",
|
| 39 |
+
"before anyone remembers to clean it",
|
| 40 |
+
]
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
def build_sft_records(count: int = DEFAULT_COUNT) -> list[dict[str, object]]:
|
| 44 |
+
if count < 1:
|
| 45 |
+
raise ValueError("count must be at least 1")
|
| 46 |
+
|
| 47 |
+
records: list[dict[str, object]] = []
|
| 48 |
+
for index in range(count):
|
| 49 |
+
example = EXAMPLE_OBJECTS[index % len(EXAMPLE_OBJECTS)]
|
| 50 |
+
mode = _mode_for_index(example, index)
|
| 51 |
+
description = _description_for_index(example, index)
|
| 52 |
+
result = generate_object_diary(
|
| 53 |
+
image_path=None,
|
| 54 |
+
description=description,
|
| 55 |
+
mode=mode,
|
| 56 |
+
save=False,
|
| 57 |
+
trace_id=f"sft-preview-{index + 1:04d}",
|
| 58 |
+
)
|
| 59 |
+
assistant_payload = {
|
| 60 |
+
"persona": result.persona.persona.model_dump(mode="json"),
|
| 61 |
+
"diary": result.diary.model_dump(mode="json"),
|
| 62 |
+
}
|
| 63 |
+
record = {
|
| 64 |
+
"id": f"sft-preview-{index + 1:04d}",
|
| 65 |
+
"source": "objectverse-diary-mock-mvp",
|
| 66 |
+
"split": "preview",
|
| 67 |
+
"mode": mode,
|
| 68 |
+
"object_description": description,
|
| 69 |
+
"object_understanding": result.object_understanding.model_dump(mode="json"),
|
| 70 |
+
"messages": [
|
| 71 |
+
{"role": "system", "content": SYSTEM_PROMPT},
|
| 72 |
+
{
|
| 73 |
+
"role": "user",
|
| 74 |
+
"content": _user_prompt(
|
| 75 |
+
result.object_understanding.model_dump(mode="json"),
|
| 76 |
+
mode,
|
| 77 |
+
),
|
| 78 |
+
},
|
| 79 |
+
{
|
| 80 |
+
"role": "assistant",
|
| 81 |
+
"content": json.dumps(assistant_payload, ensure_ascii=False),
|
| 82 |
+
},
|
| 83 |
+
],
|
| 84 |
+
}
|
| 85 |
+
records.append(record)
|
| 86 |
+
|
| 87 |
+
return records
|
| 88 |
+
|
| 89 |
+
|
| 90 |
+
def write_sft_jsonl(records: Sequence[Mapping[str, object]], output_path: Path) -> Path:
|
| 91 |
+
output_path.parent.mkdir(parents=True, exist_ok=True)
|
| 92 |
+
lines = [json.dumps(record, ensure_ascii=False, sort_keys=True) for record in records]
|
| 93 |
+
output_path.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
| 94 |
+
return output_path
|
| 95 |
+
|
| 96 |
+
|
| 97 |
+
def generate_dataset(output_path: Path = DEFAULT_OUTPUT_PATH, count: int = DEFAULT_COUNT) -> Path:
|
| 98 |
+
return write_sft_jsonl(build_sft_records(count), output_path)
|
| 99 |
+
|
| 100 |
+
|
| 101 |
+
def _mode_for_index(example: Mapping[str, str], index: int) -> str:
|
| 102 |
+
base_mode = example["mode"]
|
| 103 |
+
base_index = PERSONALITY_MODES.index(base_mode) if base_mode in PERSONALITY_MODES else 0
|
| 104 |
+
return PERSONALITY_MODES[(base_index + index) % len(PERSONALITY_MODES)]
|
| 105 |
+
|
| 106 |
+
|
| 107 |
+
def _description_for_index(example: Mapping[str, str], index: int) -> str:
|
| 108 |
+
scene = SCENE_VARIANTS[index % len(SCENE_VARIANTS)]
|
| 109 |
+
return f"{example['description']}, {scene}"
|
| 110 |
+
|
| 111 |
+
|
| 112 |
+
def _user_prompt(object_understanding: Mapping[str, object], mode: str) -> str:
|
| 113 |
+
payload = json.dumps(object_understanding, ensure_ascii=False, sort_keys=True)
|
| 114 |
+
return (
|
| 115 |
+
f"Personality mode: {mode}\n"
|
| 116 |
+
f"Object understanding JSON: {payload}\n"
|
| 117 |
+
"Return JSON with keys persona and diary."
|
| 118 |
+
)
|
| 119 |
+
|
| 120 |
+
|
| 121 |
+
def _parse_args() -> argparse.Namespace:
|
| 122 |
+
parser = argparse.ArgumentParser(description=__doc__)
|
| 123 |
+
parser.add_argument("--count", type=int, default=DEFAULT_COUNT)
|
| 124 |
+
parser.add_argument("--output", type=Path, default=DEFAULT_OUTPUT_PATH)
|
| 125 |
+
return parser.parse_args()
|
| 126 |
+
|
| 127 |
+
|
| 128 |
+
def main() -> None:
|
| 129 |
+
args = _parse_args()
|
| 130 |
+
output_path = generate_dataset(args.output, args.count)
|
| 131 |
+
print(f"wrote {args.count} SFT preview records to {output_path}")
|
| 132 |
+
|
| 133 |
+
|
| 134 |
+
if __name__ == "__main__":
|
| 135 |
+
main()
|
scripts/generate_sample_traces.py
ADDED
|
@@ -0,0 +1,55 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Generate stable public sample traces for the mock MVP."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import json
|
| 6 |
+
import sys
|
| 7 |
+
from datetime import datetime, timezone
|
| 8 |
+
from pathlib import Path
|
| 9 |
+
|
| 10 |
+
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
| 11 |
+
if str(PROJECT_ROOT) not in sys.path:
|
| 12 |
+
sys.path.insert(0, str(PROJECT_ROOT))
|
| 13 |
+
|
| 14 |
+
from src.examples import EXAMPLE_OBJECTS
|
| 15 |
+
from src.pipeline import generate_object_diary
|
| 16 |
+
|
| 17 |
+
|
| 18 |
+
DEFAULT_OUTPUT_DIR = Path("data/traces/samples")
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
def generate_sample_traces(output_dir: Path = DEFAULT_OUTPUT_DIR) -> list[Path]:
|
| 22 |
+
output_dir.mkdir(parents=True, exist_ok=True)
|
| 23 |
+
written_paths: list[Path] = []
|
| 24 |
+
|
| 25 |
+
for index, example in enumerate(EXAMPLE_OBJECTS, start=1):
|
| 26 |
+
description = example["description"]
|
| 27 |
+
mode = example["mode"]
|
| 28 |
+
result = generate_object_diary(
|
| 29 |
+
image_path=None,
|
| 30 |
+
description=description,
|
| 31 |
+
mode=mode,
|
| 32 |
+
save=False,
|
| 33 |
+
trace_id=f"sample-{index:02d}",
|
| 34 |
+
created_at=datetime(2026, 6, 5, 0, index, tzinfo=timezone.utc),
|
| 35 |
+
)
|
| 36 |
+
|
| 37 |
+
filename = f"sample-{index:02d}-{_slug(example['label'])}.json"
|
| 38 |
+
path = output_dir / filename
|
| 39 |
+
path.write_text(
|
| 40 |
+
json.dumps(result.trace.model_dump(mode="json"), ensure_ascii=False, indent=2),
|
| 41 |
+
encoding="utf-8",
|
| 42 |
+
)
|
| 43 |
+
written_paths.append(path)
|
| 44 |
+
|
| 45 |
+
return written_paths
|
| 46 |
+
|
| 47 |
+
|
| 48 |
+
def _slug(value: str) -> str:
|
| 49 |
+
slug = value.split("/", maxsplit=1)[0].strip().lower()
|
| 50 |
+
return "".join(char if char.isalnum() else "-" for char in slug).strip("-")
|
| 51 |
+
|
| 52 |
+
|
| 53 |
+
if __name__ == "__main__":
|
| 54 |
+
for generated_path in generate_sample_traces():
|
| 55 |
+
print(generated_path)
|
src/README.md
CHANGED
|
@@ -2,11 +2,12 @@
|
|
| 2 |
|
| 3 |
This directory is reserved for application source code.
|
| 4 |
|
| 5 |
-
Current status:
|
| 6 |
|
| 7 |
## Planned Areas
|
| 8 |
|
| 9 |
- `ui/`: Gradio layout, CSS, and copy.
|
|
|
|
| 10 |
- `models/`: vision runner, llama.cpp text runner, and schemas.
|
| 11 |
- `prompts/`: object understanding, persona generation, and diary generation prompts.
|
| 12 |
- `renderer/`: share card and HTML rendering.
|
|
|
|
| 2 |
|
| 3 |
This directory is reserved for application source code.
|
| 4 |
|
| 5 |
+
Current status: initial mock MVP. Real model runtimes are not connected yet.
|
| 6 |
|
| 7 |
## Planned Areas
|
| 8 |
|
| 9 |
- `ui/`: Gradio layout, CSS, and copy.
|
| 10 |
+
- `examples.py`: stable mock examples for the initial gallery.
|
| 11 |
- `models/`: vision runner, llama.cpp text runner, and schemas.
|
| 12 |
- `prompts/`: object understanding, persona generation, and diary generation prompts.
|
| 13 |
- `renderer/`: share card and HTML rendering.
|
src/__init__.py
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
"""Objectverse Diary source package."""
|
src/config.py
ADDED
|
@@ -0,0 +1,66 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Shared configuration for the initial mock MVP."""
|
| 2 |
+
|
| 3 |
+
import os
|
| 4 |
+
from collections.abc import Mapping
|
| 5 |
+
from dataclasses import dataclass
|
| 6 |
+
from pathlib import Path
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
APP_TITLE = "Objectverse Diary"
|
| 10 |
+
APP_SUBTITLE = "Every object has a secret life. / 每个物品都有秘密人生。"
|
| 11 |
+
APP_VERSION = "0.1.0-mock"
|
| 12 |
+
|
| 13 |
+
PERSONALITY_MODES = [
|
| 14 |
+
"Cynical",
|
| 15 |
+
"Dramatic",
|
| 16 |
+
"Lonely",
|
| 17 |
+
"Philosopher",
|
| 18 |
+
"Romantic",
|
| 19 |
+
]
|
| 20 |
+
|
| 21 |
+
DEFAULT_MODE = "Cynical"
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
@dataclass(frozen=True)
|
| 25 |
+
class RuntimeSettings:
|
| 26 |
+
vision_backend: str
|
| 27 |
+
text_backend: str
|
| 28 |
+
text_model_path: str
|
| 29 |
+
vision_model_id: str
|
| 30 |
+
trace_output_dir: Path
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
def get_runtime_settings(environ: Mapping[str, str] | None = None) -> RuntimeSettings:
|
| 34 |
+
env = os.environ if environ is None else environ
|
| 35 |
+
return RuntimeSettings(
|
| 36 |
+
vision_backend=env.get("OBJECTVERSE_VISION_BACKEND", "mock"),
|
| 37 |
+
text_backend=env.get("OBJECTVERSE_TEXT_BACKEND", "mock"),
|
| 38 |
+
text_model_path=env.get("TEXT_MODEL_PATH", ""),
|
| 39 |
+
vision_model_id=env.get("VISION_MODEL_ID", ""),
|
| 40 |
+
trace_output_dir=Path(env.get("TRACE_OUTPUT_DIR", "data/traces")),
|
| 41 |
+
)
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
def runtime_status(settings: RuntimeSettings | None = None) -> dict[str, str]:
|
| 45 |
+
current = settings or get_runtime_settings()
|
| 46 |
+
vision = (
|
| 47 |
+
"mock object understanding"
|
| 48 |
+
if current.vision_backend == "mock"
|
| 49 |
+
else f"{current.vision_backend} object understanding"
|
| 50 |
+
)
|
| 51 |
+
text = (
|
| 52 |
+
"mock persona and diary generation"
|
| 53 |
+
if current.text_backend == "mock"
|
| 54 |
+
else f"{current.text_backend} persona and diary generation"
|
| 55 |
+
)
|
| 56 |
+
runtime = (
|
| 57 |
+
"no llama.cpp model connected yet"
|
| 58 |
+
if current.text_backend == "mock"
|
| 59 |
+
else f"text model path: {current.text_model_path or '[not configured]'}"
|
| 60 |
+
)
|
| 61 |
+
return {"vision": vision, "text": text, "runtime": runtime}
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
SETTINGS = get_runtime_settings()
|
| 65 |
+
TRACE_DIR = SETTINGS.trace_output_dir
|
| 66 |
+
MODEL_RUNTIME_STATUS = runtime_status(SETTINGS)
|
src/examples.py
ADDED
|
@@ -0,0 +1,38 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Stable example objects for the initial mock MVP."""
|
| 2 |
+
|
| 3 |
+
EXAMPLE_OBJECTS = [
|
| 4 |
+
{
|
| 5 |
+
"label": "Coffee mug / 咖啡杯",
|
| 6 |
+
"description": "old white coffee mug on a developer desk",
|
| 7 |
+
"mode": "Cynical",
|
| 8 |
+
},
|
| 9 |
+
{
|
| 10 |
+
"label": "Mechanical keyboard / 机械键盘",
|
| 11 |
+
"description": "dusty black mechanical keyboard in an office",
|
| 12 |
+
"mode": "Philosopher",
|
| 13 |
+
},
|
| 14 |
+
{
|
| 15 |
+
"label": "Running shoe / 跑鞋",
|
| 16 |
+
"description": "worn running shoe near the bedroom door",
|
| 17 |
+
"mode": "Lonely",
|
| 18 |
+
},
|
| 19 |
+
{
|
| 20 |
+
"label": "Desk lamp / 台灯",
|
| 21 |
+
"description": "metal desk lamp over late night notes",
|
| 22 |
+
"mode": "Dramatic",
|
| 23 |
+
},
|
| 24 |
+
{
|
| 25 |
+
"label": "Water bottle / 水瓶",
|
| 26 |
+
"description": "clear plastic water bottle on a kitchen counter",
|
| 27 |
+
"mode": "Romantic",
|
| 28 |
+
},
|
| 29 |
+
{
|
| 30 |
+
"label": "Notebook / 笔记本",
|
| 31 |
+
"description": "old notebook of abandoned project ideas",
|
| 32 |
+
"mode": "Cynical",
|
| 33 |
+
},
|
| 34 |
+
]
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
def gradio_examples() -> list[list[str]]:
|
| 38 |
+
return [[item["description"]] for item in EXAMPLE_OBJECTS]
|
src/models/README.md
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Models
|
| 2 |
+
|
| 3 |
+
Planned model runtime layer.
|
| 4 |
+
|
| 5 |
+
Implemented initial files:
|
| 6 |
+
|
| 7 |
+
- `vision_runner.py`
|
| 8 |
+
- `llama_cpp_runner.py`
|
| 9 |
+
- `schema.py`
|
| 10 |
+
|
| 11 |
+
Current status: deterministic mock object understanding and text generation.
|
src/models/llama_cpp_runner.py
ADDED
|
@@ -0,0 +1,110 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Mock text generation layer reserved for future llama.cpp integration."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from src.models.schema import DiaryEntry, ObjectUnderstanding, Persona, PersonaEnvelope
|
| 6 |
+
|
| 7 |
+
|
| 8 |
+
MODE_PROFILES = {
|
| 9 |
+
"Cynical": {
|
| 10 |
+
"mood": "tired but sarcastic",
|
| 11 |
+
"fear": "being replaced by a newer object with worse opinions",
|
| 12 |
+
"voice": "dry",
|
| 13 |
+
},
|
| 14 |
+
"Dramatic": {
|
| 15 |
+
"mood": "theatrical and wounded",
|
| 16 |
+
"fear": "being forgotten before the final act",
|
| 17 |
+
"voice": "operatic",
|
| 18 |
+
},
|
| 19 |
+
"Lonely": {
|
| 20 |
+
"mood": "softly abandoned",
|
| 21 |
+
"fear": "becoming invisible in plain sight",
|
| 22 |
+
"voice": "quiet",
|
| 23 |
+
},
|
| 24 |
+
"Philosopher": {
|
| 25 |
+
"mood": "curious and needlessly profound",
|
| 26 |
+
"fear": "discovering that usefulness is not meaning",
|
| 27 |
+
"voice": "reflective",
|
| 28 |
+
},
|
| 29 |
+
"Romantic": {
|
| 30 |
+
"mood": "hopelessly sentimental",
|
| 31 |
+
"fear": "loving a human who only sees storage capacity",
|
| 32 |
+
"voice": "wistful",
|
| 33 |
+
},
|
| 34 |
+
}
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
def generate_persona(object_understanding: ObjectUnderstanding, mode: str) -> PersonaEnvelope:
|
| 38 |
+
object_name = object_understanding.object.name
|
| 39 |
+
profile = MODE_PROFILES.get(mode, MODE_PROFILES["Cynical"])
|
| 40 |
+
character_name = _character_name(object_name, mode)
|
| 41 |
+
|
| 42 |
+
persona = Persona(
|
| 43 |
+
object_name=object_name,
|
| 44 |
+
character_name=character_name,
|
| 45 |
+
mood=profile["mood"],
|
| 46 |
+
secret_fear=profile["fear"],
|
| 47 |
+
core_memory=f"survived many quiet hours as a {object_name} while humans called it normal life",
|
| 48 |
+
complaint=f"I am not just a {object_name}. I am an unpaid witness with excellent recall.",
|
| 49 |
+
tags=_tags_for_mode(mode),
|
| 50 |
+
)
|
| 51 |
+
return PersonaEnvelope(persona=persona)
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
def generate_diary(persona: PersonaEnvelope, mode: str) -> DiaryEntry:
|
| 55 |
+
p = persona.persona
|
| 56 |
+
day_number = 417 + len(p.object_name)
|
| 57 |
+
|
| 58 |
+
english = (
|
| 59 |
+
f"They touched me again today with the confidence of someone who has never asked "
|
| 60 |
+
f"a {p.object_name} for consent. I remained still, because that is my contract with gravity. "
|
| 61 |
+
f"My mood is {p.mood}, my secret fear is {p.secret_fear}, and my only comfort is knowing "
|
| 62 |
+
"I have outlived at least three urgent plans."
|
| 63 |
+
)
|
| 64 |
+
chinese = (
|
| 65 |
+
f"今天他们又理所当然地碰了我,好像一个 {p.object_name} 不会有边界感。"
|
| 66 |
+
f"我保持沉默,因为这大概是我和重力签下的合同。我的情绪是 {p.mood},"
|
| 67 |
+
f"秘密恐惧是 {p.secret_fear}。至少,我已经熬过了好几个所谓紧急计划。"
|
| 68 |
+
)
|
| 69 |
+
|
| 70 |
+
return DiaryEntry(
|
| 71 |
+
title=f"Secret Diary - Day {day_number}",
|
| 72 |
+
english=english,
|
| 73 |
+
chinese=chinese,
|
| 74 |
+
)
|
| 75 |
+
|
| 76 |
+
|
| 77 |
+
def reply_as_object(persona_data: dict, message: str) -> str:
|
| 78 |
+
persona = persona_data.get("persona", {})
|
| 79 |
+
character_name = persona.get("character_name", "The Object")
|
| 80 |
+
object_name = persona.get("object_name", "object")
|
| 81 |
+
mood = persona.get("mood", "suspicious")
|
| 82 |
+
complaint = persona.get("complaint", "I have seen enough.")
|
| 83 |
+
clean_message = message.strip() or "..."
|
| 84 |
+
|
| 85 |
+
return (
|
| 86 |
+
f"{character_name}: You ask me about '{clean_message}', as if a {object_name} "
|
| 87 |
+
f"with a {mood} mood has unlimited office hours. {complaint}"
|
| 88 |
+
)
|
| 89 |
+
|
| 90 |
+
|
| 91 |
+
def _character_name(object_name: str, mode: str) -> str:
|
| 92 |
+
compact = "".join(part.capitalize() for part in object_name.split()[:2])
|
| 93 |
+
suffix = {
|
| 94 |
+
"Cynical": "worth",
|
| 95 |
+
"Dramatic": "von Sigh",
|
| 96 |
+
"Lonely": "Afterlight",
|
| 97 |
+
"Philosopher": "the Questioning",
|
| 98 |
+
"Romantic": "de Moon",
|
| 99 |
+
}.get(mode, "worth")
|
| 100 |
+
return f"{compact} {suffix}".strip()
|
| 101 |
+
|
| 102 |
+
|
| 103 |
+
def _tags_for_mode(mode: str) -> list[str]:
|
| 104 |
+
return {
|
| 105 |
+
"Cynical": ["desk survivor", "burnt optimism", "quiet judgment"],
|
| 106 |
+
"Dramatic": ["tragic prop", "grand entrance", "minor catastrophe"],
|
| 107 |
+
"Lonely": ["forgotten corner", "soft echo", "dust companion"],
|
| 108 |
+
"Philosopher": ["tiny ontology", "useful doubt", "meaning crisis"],
|
| 109 |
+
"Romantic": ["tender witness", "hopeless glow", "secret devotion"],
|
| 110 |
+
}.get(mode, ["odd witness", "secret life", "object soul"])
|
src/models/schema.py
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Pydantic schemas for mock MVP outputs."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from datetime import datetime
|
| 6 |
+
from typing import Any
|
| 7 |
+
|
| 8 |
+
from pydantic import BaseModel, Field
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
class ObjectInfo(BaseModel):
|
| 12 |
+
name: str
|
| 13 |
+
visible_features: list[str] = Field(default_factory=list)
|
| 14 |
+
likely_context: str
|
| 15 |
+
confidence: float = Field(ge=0, le=1)
|
| 16 |
+
|
| 17 |
+
|
| 18 |
+
class ObjectUnderstanding(BaseModel):
|
| 19 |
+
object: ObjectInfo
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
class Persona(BaseModel):
|
| 23 |
+
object_name: str
|
| 24 |
+
character_name: str
|
| 25 |
+
mood: str
|
| 26 |
+
secret_fear: str
|
| 27 |
+
core_memory: str
|
| 28 |
+
complaint: str
|
| 29 |
+
tags: list[str] = Field(min_length=3, max_length=3)
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
class PersonaEnvelope(BaseModel):
|
| 33 |
+
persona: Persona
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
class DiaryEntry(BaseModel):
|
| 37 |
+
title: str
|
| 38 |
+
english: str
|
| 39 |
+
chinese: str
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
class TraceRecord(BaseModel):
|
| 43 |
+
trace_id: str
|
| 44 |
+
created_at: datetime
|
| 45 |
+
mode: str
|
| 46 |
+
input: dict[str, Any]
|
| 47 |
+
object_understanding: ObjectUnderstanding
|
| 48 |
+
persona: PersonaEnvelope
|
| 49 |
+
diary: DiaryEntry
|
| 50 |
+
model_runtime: dict[str, str]
|
| 51 |
+
fallbacks: list[str] = Field(default_factory=list)
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
class GenerationResult(BaseModel):
|
| 55 |
+
object_understanding: ObjectUnderstanding
|
| 56 |
+
persona: PersonaEnvelope
|
| 57 |
+
diary: DiaryEntry
|
| 58 |
+
trace: TraceRecord
|
| 59 |
+
trace_path: str
|
src/models/vision_runner.py
ADDED
|
@@ -0,0 +1,80 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Mock object understanding for the initial MVP."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from pathlib import Path
|
| 6 |
+
|
| 7 |
+
from src.models.schema import ObjectInfo, ObjectUnderstanding
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
KNOWN_OBJECTS = {
|
| 11 |
+
"mug": "coffee mug",
|
| 12 |
+
"cup": "coffee mug",
|
| 13 |
+
"keyboard": "keyboard",
|
| 14 |
+
"shoe": "shoe",
|
| 15 |
+
"book": "book",
|
| 16 |
+
"phone": "phone",
|
| 17 |
+
"lamp": "desk lamp",
|
| 18 |
+
"bottle": "water bottle",
|
| 19 |
+
"bag": "bag",
|
| 20 |
+
}
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
def understand_object(image_path: str | None, description: str) -> ObjectUnderstanding:
|
| 24 |
+
"""Return deterministic mock object understanding until VLM integration starts."""
|
| 25 |
+
clean_description = description.strip()
|
| 26 |
+
object_name = _infer_object_name(clean_description, image_path)
|
| 27 |
+
features = _infer_features(clean_description, image_path)
|
| 28 |
+
|
| 29 |
+
return ObjectUnderstanding(
|
| 30 |
+
object=ObjectInfo(
|
| 31 |
+
name=object_name,
|
| 32 |
+
visible_features=features,
|
| 33 |
+
likely_context=_infer_context(clean_description),
|
| 34 |
+
confidence=0.42 if clean_description else 0.32,
|
| 35 |
+
)
|
| 36 |
+
)
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
def _infer_object_name(description: str, image_path: str | None) -> str:
|
| 40 |
+
lowered = description.lower()
|
| 41 |
+
for keyword, name in KNOWN_OBJECTS.items():
|
| 42 |
+
if keyword in lowered:
|
| 43 |
+
return name
|
| 44 |
+
|
| 45 |
+
if image_path:
|
| 46 |
+
stem = Path(image_path).stem.replace("_", " ").replace("-", " ").strip()
|
| 47 |
+
if stem:
|
| 48 |
+
return stem[:40]
|
| 49 |
+
|
| 50 |
+
return "mysterious everyday object"
|
| 51 |
+
|
| 52 |
+
|
| 53 |
+
def _infer_features(description: str, image_path: str | None) -> list[str]:
|
| 54 |
+
features: list[str] = []
|
| 55 |
+
lowered = description.lower()
|
| 56 |
+
|
| 57 |
+
for word in ["old", "new", "cracked", "white", "black", "dusty", "metal", "ceramic", "plastic"]:
|
| 58 |
+
if word in lowered:
|
| 59 |
+
features.append(word)
|
| 60 |
+
|
| 61 |
+
if image_path:
|
| 62 |
+
features.append("uploaded photo provided")
|
| 63 |
+
|
| 64 |
+
if description:
|
| 65 |
+
features.append("user-supplied description")
|
| 66 |
+
|
| 67 |
+
return features[:5] or ["ordinary surface", "unknown material", "quietly suspicious"]
|
| 68 |
+
|
| 69 |
+
|
| 70 |
+
def _infer_context(description: str) -> str:
|
| 71 |
+
lowered = description.lower()
|
| 72 |
+
if "desk" in lowered:
|
| 73 |
+
return "developer desk"
|
| 74 |
+
if "kitchen" in lowered:
|
| 75 |
+
return "kitchen counter"
|
| 76 |
+
if "bedroom" in lowered:
|
| 77 |
+
return "bedroom shelf"
|
| 78 |
+
if "office" in lowered:
|
| 79 |
+
return "office corner"
|
| 80 |
+
return "everyday human environment"
|
src/pipeline.py
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Generation pipeline shared by UI, scripts, and tests."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from datetime import datetime
|
| 6 |
+
from pathlib import Path
|
| 7 |
+
|
| 8 |
+
from src.config import TRACE_DIR
|
| 9 |
+
from src.models.llama_cpp_runner import generate_diary, generate_persona
|
| 10 |
+
from src.models.schema import GenerationResult
|
| 11 |
+
from src.models.vision_runner import understand_object
|
| 12 |
+
from src.traces.logger import build_trace, save_trace
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
def generate_object_diary(
|
| 16 |
+
image_path: str | None,
|
| 17 |
+
description: str,
|
| 18 |
+
mode: str,
|
| 19 |
+
*,
|
| 20 |
+
trace_dir: Path = TRACE_DIR,
|
| 21 |
+
save: bool = True,
|
| 22 |
+
trace_id: str | None = None,
|
| 23 |
+
created_at: datetime | None = None,
|
| 24 |
+
) -> GenerationResult:
|
| 25 |
+
object_understanding = understand_object(image_path, description)
|
| 26 |
+
persona = generate_persona(object_understanding, mode)
|
| 27 |
+
diary = generate_diary(persona, mode)
|
| 28 |
+
trace = build_trace(
|
| 29 |
+
image_path=image_path,
|
| 30 |
+
description=description,
|
| 31 |
+
mode=mode,
|
| 32 |
+
object_understanding=object_understanding,
|
| 33 |
+
persona=persona,
|
| 34 |
+
diary=diary,
|
| 35 |
+
trace_id=trace_id,
|
| 36 |
+
created_at=created_at,
|
| 37 |
+
)
|
| 38 |
+
trace_path = save_trace(trace, trace_dir) if save else ""
|
| 39 |
+
|
| 40 |
+
return GenerationResult(
|
| 41 |
+
object_understanding=object_understanding,
|
| 42 |
+
persona=persona,
|
| 43 |
+
diary=diary,
|
| 44 |
+
trace=trace,
|
| 45 |
+
trace_path=trace_path,
|
| 46 |
+
)
|
| 47 |
+
|
| 48 |
+
|
| 49 |
+
def format_diary_markdown(title: str, english: str, chinese: str) -> str:
|
| 50 |
+
return f"## {title}\n\n{english}\n\n---\n\n**中文辅助**\n\n{chinese}"
|
src/prompts/README.md
CHANGED
|
@@ -2,10 +2,10 @@
|
|
| 2 |
|
| 3 |
Planned prompt templates.
|
| 4 |
|
| 5 |
-
|
| 6 |
|
| 7 |
- `object_understanding.py`
|
| 8 |
- `persona_generation.py`
|
| 9 |
- `diary_generation.py`
|
| 10 |
|
| 11 |
-
Current status:
|
|
|
|
| 2 |
|
| 3 |
Planned prompt templates.
|
| 4 |
|
| 5 |
+
Implemented placeholder files:
|
| 6 |
|
| 7 |
- `object_understanding.py`
|
| 8 |
- `persona_generation.py`
|
| 9 |
- `diary_generation.py`
|
| 10 |
|
| 11 |
+
Current status: prompt placeholders only. Real model prompts are not finalized.
|
src/prompts/diary_generation.py
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Prompt placeholder for future secret diary generation."""
|
| 2 |
+
|
| 3 |
+
DIARY_GENERATION_PROMPT = """
|
| 4 |
+
Write a short secret diary entry in English first, with Chinese helper translation.
|
| 5 |
+
Keep the object persona consistent.
|
| 6 |
+
""".strip()
|
src/prompts/object_understanding.py
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Prompt placeholder for future VLM object understanding."""
|
| 2 |
+
|
| 3 |
+
OBJECT_UNDERSTANDING_PROMPT = """
|
| 4 |
+
Identify the everyday object, visible features, likely context, and confidence.
|
| 5 |
+
Return structured JSON only.
|
| 6 |
+
""".strip()
|
src/prompts/persona_generation.py
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Prompt placeholder for future persona generation."""
|
| 2 |
+
|
| 3 |
+
PERSONA_GENERATION_PROMPT = """
|
| 4 |
+
Create a hidden first-person object persona with name, mood, backstory,
|
| 5 |
+
complaint, secret fear, core memory, and exactly three tags.
|
| 6 |
+
Return structured JSON only.
|
| 7 |
+
""".strip()
|
src/renderer/README.md
CHANGED
|
@@ -2,9 +2,9 @@
|
|
| 2 |
|
| 3 |
Planned rendering layer for diary views and share cards.
|
| 4 |
|
| 5 |
-
|
| 6 |
|
| 7 |
- `share_card.py`
|
| 8 |
- `html_templates.py`
|
| 9 |
|
| 10 |
-
Current status:
|
|
|
|
| 2 |
|
| 3 |
Planned rendering layer for diary views and share cards.
|
| 4 |
|
| 5 |
+
Implemented initial files:
|
| 6 |
|
| 7 |
- `share_card.py`
|
| 8 |
- `html_templates.py`
|
| 9 |
|
| 10 |
+
Current status: share card HTML renderer for mock MVP.
|
src/renderer/html_templates.py
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Small HTML templates for the initial MVP."""
|
| 2 |
+
|
| 3 |
+
CARD_WRAPPER_CLASS = "objectverse-card"
|