docs/07-development-plan.md

Objectverse Diary — Detailed Development Plan

Purpose

This document turns the day-by-day schedule into an execution plan for completing Objectverse Diary from the initial mock MVP to hackathon submission.

The plan is intentionally staged. Each phase has a clear goal, implementation scope, verification method, and exit criteria.

Current Baseline

As of 2026-06-06, the project has:

• initialized project structure
• root README and AGENTS instructions
• .codex/skills/ project guidance
• initial Gradio mock MVP
• six stable example objects
• mock object understanding JSON
• mock persona and diary generation
• object chat with mock persona consistency
• share card HTML preview
• anonymized trace JSON saving under data/traces/
• six stable public mock traces under data/traces/samples/
• deterministic SFT preview generator and dataset plan
• public trace JSONL exporter
• failure notes template
• scripts/generate_sample_traces.py
• scripts/generate_dataset.py
• scripts/export_traces.py
• stdlib unittest smoke tests for the mock MVP
• runtime configuration boundary documented in docs/RUNTIME.md
• initial-stage acceptance script at scripts/check_initial_stage.py
• Hugging Face Space created at build-small-hackathon/ObjectverseDiary
• optional MiniCPM-V 2.6 vision backend wiring with mock fallback
• optional llama.cpp / llama-cpp-python text runtime wiring through TEXT_MODEL_PATH
• hosted Space VLM validation tooling in scripts/check_space_vlm.py
• pending Space VLM report template in docs/SPACE_VLM_REPORT.md

Not yet done:

• GitHub repo sync / public submission confirmation
• hosted Space MiniCPM-V validation with real public images
• real GGUF selection and local TEXT_MODEL_PATH smoke test
• real curated dataset
• LoRA fine-tuning
• model card completion
• Field Notes article
• demo video
• final submission package

Phase 1 — Initial Mock MVP

Goal: validate the product loop before model integration.

Scope:

• Build app.py entrypoint.
• Build Gradio Blocks UI.
• Support image upload and optional text description.
• Add personality mode selection.
• Add six stable example objects.
• Produce deterministic mock object JSON.
• Produce deterministic mock persona JSON.
• Produce English-first diary with Chinese helper translation.
• Support chat replies using the generated persona.
• Render a share card preview.
• Save anonymized trace JSON.

Exit criteria:

• python app.py starts a Gradio app.
• User can complete Upload -> Generate -> Diary -> Share Card -> Trace.
• Trace JSON is saved locally.
• No commercial model APIs are used.

Verification:

• Import smoke test for app.
• Direct function smoke test for generation flow.
• unittest smoke tests for mock flow, chat, share card, trace save, and anonymization.
• Sample trace generation script writes six stable trace files.
• Dataset preview script writes deterministic mock SFT preview JSONL.
• Trace export script writes validated public trace JSONL.
• scripts/check_initial_stage.py validates required initial-stage artifacts.
• Manual Gradio preview.

Phase 2 — UI Polish And Example Gallery

Goal: make the app feel like an object archive instead of a default Gradio demo.

Scope:

• Refine src/ui/styles.css.
• Reference the design images under UI 参考/ for visual direction.
• Keep content, interaction flow, language hierarchy, and feature scope aligned with docs/.
• Keep six stable example objects visible in the UI.
• Add clearer empty states and error states.
• Improve mobile layout.
• Keep UI English-first and Chinese-second.

Exit criteria:

• 1366px desktop layout is usable.
• Mobile-width layout is usable.
• Example gallery can reproduce stable outputs.
• Share card is readable and screenshot-friendly.

Verification:

• Manual browser preview.
• Screenshot review at desktop and mobile widths.
• Example generation for at least six objects.

Phase 3 — Vision Understanding

Goal: replace mock object recognition with a real VLM path while preserving fallback behavior.

Status: local wiring complete; hosted ZeroGPU validation reaches the app but falls back to mock vision.

Scope:

• Add MiniCPM-V or lightweight VLM runner in src/models/vision_runner.py.
• Keep manual description fallback.
• Validate object understanding JSON with schemas.
• Add JSON repair or retry behavior.
• Cache stable examples for demo reliability.

Exit criteria:

• Uploaded object photos produce structured object JSON.
• Cups, keyboards, and shoes are recognized with useful visible features.
• Fallback path works when VLM fails.

Verification:

• Run local sample image checks.
• Confirm schema validation.
• Confirm fallback trace markers.
• Run scripts/check_space_vlm.py --configure-space --hardware zero-a10g --rollback-to-mock after external-state confirmation.
• Inspect Space runtime logs or add non-secret diagnostics before rerunning, because the 2026-06-08 hosted validation returned vision-fallback-to-mock for mug, keyboard, and shoe.

Phase 4 — Text Runtime With llama.cpp

Goal: make persona, diary, and chat generation use a small local text model runtime.

Status: optional runtime wiring complete; published LoRA v2 Q4_K_M GGUF passed local llama.cpp smoke. Hosted Space text runtime validation is still pending.

Scope:

• Add llama.cpp / llama-cpp-python runner. Completed as optional runtime wiring.
• Add model path configuration. Completed through TEXT_MODEL_PATH.
• Preserve src/pipeline.py as the UI-independent generation boundary.
• Implement persona generation.
• Implement diary generation.
• Implement chat continuation.
• Keep deterministic mock fallback for demos.

Exit criteria:

• Text generation can run through llama.cpp or documented local fallback.
• README documents runtime path and published GGUF selection.
• Trace records include runtime metadata.

Verification:

• Local runtime smoke test with objectverse-diary-qwen15b-lora-v2-q4_k_m.gguf.
• JSON schema validation.
• Compare at least three object generations for persona consistency.

Phase 5 — Dataset And Fine-Tuning Preparation

Goal: prepare Well-Tuned badge evidence.

Status: mock SFT preview complete; real candidate generation waits for verified model paths.

Scope:

• Use scripts/generate_dataset.py to validate the SFT schema locally.
• Generate 200-500 object-persona candidate samples after real model path is available.
• Manually curate at least 50 high-quality examples.
• Define SFT schema.
• Prepare dataset preview.
• Draft dataset privacy notes.

Exit criteria:

• Mock SFT preview exists and parses as JSONL.
• Training dataset is structured and inspectable.
• Public examples contain no private data.
• Dataset card draft exists.

Verification:

• Validate JSONL format.
• Spot-check curated samples.
• Confirm no obvious sensitive data.

Phase 6 — LoRA Fine-Tuning And Model Card

Goal: publish a small fine-tuned model or adapter that can be linked in submission materials.

Scope:

• Run LoRA training with Modal or local resources.
• Export adapter or merged model.
• Convert to GGUF if needed.
• Publish HF model repo.
• Complete docs/MODEL_CARD.md.

Exit criteria:

• Fine-tuned model repo exists.
• Model parameter count is documented.
• Runtime instructions are documented.

Verification:

• Run inference on sample prompts.
• Confirm HF model links.
• Confirm no private credit codes or tokens are present.

Phase 7 — Public Traces And Reproducibility

Goal: satisfy Sharing is Caring expectations.

Scope:

• Produce at least six public traces.
• Keep data/traces/samples/ in sync with the six example objects.
• Export public traces to JSONL for dataset-style sharing.
• Add prompt templates.
• Add dataset preview.
• Document failures and fallbacks.
• Ensure trace anonymization.

Exit criteria:

• Public trace files are readable JSON.
• Trace docs explain how outputs were produced.
• Example gallery aligns with public traces.

Verification:

• Validate trace JSON.
• Inspect anonymization.
• Confirm README links.

Phase 8 — Hugging Face Space Deployment

Goal: deploy the app in the required Gradio format.

Status: Space exists and mock app has been verified; MiniCPM-V L4 validation is pending.

Scope:

• Create Hugging Face Space. Completed.
• Add Space README YAML header. Completed.
• Confirm app_file: app.py. Completed.
• Configure model paths and fallback mode. Mock-safe default complete; VLM variables pending real validation.
• Check runtime resource constraints. Pending L4 validation.

Exit criteria:

• Space opens publicly or under the official hackathon organization.
• App can generate at least stable demo examples.
• README includes deployment and model notes.

Verification:

• Launch on HF Space. Completed for mock-safe runtime.
• Run demo flow in hosted environment.
• Run Space VLM validation for mug, keyboard, and shoe.
• Check logs for missing secrets or path errors.

Phase 9 — Field Notes And Demo Video

Goal: complete narrative submission assets.

Scope:

• Write Field Notes article.
• Record demo video under 2 minutes.
• Prepare social post.
• Add badge evidence to README.

Exit criteria:

• Field Notes URL exists.
• Demo video URL exists.
• Social post URL exists.
• Submission package has all required links.

Verification:

• Watch final video.
• Check all URLs.
• Confirm README and submission guide are aligned.

Phase 10 — Final Submission Audit

Goal: reduce avoidable submission risk.

Checklist:

• Space under official organization.
• Demo video ready.
• Social post ready.
• README complete.
• Model parameter count documented.
• No commercial cloud AI API.
• Fine-tuned model linked.
• Dataset linked.
• Traces linked.
• Field Notes linked.
• UI English-first and Chinese-second.
• Submit before June 15, 2026.

Risk Register

Risk	Impact	Mitigation
VLM deployment is slow	Blocks real image understanding	Keep manual description and example gallery fallback
llama.cpp setup is unstable	Blocks Llama Champion badge	Use text mock fallback for demo while isolating runtime work
Fine-tuning takes too long	Weakens Well-Tuned badge	Prepare small curated dataset and prompt-tuned fallback
HF Space resources are limited	Demo may be slow	Cache examples and support CPU fallback
Trace contains private data	Submission/privacy risk	Anonymize trace input and avoid raw private images

Working Rule

Do not start a later phase by breaking an earlier verified flow. The mock MVP should remain usable while real model paths are added behind clear fallbacks.