docs/MODEL_CARD.md

Model Card

Status

Stable local baseline plus live MiniCPM-V Space vision, one published text LoRA v2 adapter, and one published Q4_K_M GGUF. The public Gradio Space defaults to real MiniCPM-V object understanding with deterministic mock text; the GGUF has passed local llama.cpp smoke, but it has not been switched into the live Space runtime.

Local development defaults to deterministic mock backends. The hosted Space runs MiniCPM-V 2.6 vision on ZeroGPU with a hidden non-secret probe for diagnostics. Text generation has optional llama.cpp wiring for an externally configured GGUF model via TEXT_MODEL_PATH, but the live Space keeps text on the mock runtime for this release. A Modal LoRA v2 run completed, the adapter is published at https://huggingface.co/qqyule/objectverse-diary-qwen15b-lora, and the merged Q4_K_M GGUF is published in the same repo.

Hosted MiniCPM-V validation passed after adding an HF_TOKEN Space secret with access to the gated openbmb/MiniCPM-V-2_6 model. The validation uses public mug, keyboard, and shoe images on ZeroGPU, while text generation intentionally remains mock. See docs/SPACE_VLM_REPORT.md.

Planned Components

• Vision understanding: MiniCPM-V or lightweight fallback VLM.
• Text generation: fine-tuned small LLM.
• Runtime: llama.cpp / llama-cpp-python.

Candidate Architecture

Component	Candidate	Notes
Vision	openbmb/MiniCPM-V-2_6 or mock fallback	Live Space uses MiniCPM-V on ZeroGPU; local runtime can still default to mock.
Text	deterministic mock text by default; published Qwen/Qwen2.5-1.5B-Instruct LoRA v2 Q4_K_M GGUF for local runtime	Adapter and GGUF published; Space text runtime remains mock for the live vision release.
Runtime	optional GGUF through llama.cpp / llama-cpp-python	Wired with mock fallback; local GGUF smoke passed on 2026-06-08.
UI	Gradio Blocks	Required by the hackathon and project rules.

Parameter Budget

Total model parameters must remain <= 32B.

Record final numbers here before submission:

Component	Model	Parameters	Counted Toward Total
Vision	MiniCPM-V 2.6 optional path	~8B	yes, when enabled
Text base	Stable baseline mock text	0	no model parameters
Optional text base	Qwen/Qwen2.5-1.5B-Instruct	~1.5B	yes, when enabled
Published LoRA v2 GGUF	qqyule/objectverse-diary-qwen15b-lora / objectverse-diary-qwen15b-lora-v2-q4_k_m.gguf	~1.5B base, quantized file	yes, if enabled
Published LoRA adapter	qqyule/objectverse-diary-qwen15b-lora	small adapter over base model	yes, when enabled
Live Space total	MiniCPM-V vision + mock text	~8B active model parameters	<= 32B

If the optional MiniCPM-V 2.6 vision path and planned Qwen 1.5B text base are both enabled, the expected total remains about 9.5B plus a small LoRA adapter, safely under the 32B project budget.

Intended Inputs And Outputs

Inputs:

• user-uploaded everyday object photo
• optional object description
• personality mode

Outputs:

• structured object understanding JSON
• hidden object persona JSON
• short English-first diary with Chinese helper text
• object chat response
• share card preview
• anonymized trace record

Dataset Notes

Dataset planning lives in docs/DATASET.md.

Current preview data is deterministic and mock-generated. It should only be used for schema validation, dry-run validation, and workflow planning until real candidate samples are generated and curated.

The Modal training scaffold defaults to Qwen/Qwen2.5-1.5B-Instruct and saves adapter artifacts to a Modal Volume. data/train/objectverse_sft_curated_v2.jsonl contains 200 synthetic curated rows covering 40 everyday objects and 5 personality modes. It is published at https://huggingface.co/datasets/qqyule/objectverse-diary-sft-curated as objectverse_sft_curated_v2.jsonl.

Published adapter:

https://huggingface.co/qqyule/objectverse-diary-qwen15b-lora

Current v2 training run summary:

• Platform: Modal
• Run name: objectverse-diary-qwen15b-lora-v2
• Base model: Qwen/Qwen2.5-1.5B-Instruct
• Dataset: 200 synthetic curated v2 rows
• Train / eval rows: 180 / 20
• Steps: 120
• Max sequence length: 1536
• Learning rate: 0.0001
• Effective batch size: 8
• LoRA rank / alpha / dropout: 32 / 64 / 0.05
• Assistant-output-only loss: enabled
• Train loss: 0.3240
• Eval loss: 0.0162
• Epoch: 5.2222
• GGUF conversion: completed with pinned llama.cpp commit 8f83d6c271d194bde2d410145a0ce73bc42e85cd
• Published GGUF: objectverse-diary-qwen15b-lora-v2-q4_k_m.gguf

GGUF smoke status:

• Repo: qqyule/objectverse-diary-qwen15b-lora
• File: objectverse-diary-qwen15b-lora-v2-q4_k_m.gguf
• Local helper: scripts/check_llama_cpp_smoke.py
• Local result: passed on 2026-06-08 with llama-cpp text generation, no text-fallback-to-mock, schema-valid persona and diary, and non-empty chat reply.
• Space result: not run; do not claim live Space text runtime until a separate Space validation passes.

Safety And Privacy

• Do not use OpenAI, Anthropic, Gemini, Cohere, or other commercial model APIs.
• Do not publish private user photos or unconsented personal data.
• Do not include tokens, credit codes, emails, serial numbers, or credentials.
• Keep raw private traces out of public datasets.

Fallback Behavior

• If VLM loading fails, use manual description and stable example flow.
• If llama.cpp is not installed, TEXT_MODEL_PATH is missing, model loading fails, or output JSON is invalid, keep deterministic mock text fallback for demo safety.
• If model JSON is invalid, repair and validate before rendering.
• Runtime traces do not record literal TEXT_MODEL_PATH; they only record that an external GGUF path is configured.
• Hosted VLM validation evidence is preserved in data/traces/space-vlm/. These traces use real MiniCPM-V object understanding plus mock text generation and should not be described as full real-text-runtime traces.

Required Notes

• Total model parameter count must remain <= 32B.
• No commercial model APIs.
• Fallback behavior must be documented.
• Dataset provenance and privacy rules must be documented before release.