docs/EVALUATION.md

Sponsor Prizes + Bonus Quests — Evaluation

评估 — high impact, least effort.

For the model-swappable VLM backend (MODELS dict, UI dropdown, provider abstraction), see docs/ARCHITECTURE.md. For deployment specifics, see docs/DEPLOYMENT.md.

---

Hackathon Requirements (must meet to qualify)

#	Requirement	Status	Notes
1	Model ≤32B parameters	✅ Done	CohereLabs/aya-vision-32b (32B); also openbmb/mini-cpm-v-4_6 (~4B) available
2	Gradio app hosted as HF Space	⚠️ Deployed, BUILD_ERROR	Gradio 5/6 compat added, rebuild triggered
3	Demo video + social post	🔲 Not started	Required to qualify

---

Sponsor Prizes

🥇 HuggingFace — $15,000 cash (top awards)

Requirement: Best overall projects.

Our alignment: ✅ Core criteria met. ≤32B ✓, Gradio ✓, real problem ✓.

Verdict: PRIMARY TARGET.

---

🤖 OpenAI — $10,000 + $100 Codex credits

Requirement: Use OpenAI model.

Our alignment: ❌ We use CohereLabs/aya-vision-32b via HF Router (not OpenAI).

Verdict: SKIP.

---

🔬 OpenBMB — $10,000 special awards ($5K per track)

Requirement: Use OpenBMB model (MiniCPM-V 4.6 for vision).

Our alignment: ⚠️ HIGH-VALUE SWAP — $5K for one-line model name change.

• Already in MODELS dict as minicpm-v-4.6 → openbmb/mini-cpm-v-4_6
• HF Router compatible, ~4B params

Resources:

• Model: https://huggingface.co/openbmb/MiniCPM-V-4.6
• Cookbook: https://opensqz.github.io/MiniCPM-V-CookBook/site/en/index.html
• GitHub: https://github.com/OpenBMB/MiniCPM-V-Apps

Verdict: SWAP FOR. $5K for minimal effort.

---

🎮 NVIDIA — 2× RTX 5080 GPUs (top projects)

Requirement: ⚠️ UNCONFIRMED — per Discord: Nemotron models may be strictly required.

Possible models (≤32B): nvidia/Nemotron-4-2B-base, nvidia/Nemotron-4-8B-base, nvidia/Nemotron-4-15B-base

Our alignment: ❌ Currently using CohereLabs/aya-vision-32b.

If required: Already in MODELS dict as nemotron-15b. Would need to switch VLM or confirm "top project" qualifier.

Verdict: CONFIRM WITH ORGANIZERS FIRST.

---

🟦 Modal — $250 credits all + $20,000 winners

Requirement: Use Modal for deployment.

Our alignment: ✅ ALREADY DEPLOYED — narwall-tech--color-ux-access-ui.modal.run live.

Verdict: ✅ DONE.

---

🌲 Cohere — Prize support

Requirement: Use Cohere model.

Our alignment: ✅ Already using CohereLabs/aya-vision-32b.

Verdict: ✅ DONE.

---

🖼️ Black Forest Labs — Prize support

Requirement: Use FLUX.2 [klein] — 4B text-to-image model.

Our alignment: ❌ Not relevant (image generation, not VLM).

Verdict: SKIP.

---

Bonus Quests

🥇 Bonus 1: Well-Tuned — Fine-tuned model on HF

Effort: HIGH. Impact: HIGH if judges value model work. LoRA fine-tune on accessibility dataset. Risk: time-consuming.

Verdict: SKIP FOR NOW.

🥈 Bonus 2: Off-Brand — Custom frontend past Gradio defaults

Effort: LOW-MEDIUM. Impact: HIGH (first impression in demo).

• Custom CSS for accessibility theme
• CVD comparison slider (original vs simulated side-by-side)
• JavaScript for smooth transitions

Verdict: WORTH DOING after MVP is functional.

🥉 Bonus 3: Llama Champion — llama.cpp runtime

Effort: MEDIUM. Impact: MEDIUM. Hybrid: VLM for image + llama.cpp for text. GGUF model (e.g., Qwen2-VL-7B-Instruct-GGUF) for local inference.

Verdict: MEDIUM PRIORITY.

🎁 Bonus 4: Sharing is Caring — Shared agent trace on HF

Effort: LOW. Impact: MEDIUM. Save dev conversation logs as HF Dataset, organized by phase.

Verdict: DO THIS. Low effort, process quality signal.

📝 Bonus 5: Field Notes — Blog post on huggingface.co/blog

Effort: LOW-MEDIUM. Impact: MEDIUM-HIGH. Technical blog about colorblind accessibility testing methodology.

Verdict: DO THIS. Low effort, high hackathon + NARWALL brand impact.

---

Priority Order

Priority	Task	Reason
🔴 P0	Confirm NVIDIA Nemotron requirement	May require VLM swap
🔴 P0	Add HF_TOKEN secret to Space	VLM won't work without it
🔴 P0	Test first Space inference	Verify end-to-end works
🟡 P1	Implement model swappable backend	✅ Done — see ARCHITECTURE.md
🟡 P1	Blog post (Bonus 5)	Low effort, high impact
🟡 P1	OpenBMB MiniCPM-V swap	$5K for one-line change
🟢 P2	Off-brand Gradio theme (Bonus 2)	Custom CSS, CVD slider
🟢 P2	Agent trace (Bonus 4)	Low effort
🟢 P2	Demo video + social post	Required to qualify
🔵 SKIP	BFL Klein	Not relevant to VLM task
🔵 SKIP	Fine-tune (Bonus 1)	High effort, time risk

---

Last updated: 2026-06-07