HIMANSHUKUMARJHA's picture
Field Notes: add llama.cpp 0.5B engine (Llama Champion)
daff47b verified
Raw
History Blame Contribute Delete
14.3 kB
<!DOCTYPE html>
<!-- Exam Panic Rescue field notes -->
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Exam Panic Rescue — Field Notes</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Newsreader:ital,opsz,wght@0,6..72,400;0,6..72,500;0,6..72,600;1,6..72,400&family=Inter:wght@400;500;600;700&display=swap" rel="stylesheet">
<style>
:root{
--ink:#13201c; --soft:#3c4b45; --faint:#6b7a73;
--green:#005844; --green-d:#03372c; --coral:#84231b;
--paper:#ffffff; --rule:#e7e2d6; --wash:#f6f4ee;
--measure:40rem;
}
*{box-sizing:border-box}
html{-webkit-text-size-adjust:100%}
body{
margin:0; background:var(--paper); color:var(--ink);
font-family:"Inter",system-ui,sans-serif; font-size:19px; line-height:1.7;
-webkit-font-smoothing:antialiased;
}
.wrap{max-width:var(--measure); margin:0 auto; padding:0 7vw;}
@media(min-width:760px){ .wrap{padding:0 24px;} }
header.masthead{
border-bottom:2px solid var(--ink); margin-top:56px; padding-bottom:14px;
display:flex; justify-content:space-between; align-items:baseline; gap:16px;
}
.masthead .brand{font-family:"Inter"; font-weight:700; letter-spacing:.16em; text-transform:uppercase; font-size:13px; color:var(--green);}
.masthead .meta{font-size:13px; color:var(--faint); letter-spacing:.04em;}
h1{
font-family:"Newsreader",serif; font-weight:600; font-size:clamp(2.4rem,6vw,3.6rem);
line-height:1.05; letter-spacing:-.015em; margin:38px 0 10px;
}
.dek{font-family:"Newsreader",serif; font-style:italic; font-size:clamp(1.15rem,2.6vw,1.5rem); color:var(--soft); line-height:1.45; margin:0 0 26px; max-width:34rem;}
.byline{font-size:14px; color:var(--faint); border-top:1px solid var(--rule); border-bottom:1px solid var(--rule); padding:12px 0; margin-bottom:40px; letter-spacing:.02em;}
.byline b{color:var(--green-d); font-weight:600;}
article h2{
font-family:"Newsreader",serif; font-weight:600; font-size:clamp(1.5rem,3.4vw,2rem);
letter-spacing:-.01em; margin:48px 0 4px; line-height:1.15;
}
article h2 .num{display:block; font-family:"Inter"; font-weight:700; font-size:12px; letter-spacing:.18em; text-transform:uppercase; color:var(--coral); margin-bottom:8px;}
article h3{font-family:"Inter"; font-weight:700; font-size:1.06rem; margin:30px 0 2px; color:var(--green-d);}
p{margin:14px 0;}
a{color:var(--green); text-underline-offset:3px; text-decoration-thickness:1px;}
strong{color:var(--green-d);}
.lead{font-size:1.12rem;}
.lead::first-letter{
font-family:"Newsreader",serif; font-weight:600; float:left; font-size:4.2rem; line-height:.78;
padding:6px 12px 0 0; color:var(--coral);
}
ul{margin:14px 0; padding-left:1.1em;}
li{margin:9px 0;}
li::marker{color:var(--green);}
.pull{
border-left:3px solid var(--green); background:var(--wash); margin:30px 0;
padding:18px 22px; font-family:"Newsreader",serif; font-style:italic; font-size:1.3rem;
line-height:1.4; color:var(--green-d);
}
.models{border:1px solid var(--rule); border-radius:12px; overflow:hidden; margin:26px 0;}
.models .row{display:grid; grid-template-columns:1fr; gap:2px 18px; padding:14px 18px; border-bottom:1px solid var(--rule);}
@media(min-width:620px){ .models .row{grid-template-columns:15rem 1fr;} }
.models .row:last-child{border-bottom:0;}
.models .row b{color:var(--green-d); font-weight:600;}
.models .row span{color:var(--soft); font-size:.96rem;}
.note{font-size:.92rem; color:var(--faint);}
hr.div{border:0; border-top:1px solid var(--rule); margin:46px 0;}
.links{background:var(--green-d); color:#f3efe4; border-radius:14px; padding:26px 26px; margin:40px 0 8px;}
.links h2{color:#fff; margin:0 0 12px; font-size:1.3rem;}
.links a{color:#ffd9b0; display:inline-block; word-break:break-word;}
.links .lrow{padding:8px 0; border-top:1px solid rgba(255,255,255,.16); font-size:.98rem;}
.links .lrow:first-of-type{border-top:0;}
.links .lrow b{color:#fff; display:block; font-size:.8rem; letter-spacing:.12em; text-transform:uppercase; margin-bottom:2px; font-weight:700;}
footer{margin:30px 0 80px; font-size:13px; color:var(--faint); line-height:1.6;}
</style>
</head>
<body>
<div class="wrap">
<header class="masthead">
<span class="brand">Field Notes</span>
<span class="meta">Build Small Hackathon · Backyard AI</span>
</header>
<h1>A calm tool, built on small models</h1>
<p class="dek">What I learned shipping Exam Panic Rescue — a last-minute study triage app that runs entirely on models you wouldn't call large.</p>
<p class="byline">A build report for <b>Exam Panic Rescue</b> · OpenBMB MiniCPM · NVIDIA Nemotron · Hugging Face ZeroGPU · ≤32B throughout</p>
<article>
<p class="lead">Every exam season has the same bad hour. Two hours left, the syllabus is a wall, and you're rereading the same page without retaining a word. You're not studying anymore — you're panicking with a book open. I wanted a tool that does one thing for that exact moment: tell a stressed student what to do <em>next</em>, and nothing more.</p>
<p>Exam Panic Rescue is deliberately narrow. A student pastes what they half-know, what's scaring them, and how many minutes are left — or snaps a photo of their syllabus — and gets back one ranked rescue plan, five practice drills written for their own topics, a triage clock that runs in real time, and a one-page final sheet to read before they walk in. It can read that sheet aloud. That's the whole product. The hard part was resisting everything else.</p>
<h2><span class="num">★ — Real-world proof</span>What happened when a real student used it</h2>
<p>The test that matters for a tool like this isn't a benchmark — it's whether a real person reaches for it in their actual worst hour. The day before a university <strong>Machine Learning</strong> exam, a final-year student (I'll call them <strong>R.</strong>, shared with their consent and anonymized) opened the live app with eight hours left and this panic dump:</p>
<div class="pull">"I know the basics of backpropagation but I keep going blank on the mathematical derivations of backprop and CNN convolutional dimensions. I'm afraid I'll freeze when calculating output sizes."</div>
<p>The small model wrote them a recall-first plan, a triage clock across the eight hours, and five drills aimed exactly at the leak — parameter counts, a CNN output-dimension calculation, a cross-entropy gradient derivation, and a max-pooling output-size question.</p>
<p><strong>The next day, one of those model-written drills closely matched a question that actually appeared on the exam.</strong> R. said the tool was genuinely useful. That's the whole thesis in one real data point: a small model, given a tight job, produced practice that mapped onto a real exam for a real student. The full anonymized session is in the public <a href="https://huggingface.co/datasets/build-small-hackathon/exam-panic-rescue-build-trace" target="_blank" rel="noopener noreferrer">build-trace dataset</a> (the <code>real_user</code> config).</p>
<h2><span class="num">01 — The constraint</span>Small models, on purpose</h2>
<p>The hackathon rule is a ceiling: nothing over 32B. I treated it as a design value instead of a limitation. A panicking student doesn't need a frontier model to be told "stop opening new chapters and protect these five marks." The intelligence that matters here is <strong>judgment under time pressure</strong>, and small models are more than enough to write good drills and a tight plan — if you give them a tight job.</p>
<div class="models">
<div class="row"><b>OpenBMB MiniCPM-V-4.5</b><span>The primary engine. Writes the rescue plan, drills, and answers — and, being a vision-language model, reads a photo of the syllabus directly in the same call.</span></div>
<div class="row"><b>OpenBMB VoxCPM2</b><span>Reads the final sheet aloud for the last-minute walk-in.</span></div>
<div class="row"><b>NVIDIA Nemotron-Mini-4B</b><span>Selectable text-only alternate (4B).</span></div>
<div class="row"><b>OpenBMB MiniCPM4 0.5B (GGUF)</b><span>Runs through the llama.cpp runtime on CPU — the Llama Champion path, and a genuinely tiny 0.5B Tiny Titan.</span></div>
</div>
<p class="note">All five run on Hugging Face ZeroGPU. Every generation prints a runtime note saying exactly which model ran and on what hardware — so the model behind any output is never ambiguous.</p>
<h2><span class="num">02 — The lesson that cost me the most</span>Cold starts on a shared GPU</h2>
<p>ZeroGPU is wonderful and free, but it has a shape you have to design around: the GPU is only attached <em>inside</em> a decorated function, and that function has a strict time budget. My first version loaded each model inside that budget. The default model stayed warm and was instant. But the moment a student picked the Nemotron engine, or uploaded the first photo, the app had to <strong>download several gigabytes of weights inside the GPU window</strong> — and a cold download plus load plus generation blew straight through the limit. ZeroGPU aborted the call, and the user got the deterministic fallback instead of the real model.</p>
<p>The symptom was maddening: the same code that worked beautifully on the second call fell back instantly on the first. The fix, once I understood it, was small and clean — <strong>prefetch the weights on the CPU, before ever touching the GPU.</strong> Move the multi-gigabyte download out of the timed window so the GPU call only pays the fast load-and-generate cost. After that, a cold first call returns real model output instead of a template. I verified it live: the exact Nemotron call that used to fall back now returns a real packet.</p>
<div class="pull">The bug taught me the platform. The download wasn't slow — it was in the wrong place.</div>
<h2><span class="num">03 — Honesty as a feature</span>Never crash, never lie</h2>
<p>Because the model can be cold, or the GPU can be busy, or a node can fault, I decided early that the app must <strong>always return a complete, useful packet</strong> — and must always tell the truth about how it made it. So there's a deterministic study engine underneath every model path. If a model is unavailable, the student still gets a real plan, real drills, and a real triage clock from templates, with a runtime note that says plainly "fallback used." Nothing errors out in a student's face, and nothing pretends a model ran when it didn't.</p>
<p>This turned out to matter for judging too. For a Backyard AI tool, the trust comes from the thing being honest about its own limits — a runtime note that says "Generated with MiniCPM-V-4.5 on ZeroGPU" or "fallback used" is worth more than a confident black box.</p>
<h2><span class="num">04 — Small things that were not small</span>Getting the details right</h2>
<h3>The time math</h3>
<p>The triage clock splits the remaining minutes into blocks. My first version's blocks didn't sum back to the total — a rounding leak. Rewriting it with largest-remainder apportionment fixed it so the clock is always exactly the time the student actually has, from 15 minutes to a full day.</p>
<h3>Designing for a panicking human</h3>
<p>An overflowing examples table made the demo feel like a spreadsheet, so it became four large one-tap panic cases. A dark-mode rendering bug hid dark label text on dark surfaces, so I forced the cream light theme the whole design was built for. The copyable field-note block overflowed on a 390px phone until I wrapped it. None of this is glamorous; all of it is the difference between "tool" and "toy."</p>
<h3>The operational reality of a free Space</h3>
<p>A live ZeroGPU Space is a real piece of infrastructure. I hit a transient ECC GPU fault (fixed by a restart), and a Space that paused itself after a burst of heavy testing. The lesson: treat the live demo like production — verify it end to end, and check it's actually <em>running</em> before you rely on it.</p>
<h2><span class="num">05 — What's still open</span>Honest gaps</h2>
<ul>
<li><strong>The real test is a real student.</strong> The Backyard AI track is judged on whether the person you built it for actually used it. The app ships a field-note prompt to capture an honest before/after — and that's the validation I care about most.</li>
<li><strong>First-call latency on secondary models.</strong> The prefetch fix trades a fast-but-fake fallback for real-but-slower output on a cold first call. For a panic tool, I think real wins — but warming the model before a live demo is still the move.</li>
</ul>
<hr class="div">
<p>If there's one thing I'd take to the next build: a tight job for a small model, an honest fallback under it, and a runtime note that never lies, will get you a tool people can actually trust in their worst hour. You don't need a giant model to rescue one stressed student.</p>
<div class="links">
<h2>Try it / read the code</h2>
<div class="lrow"><b>Live app</b><a href="https://huggingface.co/spaces/build-small-hackathon/exam-panic-rescue" target="_blank" rel="noopener noreferrer">huggingface.co/spaces/build-small-hackathon/exam-panic-rescue</a></div>
<div class="lrow"><b>Code</b><a href="https://github.com/himanshu748/exam-panic-rescue" target="_blank" rel="noopener noreferrer">github.com/himanshu748/exam-panic-rescue</a></div>
<div class="lrow"><b>Open build traces</b><a href="https://huggingface.co/datasets/build-small-hackathon/exam-panic-rescue-build-trace" target="_blank" rel="noopener noreferrer">huggingface.co/datasets/build-small-hackathon/exam-panic-rescue-build-trace</a></div>
</div>
</article>
<footer>
Built for the Build Small Hackathon · Backyard AI track. Models: OpenBMB MiniCPM (text, vision, voice), NVIDIA Nemotron-Mini-4B, all ≤32B on Hugging Face ZeroGPU. MIT licensed.
</footer>
</div>
</body>
</html>