README.md

metadata

title: Lolaby
emoji: 🌙
colorFrom: yellow
colorTo: blue
sdk: gradio
python_version: '3.11'
sdk_version: 6.17.3
app_file: app.py
pinned: true
license: llama3.2
short_description: AI-powered lullabies.
tags:
  - lullaby
  - children
  - small-models
  - llama-cpp
  - fine-tuned
  - on-device
  - build-small-hackathon
  - backyard-ai
  - text-to-audio
  - agentic
  - gradio
  - track:backyard
  - sponsor:openbmb
  - achievement:offgrid
  - achievement:welltuned
  - achievement:offbrand
  - achievement:llama
  - achievement:sharing
  - achievement:fieldnotes
  - badge-tiny-titan
models:
  - build-small-hackathon/lolaby-llama-3b
  - openbmb/MiniCPM-V-4.6
  - hexgrad/Kokoro-82M
datasets:
  - build-small-hackathon/lolaby-traces

Lolaby — AI-powered lullabies

Meet Lola, your personal bedtime singer. A tiny AI that watches your child's drawings and sings them a personalised lullaby.

Built for the Hugging Face Build Small Hackathon 2026 · Backyard AI track 🏡

Team: andyolivers & volivers

Try it: the live Space
Demo video: YouTube walkthrough
Field notes: the build journal
Social media post: live on LinkedIn
Models:

• lolaby-llama-3b — lyrics
• MiniCPM-V 4.6 1.3B — vision
• Kokoro 82M — voice

Every runtime model in Lolaby runs under 4B parameters, keeping the full pipeline comfortably within the hackathon's 32B cap.

---

The Problem

Getting a small child to fall asleep is a daily battle for parents and anyone who looks after kids.

My partner's sister teaches kindergarten. Every day she runs nap time for fifteen 4-year-olds, and ever since they learned about music and instruments in class, it starts the same way: "sing a song for me." She'd love to give each child their own song, built from whatever they love that week — a stuffed fox, a new puppy, the rainbow. She doesn't have the time, the musical training, or a tool that could do it.

Lolaby is that tool. The child shows Lola what they love — doodling on screen, or handing over a paper drawing for the teacher to photograph. The teacher types the child's name. A small, on-device AI looks at the drawing, writes a lullaby about it, and sings it back — in about a minute.

Everything runs locally. No cloud LLM, no per-song API cost, no child's drawing or name ever leaving the device. No massive models — just genuinely tiny AI that fits everywhere.

Lola turns any child's drawing into a unique lullaby, all on-device.

How it works

The drawing is optional. No canvas drawing AND no upload? ⭢ The form's What do they love? field is used instead. Drawing AND typed loves? ⭢ Both inform the song.

Lola (Lolaby's AI character) tells you what she saw between the audio and the lyrics, so you can see how the drawing turned into the song.

What's inside: built as a tiny titan

The total parameters across the whole pipeline are well under 32B. The lyric model is 3B, the vision model is 1.3B, and the voice is 82M. The synths have zero parameters — they're DSP (Digital Signal Processors).

This app was intentionally designed as a tiny titan: every model in the pipeline is genuinely small, with lyrics, vision, and voice all running under 4B parameters. The goal wasn't just efficiency, it was portability. Lolaby was built to run locally, fit on modest hardware, and work the same way everywhere: a laptop, a CPU-only Hugging Face Space, or an offline machine with no cloud dependency at all.

No giant foundation models, no hidden API calls, and no GPU requirement — just compact, local-first AI designed to run anywhere.

Component	Model / Library	Where it runs
Lyric generation	Llama 3.2 3B, fine-tuned, via llama.cpp	CPU, locally
Drawing → words	MiniCPM-V 4.6 (1.3B) via transformers	CPU, locally
Stroke fallback	Pure NumPy color/density analysis	CPU, locally
Singing voice	Kokoro 82M (<32B)	CPU, locally
Instruments	Custom DSP synths, built from spectral analysis	CPU, locally
Content safety	Local keyword + intent filter	CPU, locally

Hardware

Lolaby runs locally on whatever machine you give it — a laptop, or a CPU-only Hugging Face Space. There's no cloud LLM in the loop at runtime: the lyric model, the vision model, and the audio synthesis all run on-device.

The whole pipeline is CPU-only by design. The fine-tuned Llama 3.2 3B runs as a Q4_K_M GGUF through llama.cpp; the MiniCPM-V vision model and Kokoro TTS run in-process on CPU. Nothing is offloaded to a GPU or an external API, so the experience is identical wherever it runs — no model is skipped and no feature degrades depending on the host.

If the vision model can't load for any reason, the app falls back to a NumPy stroke-and-colour analyzer so songs keep generating rather than breaking.

This portability is intentional: the same repo can be forked and run anywhere — a laptop, a free CPU Space, an offline box — without changing a line of code. That's the "build small" idea taken literally: a complete, personalised lullaby pipeline that fits on modest hardware and owes nothing to the cloud at runtime.

Badges

This submission satisfies all six hackathon bonus quests:

• 🔌 Off the Grid — No cloud APIs at runtime. Every model in the deployed app runs locally.
• 🎯 Well-Tuned — The lyric model is a custom fine-tune of Llama 3.2 3B, published on the Hub. Trained on a 1,500-example dataset built from scratch with mechanical anti-boilerplate gates and 99.4% line uniqueness across the lyric corpus.
• 🎨 Off-Brand — Fully custom Gradio UI served through gr.Server (Gradio's Server mode) rather than a stock launch: hand-drawn crayon aesthetic, paper-textured cards, light/dark themes with browser preference detection, and a character voice via the "Lola" persona that guides your experience by streaming the output.
• 🦙 Llama Champion — Lyric inference runs through llama-cpp-python (the llama.cpp Python binding) on CPU at Q4_K_M quantisation.
• 📡 Sharing is Caring — A full generation trace (drawing → vision → lyric prompt → raw model output → audio render parameters) is published on the Hub so anyone can study how the pipeline composes.
• 📓 Field Notes — Build journal here: the lullaby fine-tune pipeline, the instrument synthesis, the drawing-to-lyrics vision model, and the local-first approach.

Demo

→ Walkthrough powered by Google Gemini and Veo 3.

Watch the demo ↓

→ Drop a like, upvote, comment or share on LinkedIn and YouTube.

Try it yourself

Field notes: the build journal

Web version: the live Space

Or to run it locally:

git clone https://huggingface.co/spaces/build-small-hackathon/lolaby
cd lolaby
pip install -r requirements.txt
python app.py

The lyric model is fetched from the Hub on first run (~2 GB GGUF, cached after).

Dataset

The lyric model was fine-tuned on 1,500 lullabies distilled from Claude Haiku 4.5 with strict anti-boilerplate gates: per-line n-gram dedup, opener dedup, theme caps, format gates, and per-example safety screening. The dataset itself, the generator (generate_dataset.py), and the training notebook (train_lullaby.ipynb) are in the train/ folder. Regenerating the dataset requires an Anthropic API key; running the deployed app does not.

Full details in the model card and the Field Notes blog post.

Repository

.
├── app.py                  # Gradio entrypoint
├── requirements.txt
├── utils/
│   └── safety.py               # Content-safety filter
├── draw/                   # Drawing-related helpers
│   ├── vision.py               # MiniCPM-V 4.6 wrapper
│   └── strokes.py              # NumPy fallback if vision is unavailable
├── synths/                 # Custom DSP instruments + Kokoro voice
│   ├── guitar.py
│   ├── piano.py
│   ├── musicbox.py
│   ├── harp.py
│   ├── xylophone.py
│   ├── ocarina.py
│   └── voice.py
├── train/                  # Dataset + training (build-time only; not used at runtime)
│   ├── generate_dataset.py
│   ├── train_lullaby.ipynb
│   └── lullaby_dataset.jsonl
└── tests/                  # Sanity checks for audio + LLM + voice

Safety

Lolaby is built for small children, so safety isn't an afterthought — it's wired through the pipeline at three points:

• At training time — every example in the lyric model's training set was screened during dataset distillation. The model learned from already-wholesome material, not from the open web.
• At input time — when a user types a love or fear, the text is screened against a curated list of terms inappropriate for a child's lullaby (death, weapons, horror, substances, self-harm). Anything matching gets a gentle redirect ("Let's keep the lullaby to gentle, cosy things…") instead of a generation.
• At generation time — the same filter is shared between the runtime app and the dataset generator, so the training data and the live app can never enforce different rules.

Limitations

• English only. The lyric model was trained on English data.
• First-time Space cold-start might take some time while all models pre-warm at boot.
• Strange or unusual loves may be gently generalised by the lyric model into a nearby comforting concept — that's a deliberate behaviour for a bedtime song (soft landing > literal lookup) and described in the model card.
• Children's drawings are interpreted by an AI that maps rough shapes and colours onto familiar concepts — much like an adult parent guessing what a kid drew. It can miss subtle details: a stick figure becomes "a little person", a wobbly square with a triangle becomes "a house". The "Lola saw…" hint shows exactly what she understood, so you can redraw or use the What do they love? field instead.

Credits

• Meta — Llama 3.2 3B Instruct (base model, used under the Llama 3.2 Community License).
• OpenBMB — MiniCPM-V 4.6 (vision).
• hexgrad/Kokoro-82M — TTS voice.
• Unsloth — 4-bit + LoRA training stack.
• Hugging Face & Gradio — for hosting the Build Small Hackathon and creating space for small-AI projects.

License

Apache 2.0 for the app code in this repo. The lyric model weights inherit the Llama 3.2 Community License from their base — see the model card for full terms.

---

---

Built with bedtime magic. ✨
Lolaby • 2026