Spaces:
Sleeping
Sleeping
| title: Lolaby | |
| emoji: 🌙 | |
| colorFrom: yellow | |
| colorTo: blue | |
| sdk: gradio | |
| python_version: "3.11" | |
| sdk_version: "6.17.3" | |
| app_file: app.py | |
| pinned: true | |
| license: llama3.2 | |
| short_description: AI-powered lullabies. | |
| tags: | |
| - lullaby | |
| - children | |
| - small-models | |
| - llama-cpp | |
| - fine-tuned | |
| - on-device | |
| - build-small-hackathon | |
| - backyard-ai | |
| - text-to-audio | |
| - agentic | |
| - gradio | |
| - track:backyard | |
| - sponsor:openbmb | |
| - achievement:offgrid | |
| - achievement:welltuned | |
| - achievement:offbrand | |
| - achievement:llama | |
| - achievement:sharing | |
| - achievement:fieldnotes | |
| - badge-tiny-titan | |
| models: | |
| - build-small-hackathon/lolaby-llama-3b | |
| - openbmb/MiniCPM-V-4.6 | |
| - hexgrad/Kokoro-82M | |
| datasets: | |
| - build-small-hackathon/lolaby-traces | |
| # Lolaby — AI-powered lullabies | |
| Meet Lola, your personal bedtime singer. | |
| A tiny AI that watches your child's drawings and sings them a personalised lullaby. | |
| Built for the [Hugging Face **Build Small Hackathon 2026**](https://huggingface.co/build-small-hackathon) · **Backyard AI** track 🏡 | |
| **Team:** [andyolivers](https://huggingface.co/andyolivers) & [volivers](https://huggingface.co/volivers) | |
| **Try it:** [the live Space](https://huggingface.co/spaces/build-small-hackathon/lolaby) | |
| **Demo video:** [YouTube walkthrough](https://youtu.be/eY_JnijT62E) | |
| **Field notes:** [the build journal](https://huggingface.co/blog/build-small-hackathon/lolaby-blog) | |
| **Social media post**: [live on LinkedIn](https://www.linkedin.com/posts/andyolivers_huggingface-gradio-hackathon-ugcPost-7471134065912573953-4uv2) | |
| **Models:** | |
| - [`lolaby-llama-3b`](https://huggingface.co/build-small-hackathon/lolaby-llama-3b) — lyrics | |
| - MiniCPM-V 4.6 1.3B — vision | |
| - Kokoro 82M — voice | |
| Every runtime model in Lolaby runs under **4B parameters**, keeping the full pipeline comfortably within the hackathon's **32B cap**. | |
| --- | |
| ## The Problem | |
| Getting a small child to fall asleep is a daily battle for parents and anyone who looks after kids. | |
| My partner's sister teaches kindergarten. Every day she runs nap time for fifteen 4-year-olds, and ever since they learned about music and instruments in class, it starts the same way: *"sing a song for me."* She'd love to give each child their own song, built from whatever they love that week — a stuffed fox, a new puppy, the rainbow. She doesn't have the time, the musical training, or a tool that could do it. | |
| **Lolaby is that tool.** The child shows Lola what they love — doodling on screen, or handing over a paper drawing for the teacher to photograph. The teacher types the child's name. A small, on-device AI looks at the drawing, writes a lullaby about it, and sings it back — in about a minute. | |
| Everything runs locally. No cloud LLM, no per-song API cost, no child's drawing or name ever leaving the device. No massive models — just genuinely tiny AI that fits everywhere. | |
| Lola turns any child's drawing into a unique lullaby, all on-device. | |
| ## How it works | |
| <p align="center"> | |
| <img | |
| src="https://i.postimg.cc/t9Q9HdvY/pipeline.png" | |
| width="600"> | |
| </p> | |
| **The drawing is optional.** | |
| No canvas drawing AND no upload? ⭢ The form's *What do they love?* field is used instead. | |
| Drawing AND typed loves? ⭢ Both inform the song. | |
| *Lola* (Lolaby's AI character) tells you what she saw between the audio and the lyrics, so you can see how the drawing turned into the song. | |
| ## What's inside: built as a tiny titan | |
| The **total parameters** across the whole pipeline are **well under 32B**. The lyric model is 3B, the vision model is 1.3B, and the voice is 82M. The synths have zero parameters — they're DSP (Digital Signal Processors). | |
| This app was intentionally designed as a **tiny titan**: every model in the pipeline is genuinely small, with lyrics, vision, and voice all running **under 4B parameters**. The goal wasn't just efficiency, it was portability. Lolaby was built to run locally, fit on modest hardware, and work the same way everywhere: a laptop, a CPU-only Hugging Face Space, or an offline machine with no cloud dependency at all. | |
| No giant foundation models, no hidden API calls, and no GPU requirement — just compact, local-first AI designed to run anywhere. | |
| | Component | Model / Library | Where it runs | | |
| | ---------------- | -------------------------------------------------- | ------------- | | |
| | Lyric generation | **Llama 3.2 3B**, fine-tuned, via `llama.cpp` | CPU, locally | | |
| | Drawing → words | **MiniCPM-V 4.6** (1.3B) via `transformers` | CPU, locally | | |
| | Stroke fallback | Pure NumPy color/density analysis | CPU, locally | | |
| | Singing voice | **Kokoro 82M** (<32B) | CPU, locally | | |
| | Instruments | Custom DSP synths, built from spectral analysis | CPU, locally | | |
| | Content safety | Local keyword + intent filter | CPU, locally | | |
| ## Hardware | |
| Lolaby runs locally on whatever machine you give it — a laptop, or a CPU-only Hugging Face Space. There's no cloud LLM in the loop at runtime: the lyric model, the vision model, and the audio synthesis all run on-device. | |
| The whole pipeline is CPU-only by design. The fine-tuned Llama 3.2 3B runs as a Q4_K_M GGUF through llama.cpp; the MiniCPM-V vision model and Kokoro TTS run in-process on CPU. Nothing is offloaded to a GPU or an external API, so the experience is identical wherever it runs — no model is skipped and no feature degrades depending on the host. | |
| If the vision model can't load for any reason, the app falls back to a NumPy stroke-and-colour analyzer so songs keep generating rather than breaking. | |
| This portability is intentional: the same repo can be forked and run anywhere — a laptop, a free CPU Space, an offline box — without changing a line of code. That's the "build small" idea taken literally: a complete, personalised lullaby pipeline that fits on modest hardware and owes nothing to the cloud at runtime. | |
| ## Badges | |
| This submission satisfies all six hackathon bonus quests: | |
| - 🔌 **Off the Grid** — No cloud APIs at runtime. Every model in the deployed app runs locally. | |
| - 🎯 **Well-Tuned** — The lyric model is a custom fine-tune of Llama 3.2 3B, [published on the Hub](https://huggingface.co/build-small-hackathon/lolaby-llama-3b). Trained on a 1,500-example dataset built from scratch with mechanical anti-boilerplate gates and 99.4% line uniqueness across the lyric corpus. | |
| - 🎨 **Off-Brand** — Fully custom Gradio UI served through `gr.Server` (Gradio's Server mode) rather than a stock launch: hand-drawn crayon aesthetic, paper-textured cards, light/dark themes with browser preference detection, and a character voice via the "Lola" persona that guides your experience by streaming the output. | |
| - 🦙 **Llama Champion** — Lyric inference runs through `llama-cpp-python` (the `llama.cpp` Python binding) on CPU at Q4_K_M quantisation. | |
| - 📡 **Sharing is Caring** — A full generation trace (drawing → vision → lyric prompt → raw model output → audio render parameters) is [published on the Hub](https://huggingface.co/datasets/build-small-hackathon/lolaby-traces) so anyone can study how the pipeline composes. | |
| - 📓 **Field Notes** — [Build journal here](https://huggingface.co/blog/build-small-hackathon/lolaby-blog): the lullaby fine-tune pipeline, the instrument synthesis, the drawing-to-lyrics vision model, and the local-first approach. | |
| ## Demo | |
| → Walkthrough powered by **[Google Gemini](https://gemini.google.com/)** and **[Veo 3](https://deepmind.google/models/veo/)**. | |
| <p align="center">Watch the demo ↓</p> | |
| <p align="center"> | |
| <a href="https://youtu.be/eY_JnijT62E"> | |
| <img | |
| src="https://i.postimg.cc/KjY7wC1N/demo-thumbnail2.jpg" | |
| width="700"> | |
| </a> | |
| </p> | |
| → Drop a like, upvote, comment or share on [LinkedIn](https://www.linkedin.com/posts/andyolivers_huggingface-gradio-hackathon-ugcPost-7471134065912573953-4uv2) and [YouTube](https://youtu.be/eY_JnijT62E). | |
| ## Try it yourself | |
| **Field notes:** [the build journal](https://huggingface.co/blog/build-small-hackathon/lolaby-blog) | |
| **Web version:** [the live Space](https://huggingface.co/spaces/build-small-hackathon/lolaby) | |
| Or to run it locally: | |
| ```bash | |
| git clone https://huggingface.co/spaces/build-small-hackathon/lolaby | |
| cd lolaby | |
| pip install -r requirements.txt | |
| python app.py | |
| ``` | |
| The lyric model is fetched from the Hub on first run (~2 GB GGUF, cached after). | |
| ## Dataset | |
| The lyric model was fine-tuned on **1,500 lullabies distilled from Claude Haiku 4.5** with strict anti-boilerplate gates: per-line n-gram dedup, opener dedup, theme caps, format gates, and per-example safety screening. The dataset itself, the generator (`generate_dataset.py`), and the training notebook (`train_lullaby.ipynb`) are in the [`train/`](./train/) folder. Regenerating the dataset requires an Anthropic API key; running the deployed app does not. | |
| Full details in the [model card](https://huggingface.co/build-small-hackathon/lolaby-llama-3b) and the [Field Notes blog post](https://huggingface.co/blog/build-small-hackathon/lolaby-blog). | |
| ## Repository | |
| ``` | |
| . | |
| ├── app.py # Gradio entrypoint | |
| ├── requirements.txt | |
| ├── utils/ | |
| │ └── safety.py # Content-safety filter | |
| ├── draw/ # Drawing-related helpers | |
| │ ├── vision.py # MiniCPM-V 4.6 wrapper | |
| │ └── strokes.py # NumPy fallback if vision is unavailable | |
| ├── synths/ # Custom DSP instruments + Kokoro voice | |
| │ ├── guitar.py | |
| │ ├── piano.py | |
| │ ├── musicbox.py | |
| │ ├── harp.py | |
| │ ├── xylophone.py | |
| │ ├── ocarina.py | |
| │ └── voice.py | |
| ├── train/ # Dataset + training (build-time only; not used at runtime) | |
| │ ├── generate_dataset.py | |
| │ ├── train_lullaby.ipynb | |
| │ └── lullaby_dataset.jsonl | |
| └── tests/ # Sanity checks for audio + LLM + voice | |
| ``` | |
| ## Safety | |
| Lolaby is built for small children, so safety isn't an afterthought — it's wired through the pipeline at three points: | |
| - **At training time** — every example in the lyric model's training set was screened during dataset distillation. The model learned from already-wholesome material, not from the open web. | |
| - **At input time** — when a user types a *love* or *fear*, the text is screened against a curated list of terms inappropriate for a child's lullaby (death, weapons, horror, substances, self-harm). Anything matching gets a gentle redirect ("Let's keep the lullaby to gentle, cosy things…") instead of a generation. | |
| - **At generation time** — the same filter is shared between the runtime app and the dataset generator, so the training data and the live app can never enforce different rules. | |
| ## Limitations | |
| - **English only.** The lyric model was trained on English data. | |
| - **First-time Space cold-start** might take some time while all models pre-warm at boot. | |
| - **Strange or unusual loves** may be gently generalised by the lyric model into a nearby comforting concept — that's a deliberate behaviour for a bedtime song (soft landing > literal lookup) and described in the model card. | |
| - **Children's drawings are interpreted by an AI** that maps rough shapes and colours onto familiar concepts — much like an adult parent guessing what a kid drew. It can miss subtle details: a stick figure becomes "a little person", a wobbly square with a triangle becomes "a house". The "Lola saw…" hint shows exactly what she understood, so you can redraw or use the What do they love? field instead. | |
| ## Credits | |
| - **[Meta](https://ai.meta.com/llama/)** — Llama 3.2 3B Instruct (base model, used under the Llama 3.2 Community License). | |
| - **[OpenBMB](https://huggingface.co/openbmb)** — MiniCPM-V 4.6 (vision). | |
| - **[hexgrad/Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M)** — TTS voice. | |
| - **[Unsloth](https://github.com/unslothai/unsloth)** — 4-bit + LoRA training stack. | |
| - **[Hugging Face & Gradio](https://huggingface.co/build-small-hackathon)** — for hosting the Build Small Hackathon and creating space for small-AI projects. | |
| ## License | |
| Apache 2.0 for the app code in this repo. The lyric model weights inherit the **Llama 3.2 Community License** from their base — see the [model card](https://huggingface.co/build-small-hackathon/lolaby-llama-3b) for full terms. | |
| --- | |
| <hr> | |
| <p align="center"> | |
| Built with bedtime magic. ✨<br> | |
| <i>Lolaby • 2026</i> | |
| </p> | |