Spaces:
Runtime error
Runtime error
| title: Duel of Nemotron | |
| emoji: ⚔️ | |
| colorFrom: indigo | |
| colorTo: purple | |
| sdk: docker | |
| python_version: "3.11" | |
| app_port: 7860 | |
| tags: | |
| - thousand-token-wood | |
| - nemotron | |
| - fine-tuned | |
| - custom-ui | |
| - tiny-titan | |
| - self-play | |
| - rl | |
| - fighting-game | |
| - modal | |
| pinned: false | |
| # Duel of Nemotron ⚔️ — Hybrid Self-Play AI Fighter | |
| A cyberpunk fighting game where the AI opponent is powered by a | |
| **two-tier hybrid**: | |
| - **Nemotron 3 Nano 4B** (fine-tuned, on Modal A10) — the **strategist** | |
| - **Tiny Fighter** (~142k params, CPU, in this Space) — the **real-time executor** | |
| Nemotron watches the fight and outputs a *mode* (aggressive / defensive / | |
| grappling / etc.) every several moves. The Tiny Fighter, conditioned on that | |
| mode plus the last few moves, picks the actual next move in < 1ms on CPU. | |
| This is a tiny-model-implements-the-fast-loop + fine-tuned-LLM-sets-the-direction | |
| pattern: a small CPU policy network for real-time play, a larger fine-tuned | |
| model for strategic depth. | |
| ## How It Works | |
| ``` | |
| Browser (React + Three.js) ──fight state──▶ Space backend (HF Space CPU) | |
| ▲ │ | |
| │ ├──▶ Tiny Fighter (142k, <1ms) | |
| │ │ returns move + probs | |
| │ │ | |
| │ └──▶ Modal Nemotron (A10, cold start) | |
| │ every ~10 moves: | |
| │ returns strategic weights | |
| │ ▲ | |
| └──────────────────weights + reasoning────────┘ | |
| ``` | |
| ### Training Pipeline (on Modal A100-40GB) | |
| 1. **SFT Bootstrap** — 12k procedural examples teach Nemotron to output | |
| strategic weight JSON given fight state. | |
| 2. **Self-Play Rollouts** — 100 fights with the SFT model playing both sides. | |
| Win/loss outcomes provide reward signals. | |
| 3. **Reward-weighted fine-tuning** — positive-reward completions are reinforced, | |
| negative-reward completions suppressed. 3 epochs, A100-40GB. | |
| ### The Tiny Fighter | |
| - **~142k parameter MLP** with BatchNorm, trained on 20k procedurally | |
| generated (state, strategy) → move examples. | |
| - Runs on CPU in < 1ms per inference. Real-time safe. | |
| - Conditioned on Nemotron's strategic weights, so it *adapts its style* | |
| (aggressive vs. defensive vs. grappling) on the fly. | |
| - 15-move output vocabulary: jab, cross, hook, kick, uppercut, block, parry, | |
| dodge, advance, retreat, grapple, throw, sweep, feint, wait. | |
| ## Badges Targeted | |
| - ✅ **Tiny Titan** — the 142k param model is genuinely tiny and does real work | |
| - ✅ **Well-Tuned** — the Nemotron LoRA adapter is published at | |
| [sankalphs/duel-nemotron-strategist](https://huggingface.co/sankalphs/duel-nemotron-strategist) | |
| - ✅ **Off-Brand** — custom React + Three.js 3D fighting game (not default Gradio) | |
| - ✅ **Field Notes** — see blog post | |
| - ✅ **Modal Award** — training and inference both run on Modal | |
| - ✅ **Nemotron Quest** — fine-tuned Nemotron 3 Nano 4B for the fight | |
| ## Local Dev | |
| ```bash | |
| # Frontend | |
| cd 3d-game && npm install && npm run build | |
| # Space backend (CPU) | |
| pip install -r requirements.txt | |
| python app.py | |
| ``` | |
| Set `MODEL_SERVER` env var to your Modal inference endpoint to enable | |
| Nemotron strategy. Without it, the Space falls back to balanced defaults. | |
| ## Links | |
| - **Fine-tuned adapter**: https://huggingface.co/sankalphs/duel-nemotron-strategist | |
| - **Modal orchestration**: see `modal/app.py` in the repo | |
| - **Demo video**: _see social post_ | |
| - **Social post**: _see social post_ | |
| --- | |
| Built for the [Build Small Hackathon](https://huggingface.co/build-small-hackathon) | |
| by [@sankalphs](https://huggingface.co/sankalphs). 🍄 | |