File size: 6,707 Bytes
bffdb7c 0de0f67 bffdb7c 0de0f67 ce3c8c3 0de0f67 c9d8260 0de0f67 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 | ---
license: mit
language:
- en
tags:
- html
- css
- tailwind
- code-generation
- from-scratch
- small-model
- text-generation
library_name: transformers
pipeline_tag: text-generation
---
# Indensa-Coder-FrontEnd
⚠️ Note: The native Ollama UI may visually collapse indentation/newlines in generated HTML. This is a UI rendering issue, not a model issue. Verify output in Ollama CLI, VS Code, or by copying the generated text into a code editor.
# Indensa-Coder-FrontEnd
`This is a V1 model. Can be used. V2 is coming soon!`
A 33M parameter decoder-only transformer that takes a text prompt and spits out a bite-sized HTML + Tailwind CSS block. Trained from scratch on my desk in 6 minutes.
This is not a SOTA model. It's not trying to be. It's a small honest model that learned the shape of web components from a curated dataset, and now it can make new ones.
If you wanted GPT-4-level web design, this isn't it. If you wanted a tiny model you can actually run anywhere, including on Ollama with a fast prompt-to-HTML use case, you're in the right place.
 
## What it does
You give it something like:
```
PROMPT: a hero section for a SaaS landing page
HTML:
```
And it continues with a real HTML document using Tailwind classes. The output is bite-sized (≤512 tokens by design), so think landing page sections, pricing cards, contact forms, navbars — not entire 10-page websites.
## Sample output
Prompt: *"a publishing house landing page"*
```html
<html>
<link href="https://cdn.jsdelivr.net/npm/tailwindcss@2.2.19/dist/tailwind.min.css" rel="stylesheet">
<body class="bg-gray-100">
<header class="bg-white p-4">
<h1 class="text-2xl font-bold">Publishing House</h1>
</header>
<section class="p-4">
<h2 class="text-xl font-bold">Welcome to our Publishing House</h2>
<p>We are a leading publisher of high-quality books...</p>
</section>
...
<footer class="bg-white p-4">
<p>© 2022 Publishing House. All rights reserved.</p>
</footer>
</body>
</html>
```
That output was checked manually against the training set — no 100% match exists in training data. So it's actually generating, not parroting.
## The honest details
| Thing | Value |
|---|---|
| Parameters | 33.6M |
| Architecture | Decoder-only transformer |
| Layers | 8 |
| d_model | 512 |
| Heads | 8 (head_dim 64) |
| FFN dim | 2048 |
| Block size | 1024 |
| Tokenizer | Custom ByteLevel BPE, vocab 16,000 |
| Training data | ~50k samples from HuggingFaceM4/WebSight v0.2 (Tailwind) |
| Training tokens | 10.17M (8 epochs = ~81M effective) |
| Best val loss | 0.4772 |
| Training time | 7.4 minutes |
| Hardware | RTX 5080 16GB |
| Precision | bf16 mixed |
## Why custom BPE
GPT-2's BPE was trained on web prose, not code. It splits stuff like `text-3xl` into 4 tokens and `bg-gradient-to-r` into 7 tokens. My custom 16k BPE trained on WebSight compresses these to 1-2 tokens because Tailwind patterns are everywhere in the corpus.
Net result: roughly 1.5-2× better compression on the same content, which means more useful content fits inside the 512-token window.
## Why 33M params
I did the math. Chinchilla-optimal for 10M tokens is around 500k params, which is too tiny to learn anything interesting. For structured code with high pattern repetition, 5-10× Chinchilla is the empirical sweet spot. 33M lands in that range. Big enough to learn HTML structure, small enough to not just memorize 10M tokens at 8 epochs.
The train/val gap stayed around 0.1 nats throughout training. That means generalization, not memorization. Confirmed by manually searching the training data for the sample output above — no exact match exists.
## How to use it (Python)
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
tokenizer = AutoTokenizer.from_pretrained("Rohanify/Indensa-Coder-FrontEnd")
model = AutoModelForCausalLM.from_pretrained("Rohanify/Indensa-Coder-FrontEnd").to("cuda")
prompt = "PROMPT: a pricing card with three tiers\nHTML:\n"
input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to("cuda")
output = model.generate(
input_ids,
max_new_tokens=400,
temperature=0.8,
top_k=50,
do_sample=True,
)
print(tokenizer.decode(output[0]))
```
Then prompt it the same way. It's small enough to run anywhere.
## Things it's good at
- Bite-sized HTML blocks (landing page sections, components)
- Tailwind class usage that mostly looks right
- Common patterns: cards, headers, footers, forms, navbars, hero sections
- Producing valid HTML that opens in a browser without errors most of the time
- Being tiny and fast
## Things it's NOT good at
- Full multi-page websites (not what it was trained for)
- Modern Tailwind v3+ utility classes that didn't exist in WebSight v0.2 (the dataset uses Tailwind 2.2.19)
- Highly specific design requests with lots of constraints
- Anything outside the WebSight distribution (think generic-looking business websites)
- Replacing a real designer or v0.dev
It's a 33M model trained for 6 minutes. Manage your expectations and it'll surprise you. Expect SOTA and you'll be disappointed.
## Training data
[HuggingFaceM4/WebSight v0.2](https://huggingface.co/datasets/HuggingFaceM4/WebSight) — synthetic HTML+Tailwind pairs generated by Deepseek-Coder-33B from Mistral-7B prompts. I used the first 50k rows, filtered to samples ≤512 tokens after tokenization, and threw away the screenshots since this is a pure text task.
## Roadmap
- Train v2 on Ollama-generated prompts and components — more diversity, less synthetic feel
- More training data (the 50k cap was based on what I could download in a session)
- Maybe a 128M version if there's interest
- Modern Tailwind v3+ support
## License
MIT. Use it for whatever. Attribution appreciated but not required.
## Author
Made by Rohan ([Rohanify on HuggingFace](https://huggingface.co/Rohanify)). Also on YouTube as ElectroPlayin where I make AI tutorials and weird projects.
Built in a single session on a home PC with help from Claude (Anthropic). The architecture decisions, dataset curation, debugging cycles, frustration, and final ship are all mine. The math checks and code scaffolding came from a long conversation.
If you find this useful or just funny, drop a like on HF or come say hi. If you train a bigger version, I'd love to see it.
## Citation
If you use this for anything serious (you probably shouldn't, it's a hobby model, but):
```
@misc{indensa2026,
author = {Rohan},
title = {Indensa-Coder-FrontEnd: A 33M Parameter Prompt-to-HTML Model},
year = {2026},
publisher = {HuggingFace},
url = {https://huggingface.co/Rohanify/Indensa-Coder-FrontEnd}
}
``` |