Rohanify commited on
Commit
0de0f67
·
verified ·
1 Parent(s): 1ca1987

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +170 -0
README.md CHANGED
@@ -1,3 +1,173 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ language:
4
+ - en
5
+ tags:
6
+ - html
7
+ - css
8
+ - tailwind
9
+ - code-generation
10
+ - from-scratch
11
+ - small-model
12
+ - text-generation
13
+ library_name: transformers
14
+ pipeline_tag: text-generation
15
  ---
16
+
17
+ # Indensa-Coder-FrontEnd
18
+
19
+ `This is a V1 model. Can be used. V2 is coming soon!`
20
+
21
+ A 33M parameter decoder-only transformer that takes a text prompt and spits out a bite-sized HTML + Tailwind CSS block. Trained from scratch on my desk in 6 minutes.
22
+
23
+ This is not a SOTA model. It's not trying to be. It's a small honest model that learned the shape of web components from a curated dataset, and now it can make new ones.
24
+
25
+ If you wanted GPT-4-level web design, this isn't it. If you wanted a tiny model you can actually run anywhere, including on Ollama with a fast prompt-to-HTML use case, you're in the right place.
26
+
27
+ |---|---|
28
+ | ![](example.png) | ![](example1.png) |
29
+
30
+ ## What it does
31
+
32
+ You give it something like:
33
+
34
+ ```
35
+ PROMPT: a hero section for a SaaS landing page
36
+ HTML:
37
+ ```
38
+
39
+ And it continues with a real HTML document using Tailwind classes. The output is bite-sized (≤512 tokens by design), so think landing page sections, pricing cards, contact forms, navbars — not entire 10-page websites.
40
+
41
+ ## Sample output
42
+
43
+ Prompt: *"a publishing house landing page"*
44
+
45
+ ```html
46
+ <html>
47
+ <link href="https://cdn.jsdelivr.net/npm/tailwindcss@2.2.19/dist/tailwind.min.css" rel="stylesheet">
48
+ <body class="bg-gray-100">
49
+ <header class="bg-white p-4">
50
+ <h1 class="text-2xl font-bold">Publishing House</h1>
51
+ </header>
52
+ <section class="p-4">
53
+ <h2 class="text-xl font-bold">Welcome to our Publishing House</h2>
54
+ <p>We are a leading publisher of high-quality books...</p>
55
+ </section>
56
+ ...
57
+ <footer class="bg-white p-4">
58
+ <p>© 2022 Publishing House. All rights reserved.</p>
59
+ </footer>
60
+ </body>
61
+ </html>
62
+ ```
63
+
64
+ That output was checked manually against the training set — no 100% match exists in training data. So it's actually generating, not parroting.
65
+
66
+ ## The honest details
67
+
68
+ | Thing | Value |
69
+ |---|---|
70
+ | Parameters | 33.6M |
71
+ | Architecture | Decoder-only transformer |
72
+ | Layers | 8 |
73
+ | d_model | 512 |
74
+ | Heads | 8 (head_dim 64) |
75
+ | FFN dim | 2048 |
76
+ | Block size | 1024 |
77
+ | Tokenizer | Custom ByteLevel BPE, vocab 16,000 |
78
+ | Training data | ~50k samples from HuggingFaceM4/WebSight v0.2 (Tailwind) |
79
+ | Training tokens | 10.17M (8 epochs = ~81M effective) |
80
+ | Best val loss | 0.4772 |
81
+ | Training time | 7.4 minutes |
82
+ | Hardware | RTX 5080 16GB |
83
+ | Precision | bf16 mixed |
84
+
85
+ ## Why custom BPE
86
+
87
+ GPT-2's BPE was trained on web prose, not code. It splits stuff like `text-3xl` into 4 tokens and `bg-gradient-to-r` into 7 tokens. My custom 16k BPE trained on WebSight compresses these to 1-2 tokens because Tailwind patterns are everywhere in the corpus.
88
+
89
+ Net result: roughly 1.5-2× better compression on the same content, which means more useful content fits inside the 512-token window.
90
+
91
+ ## Why 33M params
92
+
93
+ I did the math. Chinchilla-optimal for 10M tokens is around 500k params, which is too tiny to learn anything interesting. For structured code with high pattern repetition, 5-10× Chinchilla is the empirical sweet spot. 33M lands in that range. Big enough to learn HTML structure, small enough to not just memorize 10M tokens at 8 epochs.
94
+
95
+ The train/val gap stayed around 0.1 nats throughout training. That means generalization, not memorization. Confirmed by manually searching the training data for the sample output above — no exact match exists.
96
+
97
+ ## How to use it (Python)
98
+
99
+ ```python
100
+ from transformers import AutoTokenizer, AutoModelForCausalLM
101
+ import torch
102
+
103
+ tokenizer = AutoTokenizer.from_pretrained("Rohanify/Indensa-Coder-FrontEnd")
104
+ model = AutoModelForCausalLM.from_pretrained("Rohanify/Indensa-Coder-FrontEnd").to("cuda")
105
+
106
+ prompt = "PROMPT: a pricing card with three tiers\nHTML:\n"
107
+ input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to("cuda")
108
+ output = model.generate(
109
+ input_ids,
110
+ max_new_tokens=400,
111
+ temperature=0.8,
112
+ top_k=50,
113
+ do_sample=True,
114
+ )
115
+ print(tokenizer.decode(output[0]))
116
+ ```
117
+
118
+ Then prompt it the same way. It's small enough to run anywhere.
119
+
120
+ ## Things it's good at
121
+
122
+ - Bite-sized HTML blocks (landing page sections, components)
123
+ - Tailwind class usage that mostly looks right
124
+ - Common patterns: cards, headers, footers, forms, navbars, hero sections
125
+ - Producing valid HTML that opens in a browser without errors most of the time
126
+ - Being tiny and fast
127
+
128
+ ## Things it's NOT good at
129
+
130
+ - Full multi-page websites (not what it was trained for)
131
+ - Modern Tailwind v3+ utility classes that didn't exist in WebSight v0.2 (the dataset uses Tailwind 2.2.19)
132
+ - Highly specific design requests with lots of constraints
133
+ - Anything outside the WebSight distribution (think generic-looking business websites)
134
+ - Replacing a real designer or v0.dev
135
+
136
+ It's a 33M model trained for 6 minutes. Manage your expectations and it'll surprise you. Expect SOTA and you'll be disappointed.
137
+
138
+ ## Training data
139
+
140
+ [HuggingFaceM4/WebSight v0.2](https://huggingface.co/datasets/HuggingFaceM4/WebSight) — synthetic HTML+Tailwind pairs generated by Deepseek-Coder-33B from Mistral-7B prompts. I used the first 50k rows, filtered to samples ≤512 tokens after tokenization, and threw away the screenshots since this is a pure text task.
141
+
142
+ ## Roadmap
143
+
144
+ - Train v2 on Ollama-generated prompts and components — more diversity, less synthetic feel
145
+ - More training data (the 50k cap was based on what I could download in a session)
146
+ - Maybe a 128M version if there's interest
147
+ - Modern Tailwind v3+ support
148
+
149
+ ## License
150
+
151
+ MIT. Use it for whatever. Attribution appreciated but not required.
152
+
153
+ ## Author
154
+
155
+ Made by Rohan ([Rohanify on HuggingFace](https://huggingface.co/Rohanify)). Also on YouTube as ElectroPlayin where I make AI tutorials and weird projects.
156
+
157
+ Built in a single session on a home PC with help from Claude (Anthropic). The architecture decisions, dataset curation, debugging cycles, frustration, and final ship are all mine. The math checks and code scaffolding came from a long conversation.
158
+
159
+ If you find this useful or just funny, drop a like on HF or come say hi. If you train a bigger version, I'd love to see it.
160
+
161
+ ## Citation
162
+
163
+ If you use this for anything serious (you probably shouldn't, it's a hobby model, but):
164
+
165
+ ```
166
+ @misc{indensa2026,
167
+ author = {Rohan},
168
+ title = {Indensa-Coder-FrontEnd: A 33M Parameter Prompt-to-HTML Model},
169
+ year = {2026},
170
+ publisher = {HuggingFace},
171
+ url = {https://huggingface.co/Rohanify/Indensa-Coder-FrontEnd}
172
+ }
173
+ ```