dineth554 commited on
Commit
9775ece
·
verified ·
1 Parent(s): 74ea875

Upload folder using huggingface_hub

Browse files
Files changed (5) hide show
  1. Dockerfile +17 -0
  2. README.md +73 -7
  3. __pycache__/app.cpython-311.pyc +0 -0
  4. app.py +598 -0
  5. requirements.txt +5 -0
Dockerfile ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ FROM python:3.11-slim
2
+
3
+ WORKDIR /app
4
+
5
+ # Install dependencies
6
+ COPY requirements.txt .
7
+ RUN pip install --no-cache-dir -r requirements.txt
8
+
9
+ # Copy app files
10
+ COPY app.py .
11
+ COPY README.md .
12
+
13
+ # Expose Streamlit port
14
+ EXPOSE 7860
15
+
16
+ # Run Streamlit
17
+ CMD ["streamlit", "run", "app.py", "--server.port=7860", "--server.address=0.0.0.0"]
README.md CHANGED
@@ -1,10 +1,76 @@
 
 
 
 
 
 
 
1
  ---
2
- title: Legion Coder
3
- emoji: 😻
4
- colorFrom: red
5
- colorTo: indigo
6
- sdk: docker
7
- pinned: false
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
  ---
9
 
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
1
+ # ⚡ Legion Coder
2
+
3
+ **A 44M Parameter Transformer for Code Generation**
4
+
5
+ [![Made with by DEATH LEGION](https://img.shields.io/badge/MADE%20WITH%20BY-DEATH%20LEGION-ff0040?style=for-the-badge)](https://huggingface.co/dineth554/legion-coder-8m)
6
+ [![Powered by nvdya-kit](https://img.shields.io/badge/POWERED%20BY-nvdya--kit-7c4dff?style=for-the-badge)]()
7
+
8
  ---
9
+
10
+ ## 🚀 About
11
+
12
+ Legion Coder is a compact yet powerful 44M parameter transformer model optimized for coding tasks. Built with precision by **DEATH LEGION** and powered by **nvdya-kit**, this model delivers high-quality code generation in a lightweight package.
13
+
14
+ ## ✨ Features
15
+
16
+ - 📝 **Clean Code Generation** - PEP 8 compliant Python and more
17
+ - 🐛 **Debug Assistance** - Help identify and fix code issues
18
+ - 📚 **Code Explanation** - Understand complex programming concepts
19
+ - 💡 **Multi-language Support** - Python, JavaScript, and more
20
+ - ⚡ **Fast Inference** - Optimized for CPU deployment
21
+
22
+ ## 📊 Model Specifications
23
+
24
+ | Attribute | Value |
25
+ |-----------|-------|
26
+ | **Parameters** | 44,341,632 (~44M) |
27
+ | **Architecture** | GPT-style Transformer |
28
+ | **Hidden Size** | 576 |
29
+ | **Layers** | 13 |
30
+ | **Attention Heads** | 16 |
31
+ | **Context Length** | 1,024 tokens |
32
+ | **Vocabulary** | 16,000 tokens |
33
+ | **Format** | Safetensors |
34
+
35
+ ## 🎯 Use Cases
36
+
37
+ - **Code Completion** - Finish partial code snippets
38
+ - **Function Generation** - Create functions from descriptions
39
+ - **Debugging** - Find and fix errors in code
40
+ - **Learning** - Get explanations for programming concepts
41
+ - **Prototyping** - Quickly generate code scaffolding
42
+
43
+ ## 🛠️ Technical Details
44
+
45
+ ### Training Data
46
+ - Python code from The Stack v2 dataset
47
+ - GitHub code repositories (filtered for quality)
48
+ - Code-specific preprocessing for indentation and special tokens
49
+
50
+ ### Training Procedure
51
+ - **Optimizer**: AdamW
52
+ - **Learning Rate**: 5e-4 with cosine decay
53
+ - **Batch Size**: 4 with gradient accumulation
54
+ - **Training Steps**: 10,000
55
+ - **Precision**: float32 (CPU-optimized)
56
+
57
+ ## 📝 License
58
+
59
+ This model is released under the **MIT License**.
60
+
61
+ ## 🔗 Links
62
+
63
+ - **Model Repository**: [dineth554/legion-coder-8m](https://huggingface.co/dineth554/legion-coder-8m)
64
+ - **Space**: This Space
65
+
66
  ---
67
 
68
+ <div align="center">
69
+
70
+ ### 🔥 MADE WITH BY DEATH LEGION 🔥
71
+
72
+ **Powered by nvdya-kit**
73
+
74
+ *© 2024 DEATH LEGION. All rights reserved.*
75
+
76
+ </div>
__pycache__/app.cpython-311.pyc ADDED
Binary file (33.7 kB). View file
 
app.py ADDED
@@ -0,0 +1,598 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Legion Coder - Hugging Face Space
3
+ A powerful coding assistant powered by the Legion Coder 8M model.
4
+
5
+ MADE WITH BY DEATH LEGION
6
+ Powered by nvdya-kit
7
+ """
8
+
9
+ import os
10
+ import sys
11
+ import torch
12
+ import streamlit as st
13
+ import math
14
+ from typing import List, Dict, Tuple
15
+
16
+ # Page config with custom branding
17
+ st.set_page_config(
18
+ page_title="Legion Coder | DEATH LEGION",
19
+ page_icon="⚡",
20
+ layout="wide",
21
+ initial_sidebar_state="expanded"
22
+ )
23
+
24
+ # Enhanced Custom CSS with DEATH LEGION branding
25
+ st.markdown("""
26
+ <style>
27
+ @import url('https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@400;600;700&family=Inter:wght@400;500;600;700&display=swap');
28
+
29
+ .main {
30
+ font-family: 'Inter', sans-serif;
31
+ background: linear-gradient(135deg, #0a0a0f 0%, #1a1a2e 50%, #16213e 100%);
32
+ min-height: 100vh;
33
+ }
34
+
35
+ /* Header Styling */
36
+ .header-container {
37
+ background: linear-gradient(90deg, #ff0040 0%, #ff4081 50%, #7c4dff 100%);
38
+ padding: 2rem;
39
+ border-radius: 16px;
40
+ margin-bottom: 2rem;
41
+ box-shadow: 0 10px 40px rgba(255, 0, 64, 0.3);
42
+ }
43
+
44
+ .header-title {
45
+ font-family: 'JetBrains Mono', monospace;
46
+ font-size: 2.5rem;
47
+ font-weight: 700;
48
+ color: #ffffff;
49
+ text-shadow: 2px 2px 4px rgba(0,0,0,0.3);
50
+ margin: 0;
51
+ }
52
+
53
+ .header-subtitle {
54
+ font-size: 1.1rem;
55
+ color: rgba(255,255,255,0.9);
56
+ margin-top: 0.5rem;
57
+ }
58
+
59
+ .branding-badge {
60
+ display: inline-block;
61
+ background: rgba(0,0,0,0.3);
62
+ padding: 0.3rem 0.8rem;
63
+ border-radius: 20px;
64
+ font-size: 0.75rem;
65
+ font-weight: 600;
66
+ color: #ff4081;
67
+ margin-top: 0.5rem;
68
+ border: 1px solid rgba(255,64,129,0.3);
69
+ }
70
+
71
+ /* Chat Styling */
72
+ .stChatMessage {
73
+ padding: 1.2rem;
74
+ border-radius: 12px;
75
+ margin-bottom: 1rem;
76
+ border: 1px solid rgba(255,255,255,0.1);
77
+ }
78
+
79
+ .stChatMessage.user {
80
+ background: linear-gradient(135deg, #1e3a5f 0%, #2d5a87 100%);
81
+ margin-left: 20%;
82
+ margin-right: 0;
83
+ }
84
+
85
+ .stChatMessage.assistant {
86
+ background: linear-gradient(135deg, #2d1b4e 0%, #4a1c6b 100%);
87
+ margin-left: 0;
88
+ margin-right: 20%;
89
+ }
90
+
91
+ /* Code Block Styling */
92
+ pre {
93
+ background: #0d1117 !important;
94
+ border: 1px solid #30363d !important;
95
+ border-radius: 8px !important;
96
+ padding: 1rem !important;
97
+ }
98
+
99
+ code {
100
+ font-family: 'JetBrains Mono', monospace !important;
101
+ font-size: 0.9rem !important;
102
+ }
103
+
104
+ /* Sidebar Styling */
105
+ .css-1d391kg {
106
+ background: linear-gradient(180deg, #1a1a2e 0%, #16213e 100%);
107
+ }
108
+
109
+ /* Input Styling */
110
+ .stTextInput > div > div > input {
111
+ background: rgba(255,255,255,0.05);
112
+ border: 1px solid rgba(255,255,255,0.1);
113
+ border-radius: 8px;
114
+ color: white;
115
+ }
116
+
117
+ /* Button Styling */
118
+ .stButton > button {
119
+ background: linear-gradient(90deg, #ff0040 0%, #ff4081 100%);
120
+ color: white;
121
+ border: none;
122
+ border-radius: 8px;
123
+ padding: 0.5rem 1.5rem;
124
+ font-weight: 600;
125
+ transition: all 0.3s ease;
126
+ }
127
+
128
+ .stButton > button:hover {
129
+ transform: translateY(-2px);
130
+ box-shadow: 0 5px 20px rgba(255, 0, 64, 0.4);
131
+ }
132
+
133
+ /* Footer */
134
+ .footer {
135
+ text-align: center;
136
+ padding: 2rem;
137
+ color: rgba(255,255,255,0.5);
138
+ font-size: 0.85rem;
139
+ border-top: 1px solid rgba(255,255,255,0.1);
140
+ margin-top: 3rem;
141
+ }
142
+
143
+ .footer-brand {
144
+ color: #ff4081;
145
+ font-weight: 600;
146
+ }
147
+ </style>
148
+ """, unsafe_allow_html=True)
149
+
150
+
151
+ # System prompt
152
+ SYSTEM_PROMPT = """You are Legion Coder, an expert coding assistant powered by DEATH LEGION and nvdya-kit. Your purpose is to help users write clean, efficient, and well-documented code.
153
+
154
+ Guidelines:
155
+ - Write code that follows best practices and PEP 8 style guidelines
156
+ - Include helpful comments explaining complex logic
157
+ - Provide complete, runnable code examples
158
+ - Explain your approach before showing code when helpful
159
+ - If asked to debug, identify the issue and provide the corrected code
160
+
161
+ Always wrap code blocks in triple backticks with the appropriate language identifier."""
162
+
163
+
164
+ # Model classes (same as original)
165
+ class MultiHeadAttention(torch.nn.Module):
166
+ def __init__(self, d_model, num_heads, dropout=0.1):
167
+ super().__init__()
168
+ assert d_model % num_heads == 0
169
+ self.d_model = d_model
170
+ self.num_heads = num_heads
171
+ self.d_k = d_model // num_heads
172
+ self.W_q = torch.nn.Linear(d_model, d_model, bias=False)
173
+ self.W_k = torch.nn.Linear(d_model, d_model, bias=False)
174
+ self.W_v = torch.nn.Linear(d_model, d_model, bias=False)
175
+ self.W_o = torch.nn.Linear(d_model, d_model, bias=False)
176
+ self.dropout = torch.nn.Dropout(dropout)
177
+
178
+ def forward(self, x, mask=None):
179
+ batch_size, seq_len, _ = x.shape
180
+ Q = self.W_q(x).view(batch_size, seq_len, self.num_heads, self.d_k).transpose(1, 2)
181
+ K = self.W_k(x).view(batch_size, seq_len, self.num_heads, self.d_k).transpose(1, 2)
182
+ V = self.W_v(x).view(batch_size, seq_len, self.num_heads, self.d_k).transpose(1, 2)
183
+ scores = torch.matmul(Q, K.transpose(-2, -1)) / math.sqrt(self.d_k)
184
+ if mask is not None:
185
+ scores = scores.masked_fill(mask == 0, float('-inf'))
186
+ attn_weights = torch.nn.functional.softmax(scores, dim=-1)
187
+ attn_weights = self.dropout(attn_weights)
188
+ context = torch.matmul(attn_weights, V)
189
+ context = context.transpose(1, 2).contiguous().view(batch_size, seq_len, self.d_model)
190
+ return self.W_o(context)
191
+
192
+
193
+ class FeedForward(torch.nn.Module):
194
+ def __init__(self, d_model, d_ff, dropout=0.1):
195
+ super().__init__()
196
+ self.linear1 = torch.nn.Linear(d_model, d_ff, bias=False)
197
+ self.linear2 = torch.nn.Linear(d_ff, d_model, bias=False)
198
+ self.dropout = torch.nn.Dropout(dropout)
199
+
200
+ def forward(self, x):
201
+ x = self.linear1(x)
202
+ x = torch.nn.functional.gelu(x)
203
+ x = self.dropout(x)
204
+ x = self.linear2(x)
205
+ return x
206
+
207
+
208
+ class TransformerBlock(torch.nn.Module):
209
+ def __init__(self, d_model, num_heads, d_ff, dropout=0.1):
210
+ super().__init__()
211
+ self.attention = MultiHeadAttention(d_model, num_heads, dropout)
212
+ self.feed_forward = FeedForward(d_model, d_ff, dropout)
213
+ self.norm1 = torch.nn.LayerNorm(d_model)
214
+ self.norm2 = torch.nn.LayerNorm(d_model)
215
+ self.dropout = torch.nn.Dropout(dropout)
216
+
217
+ def forward(self, x, mask=None):
218
+ attn_output = self.attention(self.norm1(x), mask)
219
+ x = x + self.dropout(attn_output)
220
+ ff_output = self.feed_forward(self.norm2(x))
221
+ x = x + self.dropout(ff_output)
222
+ return x
223
+
224
+
225
+ class LegionCoderModel(torch.nn.Module):
226
+ def __init__(self, vocab_size=16000, d_model=576, num_layers=13, num_heads=16, d_ff=1152, max_seq_len=1024, dropout=0.1, pad_token_id=0):
227
+ super().__init__()
228
+ self.vocab_size = vocab_size
229
+ self.d_model = d_model
230
+ self.max_seq_len = max_seq_len
231
+ self.pad_token_id = pad_token_id
232
+ self.token_embedding = torch.nn.Embedding(vocab_size, d_model)
233
+ self.position_embedding = torch.nn.Embedding(max_seq_len, d_model)
234
+ self.blocks = torch.nn.ModuleList([TransformerBlock(d_model, num_heads, d_ff, dropout) for _ in range(num_layers)])
235
+ self.norm = torch.nn.LayerNorm(d_model)
236
+ self.lm_head = torch.nn.Linear(d_model, vocab_size, bias=False)
237
+ self.lm_head.weight = self.token_embedding.weight
238
+ self.dropout = torch.nn.Dropout(dropout)
239
+ self._init_weights()
240
+
241
+ def _init_weights(self):
242
+ for module in self.modules():
243
+ if isinstance(module, torch.nn.Linear):
244
+ torch.nn.init.normal_(module.weight, mean=0.0, std=0.02)
245
+ if module.bias is not None:
246
+ torch.nn.init.zeros_(module.bias)
247
+ elif isinstance(module, torch.nn.Embedding):
248
+ torch.nn.init.normal_(module.weight, mean=0.0, std=0.02)
249
+
250
+ def _create_causal_mask(self, seq_len, device):
251
+ mask = torch.triu(torch.ones(seq_len, seq_len, device=device), diagonal=1)
252
+ return mask == 0
253
+
254
+ def forward(self, input_ids, attention_mask=None, labels=None):
255
+ batch_size, seq_len = input_ids.shape
256
+ device = input_ids.device
257
+ positions = torch.arange(0, seq_len, device=device).unsqueeze(0).expand(batch_size, -1)
258
+ token_embeds = self.token_embedding(input_ids)
259
+ pos_embeds = self.position_embedding(positions)
260
+ x = self.dropout(token_embeds + pos_embeds)
261
+ causal_mask = self._create_causal_mask(seq_len, device)
262
+ if attention_mask is not None:
263
+ attention_mask = attention_mask.unsqueeze(1).unsqueeze(2)
264
+ causal_mask = causal_mask.unsqueeze(0).unsqueeze(0) & attention_mask
265
+ for block in self.blocks:
266
+ x = block(x, causal_mask)
267
+ x = self.norm(x)
268
+ logits = self.lm_head(x)
269
+ loss = None
270
+ if labels is not None:
271
+ shift_logits = logits[..., :-1, :].contiguous()
272
+ shift_labels = labels[..., 1:].contiguous()
273
+ loss_fct = torch.nn.CrossEntropyLoss(ignore_index=-100)
274
+ loss = loss_fct(shift_logits.view(-1, self.vocab_size), shift_labels.view(-1))
275
+ return {'logits': logits, 'loss': loss}
276
+
277
+ def generate(self, input_ids, max_length=100, temperature=1.0, top_k=50, top_p=0.95, pad_token_id=0, eos_token_id=2):
278
+ self.eval()
279
+ batch_size = input_ids.shape[0]
280
+ device = input_ids.device
281
+ with torch.no_grad():
282
+ for _ in range(max_length):
283
+ if input_ids.shape[1] > self.max_seq_len:
284
+ input_ids = input_ids[:, -self.max_seq_len:]
285
+ outputs = self.forward(input_ids)
286
+ logits = outputs['logits']
287
+ next_token_logits = logits[:, -1, :] / temperature
288
+ if top_k > 0:
289
+ indices_to_remove = next_token_logits < torch.topk(next_token_logits, top_k)[0][..., -1, None]
290
+ next_token_logits[indices_to_remove] = float('-inf')
291
+ if top_p < 1.0:
292
+ sorted_logits, sorted_indices = torch.sort(next_token_logits, descending=True)
293
+ cumulative_probs = torch.cumsum(torch.nn.functional.softmax(sorted_logits, dim=-1), dim=-1)
294
+ sorted_indices_to_remove = cumulative_probs > top_p
295
+ sorted_indices_to_remove[..., 1:] = sorted_indices_to_remove[..., :-1].clone()
296
+ sorted_indices_to_remove[..., 0] = 0
297
+ indices_to_remove = sorted_indices_to_remove.scatter(1, sorted_indices, sorted_indices_to_remove)
298
+ next_token_logits[indices_to_remove] = float('-inf')
299
+ probs = torch.nn.functional.softmax(next_token_logits, dim=-1)
300
+ next_token = torch.multinomial(probs, num_samples=1)
301
+ input_ids = torch.cat([input_ids, next_token], dim=1)
302
+ if (next_token == eos_token_id).all():
303
+ break
304
+ return input_ids
305
+
306
+
307
+ # Tokenizer class
308
+ class LegionCoderTokenizer:
309
+ SPECIAL_TOKENS = {
310
+ '<|pad|>': 0,
311
+ '<|eos|>': 1,
312
+ '<|unk|>': 2,
313
+ '<|system|>': 3,
314
+ '<|user|>': 4,
315
+ '<|assistant|>': 5,
316
+ '<|code|>': 6,
317
+ '<|comment|>': 7,
318
+ '<|indent|>': 8,
319
+ '<|newline|>': 9,
320
+ '<|tab|>': 10,
321
+ '<|space|>': 11,
322
+ }
323
+
324
+ def __init__(self, vocab_size=16000):
325
+ self.vocab_size = vocab_size
326
+ self.vocab = {}
327
+ self.inverse_vocab = {}
328
+ self.merges = []
329
+ self._init_special_tokens()
330
+
331
+ def _init_special_tokens(self):
332
+ for token, idx in self.SPECIAL_TOKENS.items():
333
+ self.vocab[token] = idx
334
+ self.inverse_vocab[idx] = token
335
+
336
+ def encode(self, text, add_special_tokens=True):
337
+ import re
338
+ text = text.replace('\t', ' <|tab|> ')
339
+ text = re.sub(r' {4,}', ' <|indent|> ', text)
340
+ text = text.replace('\n', ' <|newline|> ')
341
+
342
+ tokens = []
343
+ if add_special_tokens:
344
+ tokens.append(self.SPECIAL_TOKENS['<|user|>'])
345
+
346
+ words = text.split()
347
+ for word in words:
348
+ word_tokens = list(word) + ['</w>']
349
+ i = 0
350
+ while i < len(word_tokens):
351
+ for j in range(len(word_tokens), i, -1):
352
+ substr = ''.join(word_tokens[i:j])
353
+ if substr in self.vocab:
354
+ tokens.append(self.vocab[substr])
355
+ i = j
356
+ break
357
+ else:
358
+ tokens.append(self.SPECIAL_TOKENS['<|unk|>'])
359
+ i += 1
360
+
361
+ if add_special_tokens:
362
+ tokens.append(self.SPECIAL_TOKENS['<|eos|>'])
363
+
364
+ return tokens
365
+
366
+ def decode(self, token_ids, skip_special_tokens=True):
367
+ tokens = []
368
+ for idx in token_ids:
369
+ if idx in self.inverse_vocab:
370
+ token = self.inverse_vocab[idx]
371
+ if skip_special_tokens and token.startswith('<|') and token.endswith('|>'):
372
+ continue
373
+ tokens.append(token)
374
+
375
+ text = ''.join(tokens)
376
+ text = text.replace('</w>', ' ')
377
+ text = text.replace('<|newline|>', '\n')
378
+ text = text.replace('<|tab|>', '\t')
379
+ text = text.replace('<|indent|>', ' ')
380
+ text = text.replace('<|space|>', ' ')
381
+
382
+ return text.strip()
383
+
384
+ @classmethod
385
+ def load(cls, path):
386
+ import json
387
+ with open(f"{path}/vocab.json", 'r') as f:
388
+ vocab = json.load(f)
389
+ with open(f"{path}/merges.txt", 'r') as f:
390
+ merges = [tuple(line.strip().split()) for line in f if line.strip()]
391
+ with open(f"{path}/tokenizer_config.json", 'r') as f:
392
+ config = json.load(f)
393
+
394
+ tokenizer = cls(vocab_size=config['vocab_size'])
395
+ tokenizer.vocab = vocab
396
+ tokenizer.inverse_vocab = {v: k for k, v in vocab.items()}
397
+ tokenizer.merges = merges
398
+ return tokenizer
399
+
400
+
401
+ def count_parameters(model):
402
+ return sum(p.numel() for p in model.parameters() if p.requires_grad)
403
+
404
+
405
+ @st.cache_resource
406
+ def load_model():
407
+ """Load model from HuggingFace Hub."""
408
+ with st.spinner("⚡ Initializing Legion Coder..."):
409
+ try:
410
+ from huggingface_hub import hf_hub_download
411
+ import json
412
+
413
+ repo_id = "dineth554/legion-coder-8m"
414
+ cache_dir = "/tmp/model_cache"
415
+ os.makedirs(cache_dir, exist_ok=True)
416
+
417
+ # Download files
418
+ model_path = hf_hub_download(repo_id=repo_id, filename="model.safetensors", cache_dir=cache_dir)
419
+ config_path = hf_hub_download(repo_id=repo_id, filename="config.json", cache_dir=cache_dir)
420
+ vocab_path = hf_hub_download(repo_id=repo_id, filename="vocab.json", cache_dir=cache_dir)
421
+ merges_path = hf_hub_download(repo_id=repo_id, filename="merges.txt", cache_dir=cache_dir)
422
+ tok_config_path = hf_hub_download(repo_id=repo_id, filename="tokenizer_config.json", cache_dir=cache_dir)
423
+
424
+ model_dir = os.path.dirname(model_path)
425
+ st.success(f"✅ Loaded from HuggingFace Hub: {repo_id}")
426
+ except Exception as e:
427
+ st.error(f"❌ Could not load from Hub: {e}")
428
+ return None, None
429
+
430
+ # Load tokenizer
431
+ tokenizer = LegionCoderTokenizer.load(model_dir)
432
+
433
+ # Create model with expanded architecture
434
+ model = LegionCoderModel(
435
+ vocab_size=16000,
436
+ d_model=576,
437
+ num_layers=13,
438
+ num_heads=16,
439
+ d_ff=1152,
440
+ max_seq_len=1024,
441
+ dropout=0.1
442
+ )
443
+
444
+ # Load weights
445
+ from safetensors.torch import load_file
446
+ state_dict = load_file(os.path.join(model_dir, 'model.safetensors'))
447
+ model.load_state_dict(state_dict, strict=False)
448
+ model.eval()
449
+
450
+ param_count = count_parameters(model)
451
+ st.success(f"✅ Model ready! {param_count:,} parameters ({param_count/1e6:.1f}M)")
452
+
453
+ return model, tokenizer
454
+
455
+
456
+ def generate_response(model, tokenizer, messages, max_length=200):
457
+ """Generate response from the model."""
458
+ # Format conversation
459
+ prompt = ""
460
+ for msg in messages:
461
+ if msg['role'] == 'system':
462
+ prompt += f"<|system|>\n{msg['content']}\n"
463
+ elif msg['role'] == 'user':
464
+ prompt += f"<|user|>\n{msg['content']}\n"
465
+ elif msg['role'] == 'assistant':
466
+ prompt += f"<|assistant|>\n{msg['content']}\n"
467
+
468
+ prompt += "<|assistant|>\n"
469
+
470
+ # Encode
471
+ input_ids = torch.tensor([tokenizer.encode(prompt, add_special_tokens=False)], dtype=torch.long)
472
+
473
+ # Generate
474
+ with torch.no_grad():
475
+ generated = model.generate(
476
+ input_ids,
477
+ max_length=max_length,
478
+ temperature=0.8,
479
+ top_p=0.95,
480
+ top_k=50
481
+ )
482
+
483
+ # Decode
484
+ output = tokenizer.decode(generated[0].tolist(), skip_special_tokens=True)
485
+
486
+ # Extract only the assistant's response
487
+ if "<|assistant|>" in output:
488
+ parts = output.split("<|assistant|>")
489
+ if len(parts) > 1:
490
+ return parts[-1].strip()
491
+
492
+ return output.strip()
493
+
494
+
495
+ def main():
496
+ """Main Streamlit app."""
497
+ # Header with DEATH LEGION branding
498
+ st.markdown("""
499
+ <div class="header-container">
500
+ <h1 class="header-title">⚡ LEGION CODER</h1>
501
+ <p class="header-subtitle">A 44M Parameter Transformer for Code Generation</p>
502
+ <div class="branding-badge">🔥 MADE WITH BY DEATH LEGION 🔥</div>
503
+ <div class="branding-badge" style="margin-left: 10px;">⚡ POWERED BY nvdya-kit ⚡</div>
504
+ </div>
505
+ """, unsafe_allow_html=True)
506
+
507
+ # Load model
508
+ model, tokenizer = load_model()
509
+
510
+ if model is None or tokenizer is None:
511
+ st.error("❌ Failed to load model. Please check the logs.")
512
+ return
513
+
514
+ # Initialize chat history
515
+ if "messages" not in st.session_state:
516
+ st.session_state.messages = [
517
+ {"role": "system", "content": SYSTEM_PROMPT}
518
+ ]
519
+
520
+ # Display chat messages
521
+ for msg in st.session_state.messages:
522
+ if msg["role"] != "system":
523
+ with st.chat_message(msg["role"]):
524
+ st.markdown(msg["content"])
525
+
526
+ # Chat input
527
+ if prompt := st.chat_input("Ask me to write code, debug, or explain programming concepts..."):
528
+ # Add user message
529
+ st.session_state.messages.append({"role": "user", "content": prompt})
530
+
531
+ with st.chat_message("user"):
532
+ st.markdown(prompt)
533
+
534
+ # Generate response
535
+ with st.chat_message("assistant"):
536
+ with st.spinner("⚡ Legion Coder is thinking..."):
537
+ response = generate_response(model, tokenizer, st.session_state.messages)
538
+ st.markdown(response)
539
+
540
+ # Add assistant message
541
+ st.session_state.messages.append({"role": "assistant", "content": response})
542
+
543
+ # Sidebar info with enhanced branding
544
+ with st.sidebar:
545
+ st.markdown("""
546
+ <div style="text-align: center; padding: 1rem 0; border-bottom: 2px solid #ff0040; margin-bottom: 1.5rem;">
547
+ <h2 style="color: #ff0040; margin: 0; font-family: 'JetBrains Mono', monospace;">⚡ LEGION CODER</h2>
548
+ <p style="color: #888; font-size: 0.8rem; margin: 0.5rem 0;">DEATH LEGION Presents</p>
549
+ </div>
550
+ """, unsafe_allow_html=True)
551
+
552
+ st.markdown("""
553
+ ### 🚀 About
554
+ **Legion Coder** is a compact yet powerful 44M parameter transformer model
555
+ optimized for coding tasks.
556
+
557
+ ### ✨ Features
558
+ - Clean, efficient code generation
559
+ - PEP 8 compliant Python
560
+ - Helpful comments and explanations
561
+ - Debug assistance
562
+ - Multi-language support
563
+
564
+ ### 📊 Model Specs
565
+ | Attribute | Value |
566
+ |-----------|-------|
567
+ | Parameters | 44,341,632 |
568
+ | Hidden Size | 576 |
569
+ | Layers | 13 |
570
+ | Attention Heads | 16 |
571
+ | Context Length | 1,024 tokens |
572
+ """, unsafe_allow_html=True)
573
+
574
+ st.markdown("""
575
+ <div style="background: linear-gradient(135deg, #1a1a2e 0%, #16213e 100%);
576
+ padding: 1rem; border-radius: 12px; border: 1px solid #ff0040; margin-top: 1.5rem;">
577
+ <h4 style="color: #ff4081; margin: 0 0 0.5rem 0;">🔥 DEATH LEGION</h4>
578
+ <p style="color: #888; font-size: 0.85rem; margin: 0;">Crafted with precision and power by the DEATH LEGION team.</p>
579
+ <p style="color: #7c4dff; font-size: 0.8rem; margin: 0.5rem 0 0 0;">⚡ Powered by nvdya-kit</p>
580
+ </div>
581
+ """, unsafe_allow_html=True)
582
+
583
+ if st.button("🗑️ Clear Chat", use_container_width=True):
584
+ st.session_state.messages = [{"role": "system", "content": SYSTEM_PROMPT}]
585
+ st.rerun()
586
+
587
+ # Footer with branding
588
+ st.markdown("""
589
+ <div class="footer">
590
+ <p><span class="footer-brand">🔥 MADE WITH BY DEATH LEGION 🔥</span></p>
591
+ <p>⚡ Powered by nvdya-kit | Legion Coder 8M v1.0</p>
592
+ <p style="font-size: 0.75rem; color: #666;">© 2024 DEATH LEGION. All rights reserved.</p>
593
+ </div>
594
+ """, unsafe_allow_html=True)
595
+
596
+
597
+ if __name__ == '__main__':
598
+ main()
requirements.txt ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ streamlit>=1.28.0
2
+ torch>=2.0.0
3
+ safetensors>=0.4.0
4
+ huggingface-hub>=0.19.0
5
+ numpy>=1.24.0