Upload 12 files

.gitattributes

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+axl_comment-q4_k_m_v2.gguf filter=lfs diff=lfs merge=lfs -text
+axl-comment-f16.gguf filter=lfs diff=lfs merge=lfs -text
+axl-comment-q4_k_m.gguf filter=lfs diff=lfs merge=lfs -text

Modelfile

@@ -0,0 +1,14 @@
+FROM ./axl_comment_5m-f16.gguf
+
+TEMPLATE """{{ .System }}
+
+User: {{ .Prompt }}
+Assistant: """
+
+SYSTEM """You are AXL-Comment-5M, a code generation assistant built by Koinic."""
+
+PARAMETER temperature 0.8
+PARAMETER top_k 40
+PARAMETER top_p 0.9
+PARAMETER repeat_penalty 1.1
+PARAMETER num_ctx 256

README.md

@@ -1,3 +1,190 @@
 ---
 license: apache-2.0
+language:
+  - code
+tags:
+  - code-generation
+  - multi-scale-transformer
+  - cpu-optimized
+  - koinic
+  - pytorch
+  - llama
+  - gguf
+  - byte-level
+  - commenting
+pipeline_tag: text-generation
+library_name: transformers
+datasets:
+  - bigcode/starcoderdata
+  - theblackcat102/evol-codealpaca-v1
+widget:
+  - text: "Code:\ndef quicksort(arr):\n    if len(arr) <= 1: return arr\nCommented:"
+model-index:
+  - name: AXL-Comment-5M
+    results:
+      - task:
+          type: text-generation
+        metrics:
+          - name: Perplexity (byte-level)
+            type: perplexity
+            value: 1.01
 ---
+
+# AXL-Comment-5M
+
+Code commenting. 7.2M params. PPL 1.01. Context 512 bytes. Part of the AXL model family by [KoinicLabs](https://huggingface.co/KoinicLabs).
+
+## Model Details
+
+| Property | Value |
+|----------|-------|
+| Developed by | [KoinicLabs](https://huggingface.co/KoinicLabs) |
+| Architecture | Multi-Scale Transformer |
+| Parameters | 7M |
+| Optimizer | Lion |
+| Attention | SDPA |
+| Vocab Size | 258 (byte-level) |
+| Context Window | 512 bytes |
+| d_model | 192 |
+| Attention Heads | 3 |
+| Layers per Scale | 3 |
+| Downsample Factors | [1, 2, 4] |
+| License | Apache 2.0 |
+
+### Sources
+
+- **Repository:** [GitHub](https://github.com/Koinic/AXL)
+- **Organization:** [KoinicLabs](https://huggingface.co/KoinicLabs)
+
+## Uses
+
+### Direct Use
+
+Code commenting.
+
+```python
+import torch
+from multiscale_transformer.model.model import MultiScaleTransformer
+from multiscale_transformer.training.tokenizer import ByteTokenizer
+ckpt = torch.load("axl_comment_5m.pt", map_location="cpu")
+model = MultiScaleTransformer(config)
+model.load_state_dict(ckpt["model_state_dict"])
+model.eval()
+tokenizer = ByteTokenizer()
+ids = torch.tensor([tokenizer.encode("def hello():")], dtype=torch.long)
+with torch.no_grad():
+    out = model.generate(ids, max_new_tokens=50, temperature=0.8)
+print(tokenizer.decode(out[0].tolist()))
+```
+
+### Out-of-Scope Use
+
+Not for production code generation. Not for non-code NLP tasks. For integration with tools like Continue.dev, LlamaIndex, or LangChain, use the Python API server which provides OpenAI-compatible endpoints.
+
+## Bias, Risks, and Limitations
+
+Byte-level perplexity is not comparable to BPE-level perplexity. Max context 512 bytes. Note: GGUF files for Ollama use a simplified single-stack encoder. For full AXL quality, use the Python API server.
+
+### Recommendations
+
+- Use for prototyping and experimentation, not production code generation.
+- Byte-level perplexity (258 vocab) is not comparable to BPE-level perplexity (32K vocab).
+- For better results, use the Lion-optimized version if available.
+
+## Training Details
+
+### Training Data
+
+Retrained with Lion on 20MB commenting pairs. 263 steps in 10 min.
+
+### Preprocessing
+
+Byte-level tokenization with vocabulary size 258 (256 bytes + BOS + EOS). No vocabulary training required.
+
+### Speeds, Sizes, Times
+
+| Metric | Value |
+|--------|-------|
+| Training Steps | 263 |
+| Training Time | 10 min |
+| Final Loss | 0.1476 |
+
+## Evaluation
+
+### Metrics
+
+Perplexity on held-out Python code using byte-level tokenization.
+
+### Results
+
+| Metric | Value |
+|--------|-------|
+| Perplexity (byte-level) | 1.01 |
+| Final Loss | 0.1476 |
+| Training Steps | 263 |
+| Training Time | 10 min |
+
+**Summary:** Adds inline comments to explain code logic.
+
+## Environmental Impact
+
+| Property | Value |
+|----------|-------|
+| Hardware | AMD Ryzen 5 5600G |
+| Hours Used | 0.167 |
+| Carbon Emitted | 0.0070 kg CO2 |
+| Cloud Provider | None (local CPU) |
+
+## Technical Specifications
+
+### Model Architecture
+
+Multi-Scale Transformer with three parallel encoder stacks at resolution scales 1x, 2x, and 4x. Cross-scale attention connects all scale pairs. Adaptive gating fusion. SwiGLU feed-forward. RoPE positional encoding.
+
+### Compute Infrastructure
+
+| Property | Value |
+|----------|-------|
+| Hardware | AMD Ryzen 5 5600G (6 cores, 12 threads) |
+| RAM | 16 GB |
+| GPU | None (CPU-only) |
+
+## Citation
+
+```bibtex
+@misc{axl_2026,
+  title={AXL: AXL-Comment-5M - Multi-Scale Transformer for CPU Code Generation},
+  author={Koinic},
+  year={2026},
+  url={https://huggingface.co/KoinicLabs}
+}
+```
+
+## How to Get Started
+
+### With Ollama
+
+```bash
+ollama create axl-comment-5m -f Modelfile
+ollama run axl-comment-5m "def fibonacci():"
+```
+
+### With Python
+
+```python
+import torch
+from multiscale_transformer.model.config import load_config
+from multiscale_transformer.model.model import MultiScaleTransformer
+from multiscale_transformer.training.tokenizer import ByteTokenizer
+config = load_config("config.json")
+model = MultiScaleTransformer(config)
+ckpt = torch.load("axl_comment_5m.pt", map_location="cpu")
+model.load_state_dict(ckpt["model_state_dict"])
+model.eval()
+tokenizer = ByteTokenizer()
+prompt = "def fibonacci():"
+ids = torch.tensor([tokenizer.encode(prompt)], dtype=torch.long)
+with torch.no_grad():
+    out = model.generate(ids, max_new_tokens=100, temperature=0.8, top_k=40)
+print(tokenizer.decode(out[0].tolist()))
+```

axl-comment-f16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88cc145b217319b19d85d3e929f0b669ebffcd608c576d8da59c823bfd31691b
+size 14483552

axl-comment-q4_k_m.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88cc145b217319b19d85d3e929f0b669ebffcd608c576d8da59c823bfd31691b
+size 14483552

axl_comment-q4_k_m_v2.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cacc69f96c6f9d78ac231220531ec12dc8d395c200ab7f3fbaf054e465c0e235
+size 4754400

axl_comment.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c81e6ba4d7f6a25cdaabe62b0cb97cab5b1821bbf93cc3beb0ed04d6143a95b9
+size 28806303

config.json

@@ -0,0 +1,32 @@
+{
+  "model_type": "multiscale_transformer",
+  "architectures": [
+    "MultiScaleForCausalLM"
+  ],
+  "vocab_size": 258,
+  "d_model": 192,
+  "n_heads": 3,
+  "d_ff": 512,
+  "n_layers_per_scale": 3,
+  "n_cross_attn_layers": 1,
+  "max_seq_len": 512,
+  "dropout": 0.0,
+  "bias": false,
+  "rope_theta": 10000.0,
+  "downsample_factors": [
+    1,
+    2,
+    4
+  ],
+  "num_parameters": 6732096,
+  "training_results": {
+    "model": "AXL-Comment-5M",
+    "params": 7182144,
+    "steps": 246,
+    "time": 60.12719678878784,
+    "final_loss": 0.009364727884531021,
+    "perplexity": 1.01,
+    "max_seq_len": 512,
+    "context_window": "512 bytes"
+  }
+}

generation_config.json

@@ -0,0 +1,8 @@
+{
+  "max_new_tokens": 256,
+  "temperature": 0.8,
+  "top_k": 40,
+  "top_p": 0.9,
+  "repetition_penalty": 1.1,
+  "do_sample": true
+}

index.html

@@ -0,0 +1,95 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="UTF-8">
+<meta name="viewport" content="width=device-width,initial-scale=1.0">
+<title>AXL-Comment-5M - AXL</title>
+<style>*{margin:0;padding:0;box-sizing:border-box}
+body{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,sans-serif;background:#0d1117;color:#c9d1d9;line-height:1.6}
+a{color:#58a6ff;text-decoration:none}a:hover{text-decoration:underline}
+.hero{padding:40px 20px;text-align:center;border-bottom:1px solid #30363d;background:linear-gradient(135deg,#0d1117,#161b22,#0d1117)}
+.hero h1{font-size:2.2rem;color:#fff;letter-spacing:-1px}
+.cat{display:inline-block;padding:3px 12px;border-radius:12px;font-size:.75rem;font-weight:600;margin-bottom:12px}
+.cat.Lion{background:#1f3a5f;color:#4285f4}
+.cat.SGD{background:#3d1f1f;color:#f85149}
+.cat.Specialized{background:#2d1b69;color:#bb86fc}
+.desc{color:#8b949e;font-size:.95rem;max-width:600px;margin:12px auto 0}
+.ms{display:flex;flex-wrap:wrap;gap:12px;justify-content:center;padding:24px 20px}
+.mc{background:#161b22;border:1px solid #30363d;border-radius:10px;padding:16px 24px;text-align:center;min-width:120px}
+.v{font-size:1.5rem;font-weight:700;color:#fff}.l{font-size:.75rem;color:#8b949e;margin-top:2px}
+.tabs{max-width:800px;margin:0 auto;padding:0 20px}
+.tabs>input[type=radio]{display:none}
+.tl{display:inline-block;background:#21262d;border:1px solid #30363d;color:#8b949e;padding:7px 16px;border-radius:8px;cursor:pointer;font-size:.85rem;margin:0 4px 16px;transition:all .2s}
+.tl:hover{background:#30363d;color:#c9d1d9}
+.p{display:none;background:#161b22;border:1px solid #30363d;border-radius:12px;padding:24px;margin-bottom:24px}
+#t1:checked~.p1,#t2:checked~.p2,#t3:checked~.p3,#t4:checked~.p4{display:block}
+#t1:checked+label[for=t1],#t2:checked+label[for=t2],#t3:checked+label[for=t3],#t4:checked+label[for=t4]{background:#4285f4;color:#fff;border-color:#4285f4}
+table{width:100%;border-collapse:collapse}
+th{text-align:left;color:#8b949e;font-size:.8rem;padding:8px 12px;border-bottom:1px solid #21262d;font-weight:600}
+td{padding:8px 12px;font-size:.9rem;border-bottom:1px solid #21262d}
+pre{background:#0d1117;padding:14px;border-radius:8px;overflow-x:auto;margin:12px 0}
+code{color:#c9d1d9;font-size:.82rem;line-height:1.5}
+.note{background:#21262d;border-left:3px solid #4285f4;padding:12px 16px;border-radius:0 8px 8px 0;margin:12px 0;font-size:.85rem;color:#8b949e}
+.story{font-size:.9rem;color:#8b949e;line-height:1.6;margin:8px 0}
+.back{text-align:center;padding:24px 20px 40px}
+.back a{color:#58a6ff;font-size:.9rem}
+@media(max-width:768px){.hero h1{font-size:1.6rem}.ms{flex-direction:column;align-items:center}.mc{min-width:200px}}</style>
+</head>
+<body>
+<div class="hero">
+<div class="cat Lion">Lion Optimized</div>
+<h1>AXL-Comment-5M</h1>
+<p class="desc">Code commenting. 7.2M params. PPL 1.16. Context 2048 bytes.</p>
+</div>
+<div class="ms">
+<div class="mc"><div class="v">7M</div><div class="l">Parameters</div></div>
+<div class="mc"><div class="v">1.16</div><div class="l">Perplexity</div></div>
+<div class="mc"><div class="v">10 min</div><div class="l">Training</div></div>
+<div class="mc"><div class="v">14 MB</div><div class="l">GGUF</div></div>
+</div>
+<div class="tabs">
+<input type="radio" name="t" id="t1" checked><label for="t1" class="tl">Specs</label>
+<input type="radio" name="t" id="t2"><label for="t2" class="tl">Training</label>
+<input type="radio" name="t" id="t3"><label for="t3" class="tl">Usage</label>
+<input type="radio" name="t" id="t4"><label for="t4" class="tl">Download</label>
+<div class="p p1">
+<table>
+<tr><th>Property</th><th>Value</th></tr>
+<tr><td>Architecture</td><td>Multi-Scale Transformer</td></tr>
+<tr><td>d_model</td><td>?</td></tr>
+<tr><td>Attention Heads</td><td>?</td></tr>
+<tr><td>Layers per Scale</td><td>?</td></tr>
+<tr><td>Context Window</td><td>2048 bytes</td></tr>
+<tr><td>Downsample Factors</td><td>[1, 2, 4]</td></tr>
+<tr><td>Vocab Size</td><td>258 (byte-level)</td></tr>
+<tr><td>Optimizer</td><td>Lion</td></tr>
+</table>
+</div>
+<div class="p p2">
+<div class="story">Retrained with Lion on 20MB commenting pairs. 263 steps in 10 min.</div>
+<table>
+<tr><th>Metric</th><th>Value</th></tr>
+<tr><td>Final Loss</td><td>0.1476</td></tr>
+<tr><td>Perplexity</td><td>1.16</td></tr>
+<tr><td>Training Steps</td><td>263</td></tr>
+<tr><td>Training Time</td><td>10 min</td></tr>
+</table>
+</div>
+<div class="p p3">
+<h3 style="color:#fff;margin-bottom:12px">Usage</h3>
+<pre><code>ollama create axl-comment-5m -f Modelfile
+ollama run axl-comment-5m "def fibonacci():"</code></pre>
+<div class="note">Adds inline comments to explain code logic.</div>
+</div>
+<div class="p p4">
+<table>
+<tr><th>File</th><th>Size</th><th>Format</th></tr>
+<tr><td>F16 GGUF</td><td>14 MB</td><td>Full precision</td></tr>
+<tr><td>Q4_K_M GGUF</td><td>14 MB</td><td>4-bit quantized</td></tr>
+</table>
+<div class="note" style="margin-top:16px">GGUF files work with Ollama and llama.cpp. Q4_K_M is about 3x smaller than F16.</div>
+</div>
+</div>
+<div class="back"><a href="../">← All AXL Models</a></div>
+</body>
+</html>

results.json

@@ -0,0 +1,10 @@
+{
+  "model": "AXL-Comment-5M",
+  "params": 7182144,
+  "steps": 263,
+  "time": 600.2396535873413,
+  "final_loss": 0.14764484763145447,
+  "perplexity": 1.16,
+  "optimizer": "Lion",
+  "max_seq_len": 2048
+}

special_tokens_map.json

@@ -0,0 +1,6 @@
+{
+  "pad_token": "[PAD]",
+  "bos_token": "[BOS]",
+  "eos_token": "[EOS]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,8 @@
+{
+  "tokenizer_class": "ByteTokenizer",
+  "vocab_size": 258,
+  "pad_token": "[PAD]",
+  "bos_token": "[BOS]",
+  "eos_token": "[EOS]",
+  "unk_token": "[UNK]"
+}