FINAL-Bench/POCKET-35B-GGUF
Text Generation • 35B • Updated • 533k • 84
🔄 In a Training Loop
VIDRAFT_LAB
SeaWolf-AI
AI & ML interests
Contact: arxivgpt@gmail.com
Recent Activity
updated a model about 10 hours ago
FINAL-Bench/POCKET-35B-GGUF reacted to theirpost with 😎 about 11 hours ago
🚀 We pretrained a language model from scratch on an inference-only NPU, with real backpropagation.
🤗 Model: https://huggingface.co/FINAL-Bench/VKORE-46M-NPU 📝 Technical write-up: https://huggingface.co/blog/FINAL-Bench/backpropagation-on-an-inference-only-npu-pretraini
"NPUs are inference-only, you can't train on them." We built a counterexample: VKORE-46M-NPU, trained from scratch on a first-gen (2025) inference NPU, 32 chips, no pretrained weights. A 46M decoder-only model (RoPE · SwiGLU · RMSNorm · GQA), char-level, TinyStories.
📏 Why trust it: a falling-loss curve proves nothing. So we trained the same model under identical init, data, and schedule on the NPU and on a GPU reference. · Loss trajectories correlate at 0.99 · Throughput converged to a comparable level The NPU backward pass and optimizer updates do numerically the same work as the GPU, not an approximation that merely trends down.
🧩 One accelerator now covers pretraining · LoRA · inference · quantization. We call this layer VKORE: "CUDA for NPUs."
⚖️ Public: weights, model card, NPU-vs-GPU comparison. Private (patent-pending, paper to follow): the core training technique. 46M char-level is toy scale on purpose: it proves the training path, not model quality. Larger models and real data are next.
Feedback from folks working on custom autograd backends and low-precision training kernels on non-GPU accelerators is very welcome. 🙌 reacted to theirpost with 🤗 about 11 hours ago
🚀 We pretrained a language model from scratch on an inference-only NPU, with real backpropagation.
🤗 Model: https://huggingface.co/FINAL-Bench/VKORE-46M-NPU 📝 Technical write-up: https://huggingface.co/blog/FINAL-Bench/backpropagation-on-an-inference-only-npu-pretraini
"NPUs are inference-only, you can't train on them." We built a counterexample: VKORE-46M-NPU, trained from scratch on a first-gen (2025) inference NPU, 32 chips, no pretrained weights. A 46M decoder-only model (RoPE · SwiGLU · RMSNorm · GQA), char-level, TinyStories.
📏 Why trust it: a falling-loss curve proves nothing. So we trained the same model under identical init, data, and schedule on the NPU and on a GPU reference. · Loss trajectories correlate at 0.99 · Throughput converged to a comparable level The NPU backward pass and optimizer updates do numerically the same work as the GPU, not an approximation that merely trends down.
🧩 One accelerator now covers pretraining · LoRA · inference · quantization. We call this layer VKORE: "CUDA for NPUs."
⚖️ Public: weights, model card, NPU-vs-GPU comparison. Private (patent-pending, paper to follow): the core training technique. 46M char-level is toy scale on purpose: it proves the training path, not model quality. Larger models and real data are next.
Feedback from folks working on custom autograd backends and low-precision training kernels on non-GPU accelerators is very welcome. 🙌Organizations
updated a
model about 10 hours ago
reacted to their post with 😎🤗🤗👍 about 11 hours ago
Post
606
🚀 We pretrained a language model from scratch on an inference-only NPU, with real backpropagation.
🤗 Model: FINAL-Bench/VKORE-46M-NPU 📝 Technical write-up: https://huggingface.co/blog/FINAL-Bench/backpropagation-on-an-inference-only-npu-pretraini
"NPUs are inference-only, you can't train on them." We built a counterexample: VKORE-46M-NPU, trained from scratch on a first-gen (2025) inference NPU, 32 chips, no pretrained weights. A 46M decoder-only model (RoPE · SwiGLU · RMSNorm · GQA), char-level, TinyStories.
📏 Why trust it: a falling-loss curve proves nothing. So we trained the same model under identical init, data, and schedule on the NPU and on a GPU reference. · Loss trajectories correlate at 0.99 · Throughput converged to a comparable level The NPU backward pass and optimizer updates do numerically the same work as the GPU, not an approximation that merely trends down.
🧩 One accelerator now covers pretraining · LoRA · inference · quantization. We call this layer VKORE: "CUDA for NPUs."
⚖️ Public: weights, model card, NPU-vs-GPU comparison. Private (patent-pending, paper to follow): the core training technique. 46M char-level is toy scale on purpose: it proves the training path, not model quality. Larger models and real data are next.
Feedback from folks working on custom autograd backends and low-precision training kernels on non-GPU accelerators is very welcome. 🙌
🤗 Model: FINAL-Bench/VKORE-46M-NPU 📝 Technical write-up: https://huggingface.co/blog/FINAL-Bench/backpropagation-on-an-inference-only-npu-pretraini
"NPUs are inference-only, you can't train on them." We built a counterexample: VKORE-46M-NPU, trained from scratch on a first-gen (2025) inference NPU, 32 chips, no pretrained weights. A 46M decoder-only model (RoPE · SwiGLU · RMSNorm · GQA), char-level, TinyStories.
📏 Why trust it: a falling-loss curve proves nothing. So we trained the same model under identical init, data, and schedule on the NPU and on a GPU reference. · Loss trajectories correlate at 0.99 · Throughput converged to a comparable level The NPU backward pass and optimizer updates do numerically the same work as the GPU, not an approximation that merely trends down.
🧩 One accelerator now covers pretraining · LoRA · inference · quantization. We call this layer VKORE: "CUDA for NPUs."
⚖️ Public: weights, model card, NPU-vs-GPU comparison. Private (patent-pending, paper to follow): the core training technique. 46M char-level is toy scale on purpose: it proves the training path, not model quality. Larger models and real data are next.
Feedback from folks working on custom autograd backends and low-precision training kernels on non-GPU accelerators is very welcome. 🙌
- 4 replies
replied to their post about 11 hours ago
You’re right that Pearson on two falling curves is a weak lead, and thanks for the careful read. We published the raw NPU-vs-GPU comparison precisely so it could be checked like this, and we stated the final-loss gap (0.42 vs 0.25) and attributed it to the reduced-communication (DiLoCo) sync in that same file. We did not hide it.
Two separate things got conflated in the post, and your critique separates them correctly:
Per-step numerical fidelity of the NPU backward — this is the “numerically the same work” claim, and it lives at single-chip, single-step, no sync. For this 46M model, our manual backward matches autograd to a worst-case relative gradient error of 6.6e-7 (essentially exact), and the NPU-computed gradients match the standard reference at cosine ~0.9999. That is the number we should have led with.
Multi-chip convergence — the 32-chip run used DiLoCo-style reduced-communication sync, which is deliberately NOT a per-step all-reduce. You are correct that under DiLoCo the two runs are not applying identical updates. That is exactly why the gap grows, and it is a property of the sync scheme, not of the NPU backward.
So: per-step backward = numerically faithful (6.6e-7 / cos ~0.9999); multi-chip convergence = approximate by design. We will post the exact comparison you describe — same seed, same batch, one chip, one step, no sync, gradient cosine and max|dW| per layer vs GPU, plus a GPU run on the same DiLoCo scheme for an apples-to-apples convergence curve. Appreciate you pushing on this.
reacted to their post with ❤️👀🚀🔥 about 13 hours ago
Post
606
🚀 We pretrained a language model from scratch on an inference-only NPU, with real backpropagation.
🤗 Model: FINAL-Bench/VKORE-46M-NPU 📝 Technical write-up: https://huggingface.co/blog/FINAL-Bench/backpropagation-on-an-inference-only-npu-pretraini
"NPUs are inference-only, you can't train on them." We built a counterexample: VKORE-46M-NPU, trained from scratch on a first-gen (2025) inference NPU, 32 chips, no pretrained weights. A 46M decoder-only model (RoPE · SwiGLU · RMSNorm · GQA), char-level, TinyStories.
📏 Why trust it: a falling-loss curve proves nothing. So we trained the same model under identical init, data, and schedule on the NPU and on a GPU reference. · Loss trajectories correlate at 0.99 · Throughput converged to a comparable level The NPU backward pass and optimizer updates do numerically the same work as the GPU, not an approximation that merely trends down.
🧩 One accelerator now covers pretraining · LoRA · inference · quantization. We call this layer VKORE: "CUDA for NPUs."
⚖️ Public: weights, model card, NPU-vs-GPU comparison. Private (patent-pending, paper to follow): the core training technique. 46M char-level is toy scale on purpose: it proves the training path, not model quality. Larger models and real data are next.
Feedback from folks working on custom autograd backends and low-precision training kernels on non-GPU accelerators is very welcome. 🙌
🤗 Model: FINAL-Bench/VKORE-46M-NPU 📝 Technical write-up: https://huggingface.co/blog/FINAL-Bench/backpropagation-on-an-inference-only-npu-pretraini
"NPUs are inference-only, you can't train on them." We built a counterexample: VKORE-46M-NPU, trained from scratch on a first-gen (2025) inference NPU, 32 chips, no pretrained weights. A 46M decoder-only model (RoPE · SwiGLU · RMSNorm · GQA), char-level, TinyStories.
📏 Why trust it: a falling-loss curve proves nothing. So we trained the same model under identical init, data, and schedule on the NPU and on a GPU reference. · Loss trajectories correlate at 0.99 · Throughput converged to a comparable level The NPU backward pass and optimizer updates do numerically the same work as the GPU, not an approximation that merely trends down.
🧩 One accelerator now covers pretraining · LoRA · inference · quantization. We call this layer VKORE: "CUDA for NPUs."
⚖️ Public: weights, model card, NPU-vs-GPU comparison. Private (patent-pending, paper to follow): the core training technique. 46M char-level is toy scale on purpose: it proves the training path, not model quality. Larger models and real data are next.
Feedback from folks working on custom autograd backends and low-precision training kernels on non-GPU accelerators is very welcome. 🙌
- 4 replies
posted an update about 13 hours ago
Post
606
🚀 We pretrained a language model from scratch on an inference-only NPU, with real backpropagation.
🤗 Model: FINAL-Bench/VKORE-46M-NPU 📝 Technical write-up: https://huggingface.co/blog/FINAL-Bench/backpropagation-on-an-inference-only-npu-pretraini
"NPUs are inference-only, you can't train on them." We built a counterexample: VKORE-46M-NPU, trained from scratch on a first-gen (2025) inference NPU, 32 chips, no pretrained weights. A 46M decoder-only model (RoPE · SwiGLU · RMSNorm · GQA), char-level, TinyStories.
📏 Why trust it: a falling-loss curve proves nothing. So we trained the same model under identical init, data, and schedule on the NPU and on a GPU reference. · Loss trajectories correlate at 0.99 · Throughput converged to a comparable level The NPU backward pass and optimizer updates do numerically the same work as the GPU, not an approximation that merely trends down.
🧩 One accelerator now covers pretraining · LoRA · inference · quantization. We call this layer VKORE: "CUDA for NPUs."
⚖️ Public: weights, model card, NPU-vs-GPU comparison. Private (patent-pending, paper to follow): the core training technique. 46M char-level is toy scale on purpose: it proves the training path, not model quality. Larger models and real data are next.
Feedback from folks working on custom autograd backends and low-precision training kernels on non-GPU accelerators is very welcome. 🙌
🤗 Model: FINAL-Bench/VKORE-46M-NPU 📝 Technical write-up: https://huggingface.co/blog/FINAL-Bench/backpropagation-on-an-inference-only-npu-pretraini
"NPUs are inference-only, you can't train on them." We built a counterexample: VKORE-46M-NPU, trained from scratch on a first-gen (2025) inference NPU, 32 chips, no pretrained weights. A 46M decoder-only model (RoPE · SwiGLU · RMSNorm · GQA), char-level, TinyStories.
📏 Why trust it: a falling-loss curve proves nothing. So we trained the same model under identical init, data, and schedule on the NPU and on a GPU reference. · Loss trajectories correlate at 0.99 · Throughput converged to a comparable level The NPU backward pass and optimizer updates do numerically the same work as the GPU, not an approximation that merely trends down.
🧩 One accelerator now covers pretraining · LoRA · inference · quantization. We call this layer VKORE: "CUDA for NPUs."
⚖️ Public: weights, model card, NPU-vs-GPU comparison. Private (patent-pending, paper to follow): the core training technique. 46M char-level is toy scale on purpose: it proves the training path, not model quality. Larger models and real data are next.
Feedback from folks working on custom autograd backends and low-precision training kernels on non-GPU accelerators is very welcome. 🙌
- 4 replies
upvoted an article about 13 hours ago
Article
Backpropagation on an Inference-Only NPU: Pretraining VKORE-46M-NPU From Scratch
FINAL-Bench
• • 12 published an article about 13 hours ago
Article
Backpropagation on an Inference-Only NPU: Pretraining VKORE-46M-NPU From Scratch
FINAL-Bench
• • 12 Season opening dates, Season 1 ties, and how resubmissions are scored
1
#1 opened 1 day ago
by 
1Dmeds
updated a
Space about 13 hours ago
commented on Darwin-27B-ZTC: A Single-Pass Judge and a Quantitative Look at Its Calibration about 13 hours ago
Thanks for the careful read, @Nomad-link-id . Agreed on both points: the per-type split is the number that matters, and a single shared τ is the wrong design once types behave this differently. Per-type thresholds, reported as acceptance rate at τ next to the error rate among auto-accepted items, is exactly the operational view a router needs. That is a good suggestion, and we will look at adding it.
Sampling parameters recommendation
1
#7 opened 1 day ago
by 
Cheuv
Add Darwin-27B-ZTC-v5 results (english-v1, 137/137 complete)
3
#21 opened 3 days ago
by SeaWolf-AI
liked a
model about 14 hours ago
updated a
model about 14 hours ago