VKORE-46M-NPU

The common wisdom was absolute: an inference only NPU cannot train. Not even LoRA fine tuning, let alone pretraining from scratch. Its architecture, people said, simply cannot run backpropagation.

This model is the counterexample.

A small language model pretrained entirely from scratch on an inference only NPU, using backpropagation run on the chip itself. To the best of our knowledge, this is the first publicly released model pretrained from scratch on an inference only NPU.

One line

We ran backpropagation on a chip built only for inference (Rebellions ATOM-Max) and grew a language model from zero, with no pretrained weights.

Highlights

• Pretrained from scratch on a Rebellions ATOM-Max NPU (32 chips) in a few hours.
• Throughput on par with a two GPU B200 run on the same model.
• Loss fell from 5.65 to 0.42, and the model generates coherent English stories.
• Numerically faithful backward: gradient error 6.6e-7 vs autograd, cosine about 0.9999 vs the standard reference per step.
• Achieved on a first generation inference NPU released in 2025, not the latest hardware. Newer NPUs should do even better.

Background: why an NPU

A single server of eight Blackwell class GPUs was quoted at about 1.7 billion KRW, up from the 1.2 billion KRW range not long ago. Delivery takes 40 weeks, close to ten months. Ten months is long enough to conceive and deliver a baby. On top of that there is a monthly data center colocation fee, and power draw is high. It is hard to put to work on site.

An NPU is cheaper, draws far less power, is easy to source, and can sit in an office. The chip used here is a first generation inference NPU released in 2025, not the newest silicon. The industry assumption, however, was clear. "An NPU only does inference. Its architecture cannot handle backpropagation, so training is impossible."

This model overturns that assumption. We performed backpropagation on an inference only NPU and pretrained a language model from zero.

What is first here

• NPU driven training: forward and backward (gradient) computation were performed on an inference only NPU.
• Backpropagation succeeded: gradient descent training actually ran on an inference accelerator.
• From scratch: training started from random initialization, with no pretrained weights.
• No known precedent: we did not find a comparable case in public literature.

Model specification

Item	Value
Parameters	45.8M
Architecture	Decoder only Transformer (RoPE, RMSNorm, SwiGLU, GQA)
Layers	12
Hidden width	512
Heads	8 (KV 2)
Vocabulary	256 (character level)
Context length	256
Training data	TinyStories (public dataset)
Final loss	5.65 to 0.42 (cross entropy)

Training data

• Dataset: roneneldan/TinyStories (public)
• Split used: TinyStoriesV2-GPT4 validation split
• Tokenization: character (byte) level, vocabulary 256
• Scale: about 22.5 million tokens (5,493 blocks at context 256)
• Preprocessing: used as is, with no extra processing. The training data is fully public, so anyone can reproduce training on the same data.

Training setup

• Hardware: Rebellions ATOM-Max NPU, 32 chips (an inference only accelerator)
• Under identical conditions (initialization, data, schedule), the loss trajectory and throughput matched a B200 GPU run. See the comparison report in this repository.
• The core technique that enables backpropagation based training on an inference only NPU was developed in house and is under patent application.
• The specific from scratch training method on the NPU will be disclosed in a forthcoming paper.

Generation example

Prompt: Once upon a time

Output:

Once upon a time, there was a little boy named Tim. Tim loved to play with his toy car. One day, he saw a big box in his room. Tim was very curious about the box.

Usage

import torch
state = torch.load("tiny_ckpt.pt", map_location="cpu")

A character level tokenizer is used (byte values 0 to 255).

Numerical fidelity (per step)

The claim that the NPU backward does the same work as a GPU is tested at the gradient level, not at the loss curve. On this 46M model, our hand written backward matches PyTorch autograd to a worst case relative gradient error of 6.6e-7, which is essentially exact. The NPU computed gradients match the standard reference at cosine about 0.9999 per step (single chip, no sync). This is the evidence for numerical equivalence.

NPU vs B200 (loss curve, secondary)

As a secondary check, the 32 chip run and a B200 GPU run under identical init, data, and schedule follow the same loss trajectory. A high correlation between two falling loss curves is weak on its own, so we do not lead with it. The final loss gap (NPU 0.42 vs GPU 0.25) comes from the reduced communication (DiLoCo style) sync used across 32 chips, which is deliberately not a per step all reduce. Per step numerical fidelity (above) and multi chip convergence are separate properties. See comparison/ for the full data.

Scope

This is a small research demonstration model. It is suited to generating short English stories at the TinyStories level.

Credits

VIDRAFT. Part of the VKORE project (NPU acceleration family). Published under FINAL-Bench.