CPU and runtime status

What works now

checkpoint/loader.py is self-contained and supports a PyTorch device argument, including cpu. It verifies the checkpoint, creates the Qwen3 architecture, decodes the T3+sparse base, applies WALB2 overlays, restores initialized RoPE buffers exactly, and returns an evaluation model.

No dense Qwen or Neutrino tensor file is opened during this process.

Verified release CPU smoke test

The exact v0.1 package was tested offline on the project's many-core server:

command: python checkpoint/loader.py smoke checkpoint --device cpu;
checkpoint hashes verified before loading;
total load time: 348.36 seconds;
one five-token forward pass: success;
logits: shape [1, 151936], all finite;
peak resident set: 16.08 GiB;
dense Qwen tensor files opened: no;
donor or Neutrino tensors opened: no.

This proves functional CPU compatibility for the reference loader. It does not establish production tokens/s or portability of that timing to a different CPU. The machine-readable record is evidence/cpu_smoke_v0_1.json.

What is not implemented yet

The loader materializes every parameter as BF16. It does not execute matmuls directly over packed ternary/sparse/binary structures. Consequently:

serialized disk size is about 2.95 GB;
resident parameters are approximately 16.4 GB in BF16;
additional working memory is required while decoding and generating;
CPU inference will use ordinary PyTorch BF16/FP32 kernels and may be slow;
this is not equivalent to a llama.cpp/BitNet.cpp deployment.

At least 32 GB system RAM is recommended for CPU experiments. More is safer for long contexts. CPU throughput is not yet a release claim.

Reference GPU measurements

On the project's physical GPU2:

verified V73 load: 374–377 seconds;
C4 evaluation: about 21,772 scored tokens/s at batch 2×2048;
MMLU evaluation after load: 87.68 seconds;
GSM8K first-300 generation evaluation: 436.53 seconds.

These are protocol-specific research timings, not production serving numbers.

Next runtime milestone

Implement direct packed kernels or export into a runtime that can express:

radix-3 ternary plane;
exactly-k8 signed sparse lane per group128;
binary low-rank correction paths;
INT3 embedding and INT4 head.

Only after that should CPU resident memory and tokens/s be advertised as a low-bit runtime result.