EdgeRock-4B-NF4

EdgeRock's constant-memory reasoning architecture in a 2.41 GB weight package.

This release uses bitsandbytes NF4 with double-quantized scales (ordinary scales for the shared embedding/output head), with BF16 computation. It quantizes the large backbone and RMA projection/writer matrices and stores the tied embedding/output head once. RMA admission networks, small recurrent gates and normalization parameters retain their source precision. The recurrent state, local-attention window, anchor capacity and block-commit behavior are unchanged.

Source: Rock Matrix's EdgeRock, frozen at a6a53a5573f1. This package validates text inference through the original PyTorch architecture.

Load and generate

Download this repository, install requirements.txt in a fresh environment, and use the included load_compact.py loader. The pinned Transformers wheel supplies the implementation required by EdgeRock. AutoModelForCausalLM.from_pretrained alone does not load this compact storage layout.

import sys
from huggingface_hub import snapshot_download
from transformers import AutoTokenizer

folder = snapshot_download("hkust-aaron/EdgeRock-4B-NF4")
sys.path.insert(0, folder)
from load_compact import load_compact

tokenizer = AutoTokenizer.from_pretrained(folder, trust_remote_code=True)
model = load_compact(folder)
messages = [{"role": "user", "content": "Calculate 17 * 23."}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt", add_special_tokens=False).to("cuda")
output = model.generate(**inputs, max_new_tokens=2048, do_sample=False)
print(tokenizer.decode(output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))

The validated environment uses an NVIDIA Hopper GPU. The input embedding and output head share one packed 4-bit tensor. Input lookup dequantizes only the requested embedding rows.

Size and validation

Measurement	BF16 source	This release
Weight files, decimal GB	9.156	2.407
Loaded CUDA tensor allocation, GiB	—	2.311
Peak CUDA tensor allocation in the six smoke checks, GiB	—	2.575
Teacher-forced NLL, 32 examples	0.9892	1.0228
Correct complete answers, 16 separate GSM8K examples	15/16	16/16
Long-context update/key-disambiguation checks	2/2	2/2

Saving and reloading produced identical probe logits. Shared embedding lookup matches the corresponding dequantized output-head rows bit for bit; 289 unquantized tensors match the original checkpoint exactly.

The NLL screen uses GSM8K test rows 0–31, the final up-to-64 tokens of sequences truncated to 512 tokens. Independent generation uses rows 32–47, greedy decoding and a 4,096-token output limit, plus two synthetic memory checks with 9,664 and 10,462 input tokens. A correct complete answer must match the expected answer, close its thinking section, and end with an EOS token within the budget; a correct number inside truncated reasoning is not counted. These small checks compare conversion candidates; they are not full benchmark scores and do not establish unchanged benchmark accuracy. Exact prompts, outputs and settings are in validation.json. Serving throughput has not been benchmarked for this package.

Compared 4-bit configurations

All candidates include the large RMA matrices and shared quantized embedding/head. e4 and e8 identify embedding/head precision; r8 raises the RMA matrices to 8-bit; recurrent8 raises the recurrent-layer projections to 8-bit. NLL uses the same 32-example screen; lower is better. The memory column reports the same two long-context update/key-disambiguation tasks. NF4 passed both; the HQQ candidates remain unpublished after identifier-copying or update errors in this screen.

Candidate	Weight files, GB	Loaded tensors, GiB	NLL	Memory checks
hqq4-g128-e4	2.448	2.348	1.0340	1/2
hqq4-g128-e8	2.766	2.644	1.0326	1/2
hqq4-g128-r8-e4	2.631	2.513	1.0337	1/2
hqq4-g128-recurrent8-e4	2.951	2.823	0.9972	0/2
hqq4-g256-e4	2.376	2.281	1.0406	1/2
hqq4-g32-e4	2.876	2.778	1.0644	1/2
hqq4-g64-e4	2.590	2.481	1.0603	1/2
nf4-full	2.407	2.311	1.0228	2/2

Runtime and license

The visual frontend and optimized serving adapters remain in the source repository; they are not bundled or validated by this text package. GGUF/Ollama requires an implementation of EdgeRock's RMA state and update rules, so this Safetensors package is not an Ollama model.

Weights retain the EdgeRock Model License 1.0, code retains Apache-2.0, and source notices are preserved in NOTICE. Changes comprise weight quantization, shared packed embedding lookup, the compact loader, and packaged chat/generation settings.