int-llm coordinate permutation: reversible model layouts for Q16.48 and SafeTensors

This is the third repository in my int-llm series.

The original int-llm project built a deliberately wide Q16.48 integer reference for a tiny character GPT and TinyLlama-1.1B. The int-llm-precision-ladder follow-up asked how far stored weights could move down in precision while still matching that oracle on explicit regression gates.

This project asks a different question:

How much of a Llama checkpoint can be reordered into an equivalent layout, and how can that transformation be checked without hiding numerical drift?

The repository is:

https://github.com/nmicic/int-llm-coordinate-permutation

It implements reversible permutations of vocabulary rows, SwiGLU neurons, GQA-aware attention heads, and the global residual hidden basis.

Although the project originated in int-llm, the coordinate transformations are not integer-specific. The repository therefore has two paths:

The floating-point path is not an export from the integer model. It moves the stored BF16, F16, or F32 elements directly. The integer path remains useful because it provides stronger exact-execution evidence.

How it started: a token namespace

The first prototype was much smaller than the final repository.

Let P be a bijection from the original logical token IDs to a new physical token namespace:

physical_id = P(logical_id)
logical_id  = P^-1(physical_id)

For a Llama checkpoint, token IDs are axes of the input embedding and output head. Those rows move together:

new_embedding[P(i)] = old_embedding[i]
new_lm_head[P(i)]   = old_lm_head[i]

Input IDs are mapped through P, while sampled or decoded output IDs are mapped back through P^-1. With that boundary translation, the external model interface remains in the original logical token namespace.

On TinyLlama, this first experiment covered all 32,000 rows in both model.embed_tokens.weight and lm_head.weight. The retained map changed the physical position of 31,999 token IDs, and the verifier checked 64,000 prescribed rows spanning 1,048,576,000 bytes in the Q16.48 checkpoint. The other 7,751,830,496 bytes (about 7.22 GiB), including all 22 transformer layers, had to remain byte-identical.

The dtype-independent observation

The original implementation used the Q16.48 .mgw format because that was the model representation already available in int-llm. At first, it was natural to think of the operation as:

FP checkpoint -> Q16.48 -> permutation -> Q16.48 -> FP checkpoint

But the permutation itself performs no arithmetic. It only moves complete stored elements.

A BF16 element can be copied as two opaque bytes. The same is true for F16 and F32 elements with their respective element widths. There is no reason to decode a value, convert it to integer, round it, or quantize it merely to move its row, column, vector entry, or head block.

The direct route is therefore sufficient:

BF16/F16/F32 SafeTensors -> coordinate movement -> BF16/F16/F32 SafeTensors

For the vocabulary-only TinyLlama result, the two routes even closed to the same exported integer artifact:

BF16 -> Q16.48 MGW -> permuted MGW
BF16 -> permuted BF16 -> Q16.48 MGW

Both produced the same complete permuted MGW file. Passing through Q16.48 added no benefit to the transformation itself.

That answered the dtype question, but the first result was still narrow. A reasonable objection was that it only renamed token IDs and moved two matrices. It did not rearrange the internal transformer coordinates.

That objection became the next experiment.

From token IDs to model coordinates

The general idea is simple. If an intermediate vector is stored in a permuted basis, every producer and consumer of that vector must use the same map.

For a permutation matrix P:

producer:             W' = P W
consumer:             W' = W P^-1
producer + consumer:  W' = P_out W P_in^-1

Elementwise nonlinearities commute with a pure permutation, so structured coordinates can move as long as every connected tensor axis is compensated. The details depend on which coordinate system is moving.

The repository implements four families:

SwiGLU neurons

A Llama MLP contains two projections into the intermediate space and one back to the residual stream:

m = SiLU(W_gate h) * (W_up h)
y = W_down m

The same intermediate-neuron permutation moves rows of W_gate and W_up and the corresponding columns of W_down. Each transformer layer can choose its own independent neuron order.

GQA-aware attention heads

For ordinary multi-head attention, whole Q/K/V head blocks can move together when the matching columns of the output projection move with them.

TinyLlama uses grouped-query attention: 32 Q heads share four KV heads, with eight Q heads per KV group. An arbitrary independent shuffle would break that relationship. The tool instead derives a permutation of KV groups and a permutation of Q heads inside each group. This preserves the runtime's GQA mapping.

Coordinates inside each head remain canonical. In particular, the converter does not scramble RoPE coordinate pairs or frequencies.

One global residual basis

The residual stream is different because residual additions connect every layer. A separate hidden map per layer would require runtime gather/scatter operations between layers. The implemented transformation therefore uses one global permutation of the 2,048 hidden coordinates.

That map is applied to:

• hidden columns of embeddings and the output head;
• input columns of Q, K, V, gate, and up projections;
• output rows of attention O and MLP down projections; and
• entries of every learned RMSNorm vector.

Q/K/V output coordinates stay canonical, so the hidden-basis map does not change head-internal or RoPE coordinates.

When the four families are composed, every one of TinyLlama's 201 learned tensors has at least one transformed axis.

Exact checkpoint reversal does not imply exact FP inference

Direct SafeTensors conversion is lossless as a stored-checkpoint operation. The inverse all-axis transformation restored the original 2.2 GB TinyLlama file byte-for-byte.

Floating-point execution is a separate question.

Row permutations often leave each dot product's internal term order alone. Column permutations do not. If a compensated transformation changes the order in which floating-point products are accumulated, mathematically equivalent sums can round differently.

The retained FP control compared 1,536,000 output logits across three 16-position sequences. It preserved 48/48 tested top-1 decisions while showing nonzero reduction-order drift. The complete maximum, mean, RMS, exact-value, and top-5 statistics are reported in RESULTS.md.

This is not reported as bit-identical FP inference. The 48/48 result is a bounded control, not a guarantee for arbitrary prompts, longer generation, or other framework kernels.

What was verified

The full-model validation target was TinyLlama-1.1B-Chat-v1.0:

learned scalar parameters    1,100,048,384
learned tensors              201
vocabulary                   32,000
hidden size                  2,048
intermediate size            5,632
Q / KV heads                 32 / 4
transformer layers           22
BF16 SafeTensors file        2,200,119,864 bytes
Q16.48 MGW file              8,800,406,496 bytes

The retained results separate four claims:

An additional Q16.48 TinyLlama gate tested one explicit residual-basis layout produced by composing 10,000 deterministic random coordinate transpositions. The final order moved 2,046 of 2,048 hidden coordinates. The converter applied that final order in one checkpoint pass, the verifier checked all 201 learned tensors, the inverse restored the complete 8.8 GB file byte-for-byte, and the unchanged runtime matched all 80 reference tokens. The hash-pinned record is in validation/2026-08-02-ubuntu-x86_64-hidden-random/. This was a hidden-basis-only gate, separate from the four-family composed artifact described below.

The all-axis MGW verifier reconstructs the expected source coordinate for both axes of every matrix and compares the final composed artifact directly with the original. It does not merely trust each sequential converter. A second 8.8 GB artifact reversing the complete four-stage MGW composition was not retained, so the repository does not claim that particular whole-file cmp gate. Each component has its own byte-exact inverse test.

Also, "all learned tensors are transformed" does not mean every scalar byte must differ. A permutation can have fixed points, and different coordinates can contain equal values. The claim is about prescribed coordinate coverage, not a count of numerically changed bytes.

The exact hashes, environment records, and complete qualification are in RESULTS.md and the repository's validation/ directory.

Why retain the Q16.48 path?

The direct SafeTensors path answers the practical checkpoint question. The integer path answers a stronger validation question.

The MGW converter moves stored int64_t values without dequantization or requantization. Structural checks can compare every prescribed coordinate byte-for-byte. Within the runtime's supported numeric range, signed-128 accumulation also makes reordered integer dot-product terms exact rather than merely close.

The final all-axis Q16.48 model passed the same four-prompt greedy gate as the original:

france_capital   20/20
story_beginning  20/20
simple_math      20/20
meaning_of_life  20/20
TOTAL            80/80

Both canonical result streams had the same SHA-256:

a887f7a8e27717d589dd9f3015c8a81fc9ab9544549ae5b5bd32cf693265985f

That does not turn 80 tokens into a general model-quality proof. It does make the integer runtime a useful regression oracle for checking whether the implemented coordinate maps preserve the tested computation exactly.

In short: FP makes the tool directly usable with standard checkpoints; INT makes the evidence stronger.

Using the standard SafeTensors path

Vocabulary-only permutation

For the smallest transformation, first create a complete vocabulary map:

(umask 077; set -C; head -c 32 /dev/urandom > token.key)

python3 token_permutation.py create-map \
  --vocab 32000 \
  --key-file token.key \
  --output token.tpmap

Then apply it directly to a checkpoint:

python3 safetensors_permutation.py convert \
  --input model.safetensors \
  --output model.token-permuted.safetensors \
  --map token.tpmap

python3 safetensors_permutation.py verify \
  --input model.safetensors \
  --output model.token-permuted.safetensors \
  --map token.tpmap

The TPMAP stores both the complete token map and its inverse. It is sufficient for boundary translation; it is not a hash or an encryption key.

The repository also accepts a specified vocabulary order through explicit_token_map.py --permutation-file. Its versioned JSON uses this orientation:

{
  "format": "int-llm-token-order-v1",
  "new_to_old": [2, 0, 3, 1]
}

Position j names the original logical row that should be stored at new physical row j. A real model order must list all vocabulary IDs exactly once.

All-axis permutation

The all-axis SafeTensors tool is the complete-checkpoint path. In one pass it combines the vocabulary map, independent per-layer MLP and GQA maps, and one global residual-basis map. Its verifier re-derives the prescribed source coordinate for every learned matrix axis and norm-vector entry from the original checkpoint and map inputs. On TinyLlama, this is the path that checked all 201 learned tensors and restored the original BF16 checkpoint byte-for-byte after inverse conversion.

The implementation is currently deliberately narrow: one unsharded SafeTensors file, a Llama architecture with standard Hugging Face tensor names, an external config.json, and BF16/F16/F32 learned tensors. It fails closed if it cannot classify the complete learned checkpoint.

Explicit and possible layout experiments

Once a functionally equivalent order can be baked into a checkpoint and verified, the order does not have to be random.

Explicit order files now cover vocabulary rows, MGW MLP-neuron axes, and the global MGW residual hidden basis. The attention component tool and the SafeTensors all-axis path still derive their internal maps from keys.

The smallest explicit hidden-basis example swaps coordinates 0 and 31. On the committed MicroGPT checkpoint, that one global transposition moves the required row or column lanes through all nine learned matrices. The inverse is byte-exact, and the complete oracle matched all 5,832 checked Q16.48 logits.

The systematic MLP layout experiment uses explicit orders to compare the original neuron order with smooth, spread, and seeded random physical layouts. It scores exact L1 distance between adjacent complete neuron weight profiles:

All four variants passed prescribed row/column verification and matched all 5,832 MicroGPT oracle logits. smooth and spread are deterministic heuristics, not global optima, and these scores describe physical adjacency, not sparsity or performance.

Possible follow-up experiments include:

• arranging vocabulary rows by measured token frequency for paging or storage locality;
• grouping frequently and infrequently activated MLP neurons within each layer for a sparse or offloaded runtime; and
• grouping channels or GQA-compatible whole heads for downstream layout or quantization experiments.

The remaining tooling gap is explicit internal-order input for the attention component and for the one-pass SafeTensors all-axis path. A profiler-produced MLP or hidden order can already be supplied to the corresponding MGW tool.

More importantly, permutation is not an optimization by itself. Dense GEMM shapes and operation counts do not change. This is closer to preparing a defragmented layout: a benefit appears only if a runtime, pager, sparse executor, offload scheme, or quantizer exploits the resulting contiguous groups and the result is measured under a controlled workload.

This repository contains no speedup claim.

FAQ: Is coordinate permutation the same as quantization?

No. They change different parts of the checkpoint representation.

Coordinate permutation changes where a stored value lives; quantization changes the value stored at a coordinate.

The two operations can be combined, but they are separate transformations and should be validated separately. Some production quantizers also pack or reorder weights for execution; that is an additional layout operation, not quantization itself.

Prior art and claim boundary

The underlying symmetries are known. This project does not claim to have discovered vocabulary, neuron, attention-head, or residual-basis permutation.

Git Re-Basin treats hidden-unit permutations as function-preserving transformations for model alignment and merging. Update Your Transformer to the Latest Release adapts re-basin ideas to transformers, including attention-head and residual-connection constraints. Signed-Permutation Coordinate Transport for RMSNorm Transformers describes a broader signed permutation gauge for RMSNorm models; the residual transformation here uses only the permutation subset.

ObfusLM (ACL 2025) pursues a privacy claim with additional mechanisms and attack evaluation. This repository does not implement that privacy system. A fixed vocabulary permutation is closer to a monoalphabetic substitution over BPE tokens: it leaks lengths, repetitions, and frequencies, and a public base checkpoint makes coordinate matching especially straightforward.

The accurate claim here is narrower: auditable conversion and verification tools for equivalent Llama checkpoint layouts, with exact integer regression evidence and explicit floating-point drift reporting.

What this experiment establishes

Within the documented TinyLlama and fixture gates:

• vocabulary, SwiGLU-neuron, GQA-aware head, and residual-basis permutations compose into a transformation covering every learned tensor;
• direct BF16/F16/F32 coordinate movement needs no FP-to-INT-to-FP conversion;
• the direct BF16 all-axis checkpoint has a byte-exact inverse;
• finite-precision FP inference can drift because column order changes reduction order;
• the Q16.48 path provides exact structural checks and matched all 80 tested greedy decisions; and
• the tools provide a correctness substrate for future layout experiments, not an optimization or privacy result by themselves.

This is one model family, one full-model checkpoint, and bounded regression evidence. It is not a general model-quality proof, a generic checkpoint framework, a cryptographic construction, or a performance result.

Links

• Coordinate-permutation repository
• Original int-llm repository
• Original small model artifact
• Original int-llm article
• int-llm-precision-ladder repository
• int-llm-precision-ladder article

For the shortest path through the evidence, start with the coordinate repository's README.md, then read RESULTS.md and experiments/PRIOR_ART.md.