Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference

PLDR-LLM: Power Law Decoder Representations

The Large Language Model from Power Law Decoder Representations (PLDR-LLM) and its attention mechanism, Power Law Graph Attention (PLGA), replace the fixed bilinear form of scaled dot-product attention (SDPA) with a learned, input-generated bilinear operator G_LM, built from a positive tensor A_LM via elementwise power laws.

The architecture is fully specified and verified against pinned reference releases. Claims are labeled theorem, conditional theorem, measurement, or conjecture.

Unconditional results

• PLGA contains SDPA exactly at G_LM = I
• A_LM and A_P are strictly entrywise positive, with Perron–Frobenius structure on A_LM
• The DAG regularizer has the NOTEARS walk-counting form; positivity obstructs exact acyclicity
• Under nonresonance (satisfied by standard rotary frequencies), a commutant criterion identifies which operators preserve relative-position dependence

Inference-collapse theorem

Exact input invariance of deductive outputs collapses inference to generalized SDPA with a constant operator.

Measured invariance

• Relative fluctuations of 10^-6 and below
• Perturbation bounds quantify but do not certify cached inference
• The assembled proxy misses the decoding margin

Conditional mechanism

A conditional three-stage mechanism (rotary twirl, concentration, row-map contraction) is measured on a released checkpoint.

Blockwise training and scoring

Stated with explicit target exposure under the global Gram. On tested samples, block and sequential scoring select identical answers and agree on the published TruthfulQA probability-mass metric within 5×10^-5 per item.

Self-organized criticality

Enters as a phenomenological framework with an intrinsic order parameter; open claims become falsifiable conjectures.

Selected proof cores are machine-checked in Lean 4.