Update README.md

README.md

@@ -2,689 +2,696 @@
 license: mit
 ---
 
-# Disclaimer
-
-With recent changes, the runtime of a quantization operation has gone through the roof, but models quantized this way benefit from a massive increase in model coherency.
-
-I am not going to revert these changes, but will attempt to streamline this further.
-
-To use this properly you must dump a .hxh file and enable --hessian and --analog-imatrix, otherwise it will fall back to the typical RMSE path which is still vastly superior to everything else. 
-
-It is worth the cost of using --hessian and --analog-imatrix, though.
-
-
 # HPC-Quantize
 
-### Holographic Phase Contraction for Ultra-Low-Bit LLM Quantization
+## Holographic Phase Contraction for Ultra-Low-Bit LLM Quantization
 
-**HPC-Quantize** is an MIT-licensed research quantization engine for aggressively compressing large language models while preserving inference behavior at extremely low bitrates.
+**HPC-Quantize** is an experimental, MIT-licensed quantization engine for aggressively compressing large language models into extremely low-bit formats, with a particular focus on **Q2-class quantization**.
 
-HPC was originally developed around a quantum-inspired Shor/Griffiths–Niu measurement formulation. The current engine has evolved beyond that approach: the production quantization path uses a **sequential Sieve**, bounded state back-action, and a **36-state Viterbi optimizer** for Q2 quantization.
+The central idea is simple:
 
-The central idea remains the same:
+> **At very low bitrates, quantization should be treated as a structured reconstruction problem rather than independent rounding of individual blocks.**
 
-> **Quantization error is not merely a collection of independent scalar errors. Its structure matters.**
+Instead of choosing a quantization candidate solely from its local reconstruction error, HPC generates competing reconstructions, represents them in a compact discrete state space, models interactions between neighboring blocks, and performs a global sequence optimization before writing the final GGUF.
 
-Rather than optimizing every block independently, HPC treats quantization as a structured discrete-state optimization problem.
+The current production path is entirely classical. Earlier versions explored quantum-inspired state and measurement formulations; the current implementation uses a **sequential Sieve, bounded state back-action, and a 36-state Viterbi optimizer** for Q2.
 
 ---
 
-# What makes HPC different?
+## Why HPC?
 
-Conventional low-bit quantization generally asks:
+At Q4 or Q5, a model often has enough representational freedom that many quantization strategies work reasonably well.
 
-$$
-\hat W_i =
-\arg\min_{\hat W}
-\|W_i-\hat W\|^2
-$$
-
-for each block independently.
+At Q2, the situation changes dramatically.
 
-HPC instead introduces an explicit state space for quantization candidates and considers:
+A block has very few representable values, so small decisions about scale, minimum, and code assignment can produce disproportionately large changes in the resulting weight tensor.
 
-* candidate reconstruction error
-* anisotropic error structure
-* neighboring quantization states
-* state diversity
-* candidate probability
-* cross-block state transitions
-* global sequence coherence
+A conventional quantizer often reduces the problem to:
 
-The resulting optimization is closer to a graphical-model / sequence-decoding problem than conventional round-to-nearest quantization.
+```text
+original weights
+      │
+      ▼
+find locally best parameters
+      │
+      ▼
+encode quantized block
+```
 
-Conceptually:
+HPC instead treats each block as a **discrete candidate-selection problem**:
 
 ```text
-             Original weights
-                    │
-                    ▼
-          Candidate generation
-                    │
-                    ▼
-          Error-aware scoring
-                    │
-                    ▼
-       Six-state HPC representation
-                    │
-                    ▼
-          Sequential Sieve
-                    │
-          ┌─────────┴─────────┐
-          │                   │
-     state selection     neighbor back-action
-          │                   │
-          └─────────┬─────────┘
-                    ▼
-             Q2 state lattice
-                    │
-                    ▼
-          36-state Viterbi
-                    │
-                    ▼
-          Candidate reconstruction
-                    │
-                    ▼
-              GGUF output
+original weights
+      │
+      ▼
+generate candidate reconstructions
+      │
+      ▼
+score candidate errors
+      │
+      ▼
+map candidates into a compact state space
+      │
+      ▼
+sequential Sieve
+      │
+      ▼
+Q2 state lattice
+      │
+      ▼
+global Viterbi optimization
+      │
+      ▼
+select physical reconstructions
+      │
+      ▼
+GGUF
 ```
 
+This lets the optimizer preserve competing possibilities until there is enough information to make a global decision.
+
 ---
 
-# The HPC state space
+# What HPC actually does
 
-HPC represents quantization candidates using a six-state symbolic space:
+HPC is still a **quantizer/re-quantizer**.
 
-$$
-d\in\{0,1,2,3,4,5\}.
-$$
+It does not retrain the neural network, modify the model architecture, or learn a new set of representations.
 
-The six states form the basic HPC **quhit**.
+The "reconstruction" terminology refers to what happens during candidate selection: HPC explicitly constructs multiple possible low-bit approximations of the original weights and evaluates them against the source weights.
 
-Candidate parameters are mapped into these six symbolic states through the candidate-to-quhit mapping.
+Conceptually:
 
-This provides a compact representation of the candidate landscape while retaining multiple competing possibilities rather than immediately selecting the local minimum.
+$$
+W \rightarrow
+\{\hat W_1,\hat W_2,\ldots,\hat W_n\}
+\rightarrow
+\text{structured candidate selection}
+\rightarrow
+Q(W)
+$$
+
+The final output remains a normal quantized GGUF model.
 
 ---
 
-# Q2: the 36-state space
+# Core design
 
-Q2 quantization has two coupled parameters:
+HPC combines several ideas:
 
-$$
-(d,d_{\min})
-$$
+* candidate reconstruction
+* weighted reconstruction error
+* optional importance-matrix weighting
+* discrete state mapping
+* sequential Sieve selection
+* bounded neighboring-state back-action
+* Q2 state coupling
+* global Viterbi optimization
+* local reconstruction-quality safeguards
 
-Each parameter is projected into six HPC states.
+The important distinction is that these components operate **together** rather than treating every quantization block as completely independent.
 
-Consequently, the joint state is
+---
 
-$$
-(q_D,q_M)
-$$
+# Candidate generation
+
+For an eligible Q2 block, HPC searches over possible quantization parameters rather than committing immediately to a single local solution.
 
-with
+For Q2_K, the important coupled parameters are represented conceptually as:
 
 $$
-q_D,q_M\in\{0,\ldots,5\}.
+(d,d_{\min})
 $$
 
-The complete symbolic Q2 state space is therefore:
+Candidate parameters are evaluated by reconstructing the quantized block and measuring its error against the original weights.
+
+With an importance matrix, the error can be weighted so that sensitive dimensions contribute more heavily:
 
 $$
-6\times6=36.
+E =
+\sum_i w_i(x_i-\hat{x}_i)^2.
 $$
 
-The state index is:
+This is useful because ordinary unweighted RMSE assumes every weight contributes equally to the final model behavior.
 
-$$
-s=6q_D+q_M.
-$$
+HPC can therefore evaluate:
 
-Thus:
+> **How well does this candidate reconstruct the important parts of the original block?**
 
-```text
-             dmin
-          0  1  2  3  4  5
-        ┌───────────────────
-      0 │ 0  1  2  3  4  5
-      1 │ 6  7  8  9 10 11
- d    2 │12 13 14 15 16 17
-      3 │18 19 20 21 22 23
-      4 │24 25 26 27 28 29
-      5 │30 31 32 33 34 35
-```
+rather than only:
 
-Each symbolic state corresponds to the best physical Q2 candidate belonging to that state.
+> **How small is its raw Euclidean error?**
 
 ---
 
-# Error → probability
+# From candidates to states
 
-HPC converts candidate error into a Boltzmann-like amplitude:
+Keeping every physical candidate in the global optimizer would be expensive.
 
-$$
-A_i =
-\exp\left(
--\frac{E_i-E_{\min}}{2T}
-\right).
-$$
+HPC therefore maps candidates into a compact symbolic state representation.
 
-Squaring gives the corresponding probability:
+The current implementation uses **six symbolic states** for each quantization parameter:
 
 $$
-P_i\propto
-\exp\left(
--\frac{E_i-E_{\min}}{T}
-\right).
+d \in \{0,1,2,3,4,5\}.
 $$
 
-This means the quantizer retains information about the *relative quality* of candidates rather than immediately discarding everything except the MSE winner.
+The symbolic states provide a compact representation of the candidate landscape.
 
----
+Multiple physical candidates can belong to the same symbolic state.
 
-# The Sieve
+This separation is important:
 
-The current HPC engine replaces the original Shor/Griffiths–Niu measurement path with a classical sequential Sieve.
+```text
+physical candidates
+        │
+        ▼
+symbolic state representation
+        │
+        ▼
+global optimization
+        │
+        ▼
+physical candidate selection
+```
 
-For a state \(d\), its initial probability is:
+The symbolic state space is therefore not the same thing as the number of physical reconstructions generated during candidate search.
 
-$$
-p_0(d)=|\alpha(d)|^2.
-$$
+---
 
-Each neighboring state contributes a compatibility factor.
+# The Q2 state space
 
-For neighbor \(j\):
+Q2 has two coupled parameters:
 
 $$
-B_j(d)
-=
-\sum_w p_j(w)
-\begin{cases}
-0.85,&w=d\\
-1,&w\ne d.
-\end{cases}
+(d,d_{\min}).
 $$
 
-Because the probabilities sum to one:
+Each is represented by six symbolic states.
+
+Therefore the joint Q2 space contains:
 
 $$
-B_j(d)=1-0.15p_j(d).
+6\times6=36
 $$
 
-The resulting score is therefore:
+states.
+
+The state index is:
 
 $$
-S(d)=
-p_0(d)
-\prod_{j\in N}
-\left(1-0.15p_j(d)\right).
+s=6q_D+q_M
 $$
 
-The implementation evaluates this in logarithmic form:
+where:
 
 $$
-\ell(d)=
-\log p_0(d)
-+
-\sum_{j\in N}
-\log\left(1-0.15p_j(d)\right).
+q_D,q_M\in\{0,\ldots,5\}.
 $$
 
-This gives the Sieve a simple interpretation:
+The resulting lattice is:
 
-> **A locally strong state remains strong, but states already strongly represented by neighboring blocks are gently penalized.**
-
----
-
-# Why 0.85?
-
-The factor:
+```text
+          dmin
+          0   1   2   3   4   5
+        ┌────────────────────────
+      0 │  0   1   2   3   4   5
+      1 │  6   7   8   9  10  11
+d     2 │ 12  13  14  15  16  17
+      3 │ 18  19  20  21  22  23
+      4 │ 24  25  26  27  28  29
+      5 │ 30  31  32  33  34  35
+```
 
-$$
-0.85
-$$
+The 36-state representation is a **global optimization space**, not a claim that only 36 physical quantization candidates exist.
 
-controls the strength of the local anti-correlation.
+---
 
-If a neighboring site is completely concentrated on state \(d\),
+# Candidate probabilities
 
-$$
-p_j(d)=1,
-$$
+Rather than treating each candidate as simply "best" or "not best", HPC can preserve information about the relative quality of competing candidates.
 
-then:
+Candidate errors are converted into probability-like weights using a Boltzmann-style transformation:
 
 $$
-B_j(d)=0.85.
+P_i \propto e^{-T(E_i-E_{\min})}.
 $$
 
-In log space this is:
+This creates a soft candidate distribution.
 
-$$
-\log(0.85)\approx-0.1625.
-$$
+The practical consequence is important:
 
-The Sieve therefore discourages repeated neighboring states without making them impossible.
+> A candidate that is slightly worse than the local minimum can remain relevant if it belongs to a useful region of the state space.
 
-This is intentional.
-
-HPC does **not** attempt to force an artificial alternating pattern. It merely introduces a preference for diverse state assignments when alternative candidates remain competitive.
+That makes the search less eager to collapse immediately to a single local minimum.
 
 ---
 
-# The 1.5 Sieve slack
+# The Sieve
 
-After calculating the six state scores, HPC finds:
+The current production architecture uses a **sequential Sieve**.
 
-$$
-\ell_{\max}=\max_d\ell(d).
-$$
+Rather than treating blocks as fully independent, neighboring state distributions influence one another.
 
-A state survives if:
+For a candidate state \(d\), the implementation applies a bounded compatibility penalty based on neighboring probability:
 
 $$
-\ell(d)\geq\ell_{\max}-1.5.
+B_j(d)=1-0.15p_j(d).
 $$
 
-Equivalently:
+The combined Sieve score can be written conceptually as:
 
 $$
-\frac{S(d)}{S_{\max}}
-\geq e^{-1.5}
-\approx0.2231.
+S(d)=p_0(d)\prod_j(1-0.15p_j(d)).
 $$
 
-Thus alternatives whose score is at least approximately **22.3% of the best candidate** remain eligible.
+The effect is deliberately modest.
 
-This prevents premature collapse of the candidate landscape.
+If a neighboring site is strongly concentrated on the same state, that state becomes somewhat less attractive locally.
 
-The Sieve therefore acts as a **soft constraint**, not a hard nearest-neighbor rule.
+The Sieve therefore encourages **state diversity and compatibility** without forcing an alternating pattern.
 
 ---
 
-# Sequential collapse and back-action
+# Sieve slack
 
-Once the current state is selected:
+The Sieve also avoids immediately discarding every candidate that is not the top local state.
 
-$$
-d_k=\arg\max_d S_k(d),
-$$
+A bounded slack region allows near-optimal states to survive the first selection stage.
 
-that state is collapsed.
+Conceptually:
 
-The selected state is then propagated into its remaining neighbors.
+```text
+best state
+    │
+    ├── keep
+    │
+    ├── keep near-optimal alternatives
+    │
+    └── discard clearly inferior states
+```
 
-For every live neighbor:
+This prevents the candidate distribution from collapsing too early.
 
-$$
-\alpha_j(d_k)
-\leftarrow
-0.85\alpha_j(d_k).
-$$
+---
+
+# Sequential conditioning
+
+Once the current state is selected, its influence is propagated into neighboring sites.
 
-The neighbor is then renormalized.
+The selected state therefore affects subsequent decisions.
 
-This creates sequential conditioning:
+This gives the process a sequential character:
 
 $$
 P(d_1)
 \rightarrow
-P(d_2\mid d_1)
+P(d_2|d_1)
 \rightarrow
-P(d_3\mid d_1,d_2)
-\rightarrow\cdots
+P(d_3|d_1,d_2)
+\rightarrow \cdots
 $$
 
-The graph therefore does not make independent decisions.
-
-Earlier decisions influence later decisions.
+The quantization process is consequently no longer just a collection of independent block decisions.
 
 ---
 
-# Q2 Viterbi optimization
+# Viterbi optimization
 
-After the Sieve produces the six-state distributions, Q2 reconstructs the full 36-state lattice.
+After the Sieve, Q2 state probabilities are expanded into the full **36-state joint lattice**.
 
-For each symbolic state:
+Each block receives a local cost combining reconstruction error with state probability.
 
-$$
-s=(q_D,q_M),
-$$
-
-HPC retains the lowest-error physical candidate associated with that state.
-
-The Sieve distributions provide a prior:
+Conceptually:
 
 $$
-P(s)
-\approx
-P_D(q_D)P_M(q_M).
+C_i(s)
+=
+E_i(s)-\lambda \log P_i(s).
 $$
 
-The local Viterbi cost is:
+HPC then adds a transition cost between neighboring blocks.
 
-$$
-C_i(s)=
-E_i(s)
--
-0.25\bar E\log P_i(s).
-$$
-
-HPC then imposes a transition cost between adjacent blocks:
+A simple form is:
 
 $$
-T(s',s)=
-0.08\bar E
-\left(
-|q_D-q_D'|
-+
-|q_M-q_M'|
-\right).
+T(s',s)
+\propto
+|q_D-q'_D|+
+|q_M-q'_M|.
 $$
 
-This is Manhattan distance on the 6×6 Q2 lattice.
+This is Manhattan distance on the 6×6 state lattice.
 
-The dynamic program is:
+The dynamic-programming recurrence is:
 
 $$
-DP_i(s)=
-C_i(s)+
+DP_i(s)
+=
+C_i(s)
++
 \min_{s'}
 \left[
 DP_{i-1}(s')+T(s',s)
 \right].
 $$
 
-The result is a globally optimized sequence of Q2 states rather than independently optimized blocks.
+The result is a **globally optimized sequence of Q2 states**.
+
+This is one of the major differences between HPC and purely local quantization.
 
 ---
 
-# Greedy safety override
+# Local safety
 
-HPC does not blindly trust the global optimizer.
+Global regularization should not be allowed to produce obviously poor local reconstructions.
 
-After Viterbi selection, the engine compares the selected candidate against the best local reconstruction candidate.
+After global selection, HPC can compare the chosen state against the locally best reconstruction.
 
-If the greedy candidate is sufficiently better:
+A sufficiently large local improvement can trigger a local override.
 
-$$
-E_{\text{greedy}}
-<
-0.95E_{\text{selected}},
-$$
+This gives the optimizer a safety mechanism:
 
-the local candidate wins.
+```text
+global structure
+      │
+      ▼
+candidate selected
+      │
+      ▼
+is the local reconstruction much better?
+      │
+   ┌──┴──┐
+   │     │
+  yes    no
+   │     │
+local   keep
+winner  global
+        winner
+```
 
-This prevents global regularization from sacrificing too much actual reconstruction quality.
+The goal is to prevent the global objective from becoming disconnected from actual reconstruction quality.
 
 ---
 
 # Error geometry
 
-HPC can also score candidate errors using the D₆ Vesica decomposition.
+HPC can also use structured error decomposition rather than treating every error component identically.
 
-For paired error components:
-
-$$
-v_p=e_p+e_{p+h}
-$$
+One experimental component uses a D₆/Vesica-style decomposition of paired error terms.
 
-and
+For a pair of error components:
 
 $$
-w_p=e_p-e_{p+h},
+v=e_p+e_{p+h}
 $$
 
-the error is separated into:
-
-* **Vesica / DC-like error**
-* **wave / AC-like error**
-
-The weighted objective is:
-
 $$
-E=
-\frac12
-\left(
-4E_{\mathrm{vesica}}
-+
-E_{\mathrm{wave}}
-\right).
+w=e_p-e_{p+h}.
 $$
 
-Thus:
+This separates the error into different modes before applying the final weighting.
 
-$$
-\boxed{
-E_{\mathrm{vesica}}
-\text{ receives 4× the penalty of }
-E_{\mathrm{wave}}.
-}
-$$
+The intent is to distinguish error geometry rather than assuming that all directions in weight space have identical consequences.
 
-The goal is not simply to minimize Euclidean distance. HPC attempts to distinguish error modes that behave differently during downstream computation.
+This is an experimental feature of the HPC objective rather than a requirement of GGUF or Q2_K itself.
 
 ---
 
-# Precision strategy
+# Building
 
-HPC is designed for aggressive low-bit quantization and does not require every tensor to be treated identically.
+HPC-Quantize is intended to be used alongside a GGUF/llama.cpp workflow.
 
-A typical strategy is:
+Typical dependencies include:
 
-| Tensor class              | HPC strategy            |
-| ------------------------- | ----------------------- |
-| Attention projections     | Q4_0 / higher precision |
-| FFN / MLP weights         | Q2_K                    |
-| Expert weights            | Q2_K                    |
-| Embeddings                | Preserved / promoted    |
-| Norms                     | Preserved               |
-| Router / gate parameters  | Preserved or promoted   |
-| LM head / tied embeddings | Promoted when required  |
+```bash
+sudo apt install \
+    gcc \
+    libgmp-dev \
+    libmpfr-dev \
+    python3 \
+    python3-numpy
+```
 
-The exact routing is model-dependent.
+Build the native quantization component:
 
-The purpose is straightforward:
+```bash
+make -f makefile.quantize
+```
 
-> Spend precision where small perturbations are most expensive, and aggressively compress where the model has greater tolerance.
+The resulting library/binary names may vary with the current revision.
 
 ---
 
-# Calibration
+# Mixed-precision workflows
 
-HPC supports importance-matrix calibration for aggressive quantization.
+HPC is primarily intended to solve the problem of **aggressive compression**, not to force every tensor in a model into identical precision.
 
-A typical workflow is:
+A practical deployment may therefore retain higher precision for particularly sensitive tensors and use Q2 for the bulk of the model.
 
-```bash
-python3 LLM/generate_imatrix.py \
-    model.gguf \
-    calibration_data.txt \
-    -o imatrix.dat \
-    --chunks 5 \
-    --verbose
+For example:
+
+```text
+Model
+ ├── embeddings        → higher precision
+ ├── normalization     → preserved
+ ├── attention         → Q4 / promoted
+ ├── FFN / experts     → Q2
+ └── other large mats  → Q2
 ```
 
-The resulting importance data can then be supplied to the re-quantizer.
+The optimal allocation is model-dependent.
 
 ---
 
-# Building
+# Why Q2?
 
-Install the required dependencies on Ubuntu/Debian:
+Q2 is where conventional quantization becomes particularly unforgiving.
 
-```bash
-sudo apt install \
-    gcc \
-    libgmp-dev \
-    libmpfr-dev \
-    python3 \
-    python3-numpy
-```
+At higher precision, the quantizer has many representational degrees of freedom.
 
-Build the HPC engine:
+At Q2, many distinct original weight values must share a very small set of reconstruction values.
 
-```bash
-make -f makefile.quantize
-```
+This means:
 
-The build should produce:
+$$
+\text{small parameter change}
+\rightarrow
+\text{large discrete reconstruction change}.
+$$
 
-```text
-libhexstate_q2k.so
-```
+HPC is designed around this regime.
+
+Rather than assuming the locally nearest reconstruction is always globally best, it explicitly searches among competing discrete configurations.
 
 ---
 
-# Quantization
+# HPC versus conventional quantization
 
-Convert the source model to BF16 GGUF using llama.cpp:
+A simplified conventional pipeline is:
 
-```bash
-python3 llama.cpp/convert_hf_to_gguf.py \
-    /path/to/model \
-    --outfile Model-BF16.gguf \
-    --outtype bf16
+```text
+weight block
+    │
+    ▼
+estimate scale/minimum
+    │
+    ▼
+round values
+    │
+    ▼
+write block
 ```
 
-Generate calibration data:
+A simplified HPC pipeline is:
 
-```bash
-python3 LLM/generate_imatrix.py \
-    Model-BF16.gguf \
-    calibration_data.txt \
-    -o imatrix.dat \
-    --chunks 5 \
-    --verbose
+```text
+weight block
+    │
+    ▼
+generate competing reconstructions
+    │
+    ▼
+score candidates
+    │
+    ▼
+map to symbolic states
+    │
+    ▼
+Sieve + neighboring interaction
+    │
+    ▼
+construct Q2 state lattice
+    │
+    ▼
+Viterbi global optimization
+    │
+    ▼
+select physical reconstructions
+    │
+    ▼
+write Q2_K
 ```
 
-Run HPC quantization:
+The difference is not that HPC stops being quantization.
 
-```bash
-python3 hexstate_requantize.py \
-    Model-BF16.gguf \
-    Model-Q2_K-HPC.gguf \
-    --keep-metadata \
-    --imatrix imatrix.gguf
-```
+The difference is **how much structure it retains before committing to the final quantized representation**.
 
 ---
 
-# Historical architecture
+# A useful way to think about HPC
 
-HPC originally used a quantum-inspired implementation based on:
+HPC can be viewed as three nested optimization problems:
 
-* Z₆ state encoding
-* complex amplitudes
-* graph coupling
-* phase operations
-* IDFT₆
-* Griffiths–Niu-style sequential measurement
-* collapse/back-action
-* beam search
+### Local reconstruction
 
-That architecture was useful for exploring the underlying hypothesis:
+> Which low-bit approximation best represents this block?
 
-> **quantization states can be treated as interacting discrete variables rather than independent rounding decisions.**
+### State inference
 
-The current Sieve architecture keeps the useful structural concepts while replacing the Fourier/phase machinery with explicit probability scoring and dynamic programming.
+> Which region of the discrete candidate space is promising?
 
-In other words:
+### Global sequence optimization
 
-```text
-Original HPC
-
-error
-  ↓
-complex amplitudes
-  ↓
-phase graph
-  ↓
-IDFT₆
-  ↓
-measurement
-  ↓
-back-action
+> Which sequence of candidate states produces the best overall configuration?
+
+That can be summarized as:
 
+$$
+\boxed{
+\text{reconstruction}
++
+\text{state inference}
++
+\text{global optimization}
+}
+$$
 
-Current HPC
+rather than:
 
-error
-  ↓
-probability
-  ↓
-Sieve
-  ↓
-bounded back-action
-  ↓
-36-state lattice
-  ↓
-Viterbi
-```
+$$
+\boxed{
+\text{independent rounding}
+}
+$$
 
-The latter is substantially easier to analyze, reproduce, and tune.
+---
+
+# Experimental nature
+
+HPC is research software.
+
+It should not be assumed that:
+
+* lower RMSE always produces better model behavior;
+* lower perplexity always produces better reasoning;
+* one quantization strategy wins on every architecture;
+* Q2 quality transfers perfectly between models;
+* state-interaction parameters are universally optimal.
+
+The correct way to evaluate HPC is with a combination of:
+
+* reconstruction error
+* perplexity
+* reasoning benchmarks
+* mathematical evaluation
+* coding tasks
+* long-context tests
+* instruction following
+* qualitative generation
+* memory usage
+* inference speed
+
+The objective of HPC is not to optimize one number in isolation.
 
 ---
 
-# Design philosophy
+# Current architecture
 
-HPC is built around several principles.
+The project originally explored a more explicitly quantum-inspired formulation involving state amplitudes, phase operations, graph coupling, Fourier/IDFT transforms, and sequential measurement.
 
-### 1. Do not assume minimum MSE is the whole objective
+The current engine has moved toward a more explicit classical formulation:
 
-A quantization configuration with slightly higher local error can still be preferable if its error structure is better behaved globally.
+```text
+Historical approach
+───────────────────
+candidate error
+      ↓
+amplitudes / phase
+      ↓
+graph coupling
+      ↓
+measurement
+      ↓
+back-action
 
-### 2. Preserve alternatives
 
-Low-bit quantization has a highly discrete error landscape. Prematurely selecting one candidate can eliminate useful configurations.
+Current approach
+────────────────
+candidate error
+      ↓
+probability distribution
+      ↓
+sequential Sieve
+      ↓
+bounded back-action
+      ↓
+36-state Q2 lattice
+      ↓
+Viterbi
+```
 
-### 3. Model interactions explicitly
+The mathematical intuition of interacting discrete states remains, but the current production implementation is classical and deterministic.
 
-Quantization blocks do not exist in complete isolation during inference.
+---
 
-HPC therefore gives neighboring state assignments an explicit influence.
+# Future directions
 
-### 4. Encourage diversity without forcing it
+The state lattice is intentionally compact.
 
-The Sieve's 0.85 penalty is deliberately bounded.
+The 36-state Q2 space is:
 
-It discourages repeated state assignments but does not prohibit them.
+$$
+6\times6=36.
+$$
 
-### 5. Separate local inference from global optimization
+That does **not** mean the physical candidate space must contain only 36 candidates.
 
-The Sieve supplies local state information.
+Possible future work includes:
 
-Viterbi then searches the resulting 36-state Q2 lattice globally.
+* more symbolic states per parameter;
+* multiple physical candidates retained per symbolic state;
+* beam search inside individual states;
+* hierarchical state refinement;
+* adaptive state resolution;
+* larger candidate beams for difficult tensors;
+* tensor-dependent state cardinality;
+* improved transition models;
+* architecture-specific state priors.
 
-### 6. Keep a reconstruction-quality escape hatch
+One particularly interesting extension is to retain multiple physical candidates for each symbolic state:
 
-The 5% greedy override prevents the global model from making obviously bad local decisions.
+$$
+36\times K.
+$$
+
+This would preserve more of the physical reconstruction landscape while keeping the coarse 6×6 state structure.
 
 ---
 
-# What HPC is — and isn't
+# What HPC is trying to preserve
 
-HPC is:
+At ultra-low precision, numerical error is inevitable.
 
-* an experimental ultra-low-bit quantization engine
-* MIT licensed
-* designed for GGUF/llama.cpp workflows
-* focused on Q2-class compression
-* based on structured discrete-state optimization
-* capable of combining local reconstruction error with state interactions
+The goal is therefore not:
 
-HPC is **not**:
+> **make every weight numerically perfect.**
 
-* a claim that LLM inference is literally quantum computation
-* a replacement for llama.cpp itself
-* a guarantee of improved perplexity on every model
-* a conventional round-to-nearest quantizer
-* dependent on quantum hardware
+The goal is:
 
-The current Sieve is entirely classical.
+> **spend the available representational capacity where it matters most, preserve competitive reconstruction alternatives long enough for global selection, and avoid treating every block as an isolated rounding problem.**
 
-The "quantum-inspired" terminology refers to the historical mathematical lineage and the state/graph representation—not to a requirement for quantum hardware.
+That is the central design philosophy of HPC-Quantize.
 
 ---
 
@@ -692,56 +699,38 @@ The "quantum-inspired" terminology refers to the historical mathematical lineage
 
 HPC-Quantize is released under the **MIT License**.
 
-Quantized model files remain subject to the license and terms of the underlying base model.
+The licensing terms of any model quantized with HPC remain separate from the HPC software license.
+
+Always verify the license of the underlying base model before redistribution.
 
 ---
 
 # Status
 
-HPC-Quantize is research software.
+**Experimental / research software**
 
-The quantization architecture is intentionally experimental, particularly at extremely low bitrates such as Q2.
+The current engine is actively evolving, particularly around ultra-low-bit Q2 quantization.
 
-Results should therefore be evaluated using:
+The project currently prioritizes:
 
-* perplexity
-* task benchmarks
-* generation quality
-* long-context behavior
-* mathematical reasoning
-* coding benchmarks
-* model-specific evaluations
+* low-bit reconstruction quality
+* model coherence
+* reasoning preservation
+* structured state selection
+* aggressive memory reduction
 
-rather than RMSE alone.
+over compatibility with any single traditional quantization metric.
 
 ---
 
-## The short version
+## In one sentence
 
-Conventional Q2:
+**HPC-Quantize is a structured ultra-low-bit quantizer that searches over competing Q2 reconstructions, reasons about them as interacting discrete states, and uses global sequence optimization to choose the final GGUF configuration.**
 
-$$
-\boxed{
-\text{pick the lowest-error candidate}
-}
-$$
+---
 
-HPC:
+## Acknowledgements
 
-$$
-\boxed{
-\text{score candidates}
-\rightarrow
-\text{encode state}
-\rightarrow
-\text{sieve}
-\rightarrow
-\text{condition neighbors}
-\rightarrow
-\text{search the 36-state lattice}
-\rightarrow
-\text{select a globally coherent configuration}
-}
-$$
+HPC-Quantize builds on the broader GGUF and llama.cpp ecosystem and is intended to interoperate with existing llama.cpp-based tooling.
 
-**HPC-Quantize treats ultra-low-bit quantization as a structured state-selection problem, not merely a rounding problem.**
+The project also explores ideas inspired by discrete graphical models, sequential inference, information-weighted reconstruction, and quantum-inspired state representations.