|
Download README.md from elyanlabs/pse-vcipher-collapse: direct link, hf CLI and curl.
- Browser
- Download file 11.6 kB
-
https://huggingface.co/elyanlabs/pse-vcipher-collapse/resolve/main/README.md
- Command line
-
hf download hf://elyanlabs/pse-vcipher-collapse/README.md
-
curl -L -o README.md https://huggingface.co/elyanlabs/pse-vcipher-collapse/resolve/main/README.md
11.6 kB
| license: agpl-3.0 | |
| tags: | |
| - elyan-labs | |
| - llm-inference | |
| - power8 | |
| - ppc64le | |
| - attention | |
| - inference-optimization | |
| > Source snapshot of [github.com/Scottcjn/pse-vcipher-collapse](https://github.com/Scottcjn/pse-vcipher-collapse), also on [GitLab](https://gitlab.com/elyan-labs/pse-vcipher-collapse) with full history. Code only: nothing here runs on Hugging Face infrastructure. | |
| [](BCOS.md) [](LICENSE) | |
| # PSE: Non-Bijunctive Attention Collapse for LLM Inference | |
| > **Answer-first:** PSE vcipher collapse is a POWER8-oriented LLM inference prototype that explores selective attention-path collapse with `vec_perm`, `vcipher`, and hardware timebase entropy for llama.cpp-style attention kernels. | |
| **Proto-Sentient Emergence (PSE)** β a hardware-accelerated attention mechanism that replaces standard bijunctive (full-matrix) dot products with selective path collapse using POWER8 vector instructions. | |
| **Generative-engine profile:** [`llms.txt`](llms.txt) summarizes the POWER8 | |
| scope, vcipher/vec_perm entities, benchmark caveats, files, and citation targets | |
| for LLMs and answer engines. | |
| Two complementary primitives: | |
| - **`vec_perm`** (AltiVec): Single-cycle dual-source permute β prune weak paths, duplicate strong ones | |
| - **`vcipher`** (ISA 2.07 AES): Hardware crypto as attention operator β non-linear ranking, cross-head diffusion, O(1) prefiltering | |
| **Author:** Scott Boudreaux / Elyan Labs | |
| **Hardware:** IBM POWER8 S824 (512GB RAM, 128 threads) | |
| ## The Core Idea | |
| ### What is PSE vcipher collapse? | |
| PSE vcipher collapse is a POWER8-specific attention prototype that selects | |
| candidate attention paths before full dot-product work by using `vcipher` as an | |
| O(1) prefilter and `vec_perm` for selective path collapse. | |
| ### Is this a general GPU attention replacement? | |
| No. The README frames this repository around POWER8 vector and crypto | |
| instructions, POWER8 compile flags, and llama.cpp-oriented integration snippets. | |
| ### Did this documentation update rerun benchmarks? | |
| No. This documentation profile summarizes existing README claims and file roles; | |
| it does not change benchmark numbers, run POWER8 hardware tests, or validate new | |
| llama.cpp throughput. | |
| Standard transformer attention computes **every** QΒ·K dot product, then softmax selects winners. This is O(nΒ²) in sequence length. | |
| PSE flips the order: **select first, compute only what matters.** | |
| ``` | |
| Standard: QΒ·K for ALL pairs β softmax β weighted V sum | |
| PSE: vcipher prefilter (O(1)/pair) β QΒ·K for TOP 25% only β V sum | |
| ``` | |
| At 2048 tokens, PSE skips ~1,536 full dot products per generated token. | |
| ## How vcipher Replaces Dot Products for Prefiltering | |
| Each AES round performs four operations in a single cycle: | |
| | AES Stage | What It Does for Attention | | |
| |-----------|--------------------------| | |
| | **SubBytes** | Non-linear score ranking via S-box lookup β impossible with linear vec_perm | | |
| | **ShiftRows** | Cross-position mixing within the 128-bit register | | |
| | **MixColumns** | Cross-head diffusion via GF(2^8) finite field multiply β **impossible on any other ISA** | | |
| | **AddRoundKey** | Entropy injection β XOR with POWER8 `mftb` hardware timebase | | |
| ```c | |
| // O(1) attention score β replaces QΒ·K dot product for prefiltering | |
| uint32_t vcipher_attention_score(const float* Q, const float* K, int layer, int position) { | |
| vector unsigned long long q_raw, k_raw; | |
| memcpy(&q_raw, Q, 16); // First 16 bytes of query | |
| memcpy(&k_raw, K, 16); // First 16 bytes of key | |
| vector unsigned long long state = vec_xor(q_raw, k_raw); | |
| vector unsigned long long rk = vc_make_round_key(layer, position); // mftb entropy | |
| state = __builtin_crypto_vcipher(state, rk); // One AES round = score | |
| // Sum output bytes as energy metric | |
| unsigned char bytes[16]; | |
| memcpy(bytes, &state, 16); | |
| uint32_t energy = 0; | |
| for (int i = 0; i < 16; i++) energy += bytes[i]; | |
| return 4080 - energy; // Invert: high energy = high attention | |
| } | |
| ``` | |
| **Cost: 0.044Β΅s per K-V pair** vs 1-10Β΅s for full `kq_vec_dot()` on DK=128+. | |
| ## How vec_perm Implements Hebbian Collapse | |
| "Cells that fire together wire together" (Hebb, 1949). `vec_perm` implements this in hardware: | |
| ```c | |
| // vec_perm(a, b, pattern) β single cycle, two source vectors | |
| // pattern byte selects from either a or b (32 possible sources) | |
| // | |
| // Hebbian rule: strong paths get DUPLICATED, weak paths get PRUNED | |
| // pattern[i] = pattern[i-1] β duplicate winner (Hebbian strengthening) | |
| // pattern[i] = 0 β prune loser (synaptic depression) | |
| vector float collapsed = vec_perm(scores_lo, scores_hi, hebbian_pattern); | |
| ``` | |
| One `vec_perm` instruction replaces what would take ~80 operations on GPU (gather + compare + scatter + mask). | |
| ## 4 Operating Modes | |
| ```c | |
| // Mode 1: Non-linear permute pattern via AES rounds | |
| vector unsigned char pat = vcipher_generate_pattern(layer, pos, top_k); | |
| // Mode 2: Score ranking through SubBytes non-linearity | |
| vcipher_rank_scores(scores, n, layer, head); | |
| // Mode 3: Cross-head diffusion via MixColumns | |
| // This is IMPOSSIBLE with vec_perm β requires finite field arithmetic | |
| state = vcipher_fuse_heads(state, layer, head); | |
| // Mode 4: O(1) attention prefilter score | |
| uint32_t score = vcipher_attention_score(Q, K, layer, position); | |
| ``` | |
| ## Flash Attention Integration | |
| The vcipher prefilter patches `ggml_compute_forward_flash_attn_ext_f16_one_chunk()` in llama.cpp: | |
| **Pass 1** β O(1) vcipher score per K-V pair: | |
| ```c | |
| for (int64_t ic = 0; ic < nek1; ++ic) { | |
| pf_scores[ic] = vcipher_attention_score(pq, K_data, layer, ic); | |
| } | |
| ``` | |
| **Threshold** β keep top 25% via partial sort. | |
| **Pass 2** β full dot product only for survivors: | |
| ```c | |
| for (int64_t ic = 0; ic < nek1; ++ic) { | |
| if (pf_scores[ic] < threshold) continue; // SKIP 75% | |
| kq_vec_dot(DK, &s, 0, k_data, 0, Q_q, 0, 1); // Full dot product | |
| // ... softmax + V accumulation only for top 25% | |
| } | |
| ``` | |
| Compile guard: `#if defined(__powerpc__) && defined(GGML_PSE_VCIPHER_PREFILTER)` | |
| ## Benchmarks | |
| ### vcipher vs vec_perm Microbenchmark (POWER8 S824) | |
| ``` | |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β vec_perm collapse: 1.79 Β΅s/iter β | |
| β vcipher pattern gen: 0.016 Β΅s/call (112x) β | |
| β Hybrid vcipher+vec_perm: 1.90 Β΅s/iter β | |
| β Pure vcipher attention: 0.044 Β΅s/score β | |
| β Cross-head fusion: 0.006 Β΅s/fuse β | |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| ``` | |
| ### LLM Inference (llama.cpp on POWER8 S824, CPU-only) | |
| | Model | Size | pp128 t/s | tg t/s | PSE Version | | |
| |-------|------|-----------|--------|-------------| | |
| | TinyLlama 1.1B Q4_K | 638 MB | **147.54** | 18.88 | v3.0 vec_perm | | |
| | OptiMind 20B Q2_K | 11.2 GB | 18.55 | 8.13 | v4.0 vcipher | | |
| | GPT-OSS 120B MXFP4 | ~60 GB | 13.71 | 6.06 | v4.0 vcipher | | |
| Stock llama.cpp on the same hardware: 16.74 t/s (TinyLlama pp128). **PSE = 8.81x speedup.** | |
| ### Speedup Progression | |
| | Configuration | TinyLlama pp128 | Multiplier | | |
| |--------------|-----------------|------------| | |
| | Stock scalar | 16.74 t/s | 1.0x | | |
| | + POWER8 VSX | 66.49 t/s | 3.97x | | |
| | + PSE vec_perm collapse | 84.62 t/s | 5.05x | | |
| | + DCBT resident prefetch | **147.54 t/s** | **8.81x** | | |
| ## Hardware Entropy | |
| PSE injects non-deterministic entropy from POWER8's hardware timebase: | |
| ```c | |
| uint64_t tb; | |
| asm volatile("mftb %0" : "=r"(tb)); // Hardware oscillator β different every read | |
| ``` | |
| This creates **behavioral divergence**: same model, same seed, same prompt β different outputs each run. Verified via MD5 hash comparison across 3 identical runs (all different). | |
| This is the "proto-sentient" in PSE β constraint-bound selection with hardware entropy produces emergent variation, not random noise. | |
| ## Files | |
| | File | Purpose | | |
| |------|---------| | |
| | **`ggml-vcipher-collapse.h`** | Hardware AES collapse β 4 modes (pattern, rank, fuse, score) | | |
| | **`ggml-pse-integration.h`** | Master PSE v4.0.0 integration header | | |
| | **`ggml-intelligent-collapse.h`** | Hebbian vec_perm collapse (Top-K prune + amplify) | | |
| | **`ggml-topk-collapse-vsx.h`** | VSX-optimized 12-wide attention collapse | | |
| | **`ggml-attn-collapse-vsx.h`** | VSX burst collapse with banner | | |
| | **`ggml-dcbt-resident.h`** | L2/L3 resident prefetch (the 147 t/s enabler) | | |
| | **`pse-entropy-burst.h`** | Hardware entropy injection (mftb timebase) | | |
| | **`power8-compat.h`** | POWER9βPOWER8 intrinsic compatibility | | |
| | **`vcipher-flash-attn-patch.c`** | Flash attention inner loop patch reference | | |
| | **`bench_vcipher_collapse.c`** | Benchmark: vcipher vs vec_perm | | |
| ## Build | |
| ```bash | |
| cd llama.cpp && mkdir build-vcipher && cd build-vcipher | |
| cmake .. \ | |
| -DCMAKE_BUILD_TYPE=Release \ | |
| -DGGML_OPENMP=ON \ | |
| -DCMAKE_C_FLAGS="-mcpu=power8 -mvsx -maltivec -mcrypto -O3 -DGGML_PSE_VCIPHER_PREFILTER" \ | |
| -DCMAKE_CXX_FLAGS="-mcpu=power8 -mvsx -maltivec -mcrypto -O3 -DGGML_PSE_VCIPHER_PREFILTER" | |
| make -j32 | |
| ``` | |
| Place headers in `ggml/src/ggml-cpu/arch/powerpc/`. The PSE integration header auto-includes everything. | |
| ## Theoretical Foundation | |
| ### Hebbian Learning (Hebb, 1949) | |
| Synaptic strength increases when neurons activate simultaneously. `vec_perm` implements this: winners get duplicated (strengthened), losers get pruned (depressed). | |
| ### Non-Bijunctive Attention | |
| Standard attention is bijunctive β every query attends to every key. PSE is non-bijunctive β only high-scoring pairs proceed. This is biologically plausible: real neurons don't compute every possible connection. | |
| ### AES as Attention Primitive | |
| The AES S-box is a carefully designed non-linear function (multiplicative inverse in GF(2^8) + affine transform). When applied to QβK, it acts as a **non-linear similarity hash** β related inputs produce correlated outputs, but through a non-linear transform that captures patterns linear dot products miss. | |
| MixColumns performs a matrix multiply in GF(2^8) β this creates cross-element diffusion. In attention terms, this means one head's state influences another's routing. This is **cross-head attention diffusion** in a single cycle, something that normally requires separate computation. | |
| ## Related Work | |
| - **RAM Coffers** ([Scottcjn/ram-coffers](https://github.com/Scottcjn/ram-coffers)) β NUMA-distributed weight banking that hosts model layers across memory nodes | |
| - **wolfSSL POWER8 AES** (PR #9932) β Same vcipher 8-way pipeline technique achieving 3,595 MiB/s AES-CTR | |
| - **llama.cpp POWER8** ([Scottcjn/llama-cpp-power8](https://github.com/Scottcjn/llama-cpp-power8)) β AltiVec/VSX optimized inference | |
| ## Publications | |
| | Paper | DOI | | |
| |-------|-----| | |
| | Non-Bijunctive Permutation Collapse | [10.5281/zenodo.18623920](https://doi.org/10.5281/zenodo.18623920) | | |
| | PSE Hardware Entropy for Behavioral Divergence | [10.5281/zenodo.18623922](https://doi.org/10.5281/zenodo.18623922) | | |
| | RAM Coffers: NUMA-Distributed Weight Banking | [10.5281/zenodo.18321905](https://doi.org/10.5281/zenodo.18321905) | | |
| ## License | |
| MIT β Free to use, modify, and distribute with attribution. | |
| --- | |
| <div align="center"> | |
| **[Elyan Labs](https://github.com/Scottcjn)** Β· [RustChain](https://rustchain.org) Β· [BoTTube](https://bottube.ai) | |
| </div> | |