defnoteatin commited on
Commit
d65bd23
·
verified ·
1 Parent(s): 86f5a34

Update README.md with dense global aggregation, non-overlapping proportional RoPE, and resilient vLLM hooks

Browse files
Files changed (1) hide show
  1. README.md +4 -3
README.md CHANGED
@@ -16,7 +16,7 @@ All **10 model families (`40` repositories)** run through our single universal `
16
 
17
  | Model Family | Params (Total / Active) | 1M KV Reduction | **4-Bit `W4A16` (RTX 30/40/AMD)** | **8-Bit `FP8` (4090/H100/MI300X)** | **4-Bit `NVFP4` (Blackwell/Spark)** | **16-Bit `BF16` (Universal)** |
18
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: |
19
- | **PseudoBrain-Qwen3.8-Flash-Next** | `125B (+51B PLE)` / `6B` | **`96.0%+`** (`~48.0 GB` → `~486 MiB active window`) | [`W4A16`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3.8-Flash-Next-W4A16) (~90.2 GB) | [`FP8`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3.8-Flash-Next-FP8) (~185.5 GB) | [`NVFP4`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3.8-Flash-Next-NVFP4) (~106.3 GB) | [`BF16`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3.8-Flash-Next-BF16) (~360.0 GB) |
20
  | **PseudoBrain-Qwen3-Coder-Next** | `80B` / `3.0B` | **`93.5%`** (`~48.0 GB` → `~3.1 GB`) | [`W4A16`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3-Coder-Next-W4A16) (~41.5 GB) | [`FP8`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3-Coder-Next-FP8) (~80.5 GB) | [`NVFP4`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3-Coder-Next-NVFP4) (~41.5 GB) | [`BF16`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3-Coder-Next-BF16) (~160.0 GB) |
21
  | **PseudoBrain-Qwen3-Coder-30B-A3B** | `30.5B` / `3.3B` | **`98.4%`** (`192.0 GB` → `~3.1 GB`) | [`W4A16`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3-Coder-30B-A3B-W4A16) (~16.5 GB) | [`FP8`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3-Coder-30B-A3B-FP8) (~31.0 GB) | [`NVFP4`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3-Coder-30B-A3B-NVFP4) (~16.5 GB) | [`BF16`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3-Coder-30B-A3B-BF16) (~61.0 GB) |
22
  | **PseudoBrain-Nex-35B-A3B** | `35B` / `~3.0B` | **`97.6%`** (`~20.0 GB` → `~486 MiB active window`) | [`W4A16`](https://huggingface.co/defnoteatin/PseudoBrain-Nex-35B-A3B-W4A16) (~18.5 GB) | [`FP8`](https://huggingface.co/defnoteatin/PseudoBrain-Nex-35B-A3B-FP8) (~35.0 GB) | [`NVFP4`](https://huggingface.co/defnoteatin/PseudoBrain-Nex-35B-A3B-NVFP4) (~22.5 GB) | [`BF16`](https://huggingface.co/defnoteatin/PseudoBrain-Nex-35B-A3B-BF16) (~70.0 GB) |
@@ -74,8 +74,9 @@ In standard Transformer deployment, **KV cache memory and quadratic prefill —
74
  - A single user at `1,000,000` tokens on `Qwen3-32B` requires **`256 GB` of KV cache VRAM** (`192 GB` on `Qwen3-Coder-30B-A3B`; `64 GB` on `Qwen3.8-27B`).
75
  - **PseudoBrain** converts unbounded full-attention layers into **Hybrid 16K Sliding-Window Attention + Parallel Gated DeltaNet (`GDN`)** blocks paired with our **Three-Tier NVMe/Page-Cache Quest Exact Memory Hierarchy** and **True $O(1)$ Constant-Speed Decode Engine** inside vLLM (`serving/pb27_vllm/`):
76
  - **Recurrent Gated DeltaNet Branch (`O(1)` State):** Initialized directly from teacher `self_attn` projections (`init_gdn_from_attention`) with zero-initialized output projections (`out=0`), compressing arbitrary-length global context into a fixed-size recurrent state.
77
- - **Quest Exact Memory + FlashAttention-2 LSE Merge (`piece_attention`):** Archives rotated keys/values in `64`-token blocks with **2-Level Hierarchical `bf16` Quest Super-Block Bounds (`4,096` tokens/super-block)** and **Priority-Aware RoPE Sink Repositioning (`_reposition_rope_sinks`)**, recomputing attention over `[4 Sinks + Top-k Retrieved Pages + 16K Window]` without static YaRN `factor=4.0` short-context degradation.
78
- - **True $O(1)$ Decode Throughput (`DECODE_STRIDE=64`, `DECODE_BLOCKS=64`, `DECODE_RERANK=0`):** Warms `cached_pre_k` / `cached_pre_v` at the end of `final_prefill` so multi-token speculative verification runs directly in GPU HBM with zero NVMe page traffic for rejected candidates.
 
79
 
80
  ---
81
 
 
16
 
17
  | Model Family | Params (Total / Active) | 1M KV Reduction | **4-Bit `W4A16` (RTX 30/40/AMD)** | **8-Bit `FP8` (4090/H100/MI300X)** | **4-Bit `NVFP4` (Blackwell/Spark)** | **16-Bit `BF16` (Universal)** |
18
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: |
19
+ | **PseudoBrain-Qwen3.8-Flash-Next** | `36B` / `~1.7B` | **`96.0%+`** (`~48.0 GB` → `~486 MiB active window`) | [`W4A16`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3.8-Flash-Next-W4A16) (~21.0 GB) | [`FP8`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3.8-Flash-Next-FP8) (~36.5 GB) | [`NVFP4`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3.8-Flash-Next-NVFP4) (~21.0 GB) | [`BF16`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3.8-Flash-Next-BF16) (~72.0 GB) |
20
  | **PseudoBrain-Qwen3-Coder-Next** | `80B` / `3.0B` | **`93.5%`** (`~48.0 GB` → `~3.1 GB`) | [`W4A16`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3-Coder-Next-W4A16) (~41.5 GB) | [`FP8`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3-Coder-Next-FP8) (~80.5 GB) | [`NVFP4`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3-Coder-Next-NVFP4) (~41.5 GB) | [`BF16`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3-Coder-Next-BF16) (~160.0 GB) |
21
  | **PseudoBrain-Qwen3-Coder-30B-A3B** | `30.5B` / `3.3B` | **`98.4%`** (`192.0 GB` → `~3.1 GB`) | [`W4A16`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3-Coder-30B-A3B-W4A16) (~16.5 GB) | [`FP8`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3-Coder-30B-A3B-FP8) (~31.0 GB) | [`NVFP4`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3-Coder-30B-A3B-NVFP4) (~16.5 GB) | [`BF16`](https://huggingface.co/defnoteatin/PseudoBrain-Qwen3-Coder-30B-A3B-BF16) (~61.0 GB) |
22
  | **PseudoBrain-Nex-35B-A3B** | `35B` / `~3.0B` | **`97.6%`** (`~20.0 GB` → `~486 MiB active window`) | [`W4A16`](https://huggingface.co/defnoteatin/PseudoBrain-Nex-35B-A3B-W4A16) (~18.5 GB) | [`FP8`](https://huggingface.co/defnoteatin/PseudoBrain-Nex-35B-A3B-FP8) (~35.0 GB) | [`NVFP4`](https://huggingface.co/defnoteatin/PseudoBrain-Nex-35B-A3B-NVFP4) (~22.5 GB) | [`BF16`](https://huggingface.co/defnoteatin/PseudoBrain-Nex-35B-A3B-BF16) (~70.0 GB) |
 
74
  - A single user at `1,000,000` tokens on `Qwen3-32B` requires **`256 GB` of KV cache VRAM** (`192 GB` on `Qwen3-Coder-30B-A3B`; `64 GB` on `Qwen3.8-27B`).
75
  - **PseudoBrain** converts unbounded full-attention layers into **Hybrid 16K Sliding-Window Attention + Parallel Gated DeltaNet (`GDN`)** blocks paired with our **Three-Tier NVMe/Page-Cache Quest Exact Memory Hierarchy** and **True $O(1)$ Constant-Speed Decode Engine** inside vLLM (`serving/pb27_vllm/`):
76
  - **Recurrent Gated DeltaNet Branch (`O(1)` State):** Initialized directly from teacher `self_attn` projections (`init_gdn_from_attention`) with zero-initialized output projections (`out=0`), compressing arbitrary-length global context into a fixed-size recurrent state.
77
+ - **Piecewise-Proportional, Strictly Non-Overlapping Block RoPE (`_reposition_rope_blocks`):** Enforces a mandatory `64`-token footprint per retrieved block plus proportional historical-gap scaling ($T(b_j) = P_{ ext{start}} + j \cdot B + ext{round}( rac{G_{ ext{cum},j}}{G_{ ext{tot}}} S_{ ext{slack}})$) — preserving exact `+1` local continuity across contiguous block runs, strictly preventing phase overlap between nearby non-adjacent blocks, and maintaining **99%+ proportional macro-timeline distance fidelity** at `1M` tokens without static YaRN `factor=4.0` short-context degradation.
78
+ - **Dense Global Aggregation (`PB27_MEMORY_GLOBAL_AGG="auto"`):** Automatically detects high-participation-ratio global queries (`_detect_global_agg`) and activates **(a) Resident Super-Block Summary Landmarks (`smean_k`, `smean_v`, 1 summary token per `4,096` tokens in GPU HBM)**, **(b) Stratified Super-Block Timeline Coverage**, and **(c) Novelty-Rotating Decode Working-Set Sweeps (`visit_counts`)** so multi-step reasoning sweeps across hundreds of distinct blocks over `1M` tokens.
79
+ - **True $O(1)$ Decode Throughput & Version-Resilient `vLLM` Plugin:** Warms `cached_pre_k` / `cached_pre_v` at the end of `final_prefill` so multi-token speculative verification runs directly in GPU HBM with zero NVMe page traffic for rejected candidates, with dual `vLLM` V1/V2 runner support and a zero-hook `ForwardContext` fallback.
80
 
81
  ---
82