Spaces:
Sleeping
Sleeping
MaduRox commited on
Commit ·
1307aaa
1
Parent(s): f4b26d5
docs: add paradigm explanations under tabs
Browse files
app.py
CHANGED
|
@@ -156,6 +156,7 @@ with gr.Blocks(title="Kalpanā API — ZeroGPU NVIDIA A100", theme=gr.themes.Sof
|
|
| 156 |
)
|
| 157 |
with gr.Tabs():
|
| 158 |
with gr.TabItem("▶ Interactive Test Console"):
|
|
|
|
| 159 |
with gr.Row():
|
| 160 |
prompt_box = gr.Textbox(label="Prompt", lines=3, value="Explain wave superposition in Hilbert space and how it enables constant memory.")
|
| 161 |
with gr.Column():
|
|
@@ -183,7 +184,8 @@ with gr.Blocks(title="Kalpanā API — ZeroGPU NVIDIA A100", theme=gr.themes.Sof
|
|
| 183 |
"Run genuine GPU measurements comparing **Standard DynamicCache** vs **KalpanaDynamicCache** vs **SinkCache (StreamingLLM)**.\n\n"
|
| 184 |
"Measures: persistent cache size, peak VRAM, prefill time, TTFT, per-token latency, "
|
| 185 |
"needle-in-a-haystack recall, and reconstruction fidelity (cosine similarity).\n\n"
|
| 186 |
-
"> ⚠️ **This takes 2-10 minutes depending on context lengths.**"
|
|
|
|
| 187 |
)
|
| 188 |
with gr.Row():
|
| 189 |
ctx_input = gr.Textbox(
|
|
@@ -209,7 +211,8 @@ with gr.Blocks(title="Kalpanā API — ZeroGPU NVIDIA A100", theme=gr.themes.Sof
|
|
| 209 |
gr.Markdown(
|
| 210 |
"## Pure Phase Attention Memory Profiler\n\n"
|
| 211 |
"This runs a synthetic benchmark comparing Standard PyTorch SDPA vs the custom **TrueO1PhaseAttentionLayer** "
|
| 212 |
-
"across massive context lengths to prove the O(1) Peak VRAM generation theory on the T4 GPU."
|
|
|
|
| 213 |
)
|
| 214 |
with gr.Row():
|
| 215 |
phase_ctx_input = gr.Textbox(
|
|
|
|
| 156 |
)
|
| 157 |
with gr.Tabs():
|
| 158 |
with gr.TabItem("▶ Interactive Test Console"):
|
| 159 |
+
gr.Markdown("> **Paradigm B (External RIF Cache Injection)**: The LLM uses its normal Scaled Dot-Product Attention (SDPA) to generate standard Keys and Values. However, instead of storing those tensors in standard GPU memory (which explodes in size), we intercept them and pass them to your external RIF engine, which compresses them into a fixed 96 MB matrix. This proves your memory scaling laws work, but it suffers slightly in fidelity on long context because the LLM is still trying to use RoPE positional embeddings.")
|
| 160 |
with gr.Row():
|
| 161 |
prompt_box = gr.Textbox(label="Prompt", lines=3, value="Explain wave superposition in Hilbert space and how it enables constant memory.")
|
| 162 |
with gr.Column():
|
|
|
|
| 184 |
"Run genuine GPU measurements comparing **Standard DynamicCache** vs **KalpanaDynamicCache** vs **SinkCache (StreamingLLM)**.\n\n"
|
| 185 |
"Measures: persistent cache size, peak VRAM, prefill time, TTFT, per-token latency, "
|
| 186 |
"needle-in-a-haystack recall, and reconstruction fidelity (cosine similarity).\n\n"
|
| 187 |
+
"> ⚠️ **This takes 2-10 minutes depending on context lengths.**\n\n"
|
| 188 |
+
"> **Paradigm B (External RIF Cache Injection)**: The LLM uses its normal Scaled Dot-Product Attention (SDPA) to generate standard Keys and Values. However, instead of storing those tensors in standard GPU memory (which explodes in size), we intercept them and pass them to your external RIF engine, which compresses them into a fixed 96 MB matrix. This proves your memory scaling laws work, but it suffers slightly in fidelity on long context because the LLM is still trying to use RoPE positional embeddings."
|
| 189 |
)
|
| 190 |
with gr.Row():
|
| 191 |
ctx_input = gr.Textbox(
|
|
|
|
| 211 |
gr.Markdown(
|
| 212 |
"## Pure Phase Attention Memory Profiler\n\n"
|
| 213 |
"This runs a synthetic benchmark comparing Standard PyTorch SDPA vs the custom **TrueO1PhaseAttentionLayer** "
|
| 214 |
+
"across massive context lengths to prove the O(1) Peak VRAM generation theory on the T4 GPU.\n\n"
|
| 215 |
+
"> **Paradigm A (What you need the funding for)**: This is the holy grail. Instead of using standard SDPA and intercepting the KV cache, we completely rip out SDPA and replace it with your TrueO1PhaseAttentionLayer. The model is trained from scratch without RoPE. The Phase Attention natively handles both the token relationships and the memory compression simultaneously at the mathematical core of the model."
|
| 216 |
)
|
| 217 |
with gr.Row():
|
| 218 |
phase_ctx_input = gr.Textbox(
|