Spaces:
Running
Running
| <html lang="en"> | |
| <head> | |
| <meta charset="utf-8"> | |
| <meta name="viewport" content="width=device-width,initial-scale=1"> | |
| <title>Snip Scope</title> | |
| <style> | |
| :root { color-scheme: dark; font-family: Inter, ui-sans-serif, system-ui; } | |
| * { box-sizing: border-box; } | |
| body { margin: 0; min-height: 100vh; background: #060817; color: #edf4ff; } | |
| canvas { position: fixed; inset: 0; width: 100%; height: 100%; opacity: .55; } | |
| main { position: relative; z-index: 1; width: min(1080px, 92vw); margin: auto; | |
| padding: 72px 0 96px; } | |
| .eyebrow { color: #73e6ff; letter-spacing: .18em; text-transform: uppercase; | |
| font-size: .75rem; font-weight: 800; } | |
| h1 { font-size: clamp(3rem, 8vw, 7rem); line-height: .9; margin: 14px 0 24px; | |
| background: linear-gradient(120deg,#fff,#74e7ff 55%,#b48cff); | |
| -webkit-background-clip: text; color: transparent; } | |
| .lead { max-width: 760px; color: #b8c7e6; font-size: 1.2rem; line-height: 1.65; } | |
| .actions { display: flex; flex-wrap: wrap; gap: 12px; margin: 30px 0 48px; } | |
| a { color: inherit; } | |
| .button { padding: 12px 18px; border-radius: 999px; text-decoration: none; | |
| background: #eaf8ff; color: #07101c; font-weight: 800; } | |
| .button.alt { background: #171d38cc; color: #dce8ff; border: 1px solid #415078; } | |
| .grid { display: grid; grid-template-columns: 1.1fr .9fr; gap: 20px; } | |
| .card { border: 1px solid #344269; background: #0c1128dd; border-radius: 24px; | |
| padding: 24px; backdrop-filter: blur(18px); box-shadow: 0 24px 80px #0008; } | |
| h2 { margin-top: 0; } | |
| pre { white-space: pre-wrap; word-break: break-word; color: #a9bddf; | |
| max-height: 520px; overflow: auto; } | |
| ul { max-height: 520px; overflow: auto; padding-left: 1.2rem; color: #a9bddf; } | |
| li { margin: 8px 0; } | |
| input { width: 100%; padding: 12px; border-radius: 12px; border: 1px solid #344269; | |
| background: #070b1a; color: white; margin-bottom: 12px; } | |
| @media (max-width: 780px) { .grid { grid-template-columns: 1fr; } } | |
| </style> | |
| </head> | |
| <body> | |
| <canvas id="field"></canvas> | |
| <main> | |
| <div class="eyebrow">Jacob Garcia · Hugging Face Model Foundry</div> | |
| <h1>Snip Scope</h1> | |
| <p class="lead">Interactive sparse-feature exemplar browser. This showcase backs up the | |
| trained artifacts, measured evaluation, and complete runnable source.</p> | |
| <div class="actions"> | |
| <a class="button" href="https://huggingface.co/spaces/ARotting/snip-scope/tree/main">Explore every file</a> | |
| <a class="button alt" href="https://huggingface.co/ARotting">View the full foundry</a> | |
| </div> | |
| <div class="grid"> | |
| <section class="card"> | |
| <h2>Verified project card</h2> | |
| <pre># SNIP Scope | |
| SNIP Scope trains a top-k sparse autoencoder on final-layer hidden activations from | |
| the pretrained SNIP-0.4M transformer. A 96-dimensional activation is encoded into a | |
| 384-feature overcomplete dictionary, but only the 16 largest positive features may | |
| fire for each token. | |
| The benchmark measures held-out reconstruction, explained variance, active-feature | |
| count, dead-feature rate, and token exemplars for each learned feature. A 16-component | |
| PCA reconstruction is retained as a dense low-rank control. Feature exemplars are | |
| descriptive clues, not proof that a neuron represents one human concept. | |
| ## Verified results | |
| The SAE trained on 180,000 final-layer token activations and was measured on 40,000 | |
| held-out activations. | |
| | Metric | Top-k SAE | PCA-16 control | | |
| | --- | ---: | ---: | | |
| | Reconstruction MSE | 0.02585 | 0.29824 | | |
| | Explained variance | 97.41% | 70.11% | | |
| | Active features per token | 16.00 | 16 dense components | | |
| | Dead dictionary features | 3.39% | not applicable | | |
| The learned dictionary contains 384 features and the SAE has 74,112 parameters. | |
| The Space exposes each feature's five highest-activating held-out BPE tokens and | |
| firing rate. Those exemplars may reflect token identity, syntax, position, or mixed | |
| causes; the project does not assign automatic human-readable concepts. | |
| ## Reproduce | |
| ```powershell | |
| uv run python projects/snip-scope/train.py | |
| ``` | |
| </pre> | |
| <h2>Evaluation snapshot</h2> | |
| <pre>{ | |
| "model": "SNIP Scope top-k sparse autoencoder", | |
| "source_model": "SNIP-0.4M base final hidden layer", | |
| "parameters": 74112, | |
| "training_tokens": 180000, | |
| "heldout_tokens": 40000, | |
| "input_dimension": 96, | |
| "dictionary_features": 384, | |
| "top_k": 16, | |
| "heldout": { | |
| "reconstruction_mse": 0.02585437148809433, | |
| "explained_variance": 0.9740850101195055, | |
| "mean_active_features": 16.0, | |
| "median_active_features": 16.0, | |
| "dead_feature_fraction": 0.033854166666666664 | |
| }, | |
| "pca_16_control": { | |
| "reconstruction_mse": 0.298243910074234, | |
| "explained_variance": 0.7010568067061413 | |
| }, | |
| "training_history": [ | |
| { | |
| "training_step": 100, | |
| "training_loss": 0.18427926301956177, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 200, | |
| "training_loss": 0.09680552780628204, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 300, | |
| "training_loss": 0.0745527446269989, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 400, | |
| "training_loss": 0.0621606670320034, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 500, | |
| "training_loss": 0.051943834871053696, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 600, | |
| "training_loss": 0.048014383763074875, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 700, | |
| "training_loss": 0.04393930733203888, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 800, | |
| "training_loss": 0.04151527211070061, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 900, | |
| "training_loss": 0.03839843347668648, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 1000, | |
| "training_loss": 0.03772205859422684, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 1100, | |
| "training_loss": 0.03755463287234306, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 1200, | |
| "training_loss": 0.03328678011894226, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 1300, | |
| "training_loss": 0.03515448421239853, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 1400, | |
| "training_loss": 0.03331328555941582, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 1500, | |
| "training_loss": 0.03026575781404972, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 1600, | |
| "training_loss": 0.03202933445572853, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 1700, | |
| "training_loss": 0.030269039794802666, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 1800, | |
| "training_loss": 0.029401661828160286, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 1900, | |
| "training_loss": 0.028512846678495407, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 2000, | |
| "training_loss": 0.02790793403983116, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 2100, | |
| "training_loss": 0.029049385339021683, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 2200, | |
| "training_loss": 0.026964839547872543, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 2300, | |
| "training_loss": 0.026676125824451447, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 2400, | |
| "training_loss": 0.024650562554597855, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 2500, | |
| "training_loss": 0.025682823732495308, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 2600, | |
| "training_loss": 0.025776153430342674, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 2700, | |
| "training_loss": 0.02404068596661091, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 2800, | |
| "training_loss": 0.025123972445726395, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 2900, | |
| "training_loss": 0.025397324934601784, | |
| "batch_active_features": 16.0 | |
| }, | |
| { | |
| "training_step": 3000, | |
| "training_loss": 0.02425968647003174, | |
| "batch_active_features": 16.0 | |
| } | |
| ] | |
| }</pre> | |
| </section> | |
| <section class="card"> | |
| <h2>Backed-up artifact tree</h2> | |
| <input id="filter" placeholder="Filter files…" autocomplete="off"> | |
| <ul id="files"><li><code>README.md</code></li> | |
| <li><code>__pycache__/app.cpython-311.pyc</code></li> | |
| <li><code>__pycache__/model.cpython-311.pyc</code></li> | |
| <li><code>app.py</code></li> | |
| <li><code>artifacts/snip-scope/evaluation.json</code></li> | |
| <li><code>artifacts/snip-scope/normalization.npz</code></li> | |
| <li><code>artifacts/snip-scope/sae.safetensors</code></li> | |
| <li><code>data/feature_exemplars.parquet</code></li> | |
| <li><code>model.py</code></li> | |
| <li><code>requirements.txt</code></li> | |
| <li><code>train.py</code></li></ul> | |
| </section> | |
| </div> | |
| </main> | |
| <script> | |
| const canvas=document.querySelector('#field'),ctx=canvas.getContext('2d'); | |
| let dots=[]; | |
| function resize(){canvas.width=innerWidth;canvas.height=innerHeight; | |
| dots=Array.from({length:90},()=>({x:Math.random()*innerWidth, | |
| y:Math.random()*innerHeight,vx:(Math.random()-.5)*.35,vy:(Math.random()-.5)*.35}));} | |
| function draw(){ctx.clearRect(0,0,canvas.width,canvas.height); | |
| for(const d of dots){d.x=(d.x+d.vx+innerWidth)%innerWidth; | |
| d.y=(d.y+d.vy+innerHeight)%innerHeight;ctx.fillStyle='#65dcff99'; | |
| ctx.beginPath();ctx.arc(d.x,d.y,1.4,0,7);ctx.fill();}requestAnimationFrame(draw);} | |
| addEventListener('resize',resize);resize();draw(); | |
| document.querySelector('#filter').addEventListener('input',e=>{ | |
| const q=e.target.value.toLowerCase();for(const li of document.querySelectorAll('li')) | |
| li.hidden=!li.textContent.toLowerCase().includes(q);}); | |
| </script> | |
| </body> | |
| </html> |