Spaces:

ARotting
/

factorial-code-forge-lab

Running

File size: 13,117 Bytes

fc5de42
9220556

<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>Factorial Code Forge Lab</title>
<style>
:root { color-scheme: dark; font-family: Inter, ui-sans-serif, system-ui; }
* { box-sizing: border-box; }
body { margin: 0; min-height: 100vh; background: #060817; color: #edf4ff; }
canvas { position: fixed; inset: 0; width: 100%; height: 100%; opacity: .55; }
main { position: relative; z-index: 1; width: min(1080px, 92vw); margin: auto;
  padding: 72px 0 96px; }
.eyebrow { color: #73e6ff; letter-spacing: .18em; text-transform: uppercase;
  font-size: .75rem; font-weight: 800; }
h1 { font-size: clamp(3rem, 8vw, 7rem); line-height: .9; margin: 14px 0 24px;
  background: linear-gradient(120deg,#fff,#74e7ff 55%,#b48cff);
  -webkit-background-clip: text; color: transparent; }
.lead { max-width: 760px; color: #b8c7e6; font-size: 1.2rem; line-height: 1.65; }
.actions { display: flex; flex-wrap: wrap; gap: 12px; margin: 30px 0 48px; }
a { color: inherit; }
.button { padding: 12px 18px; border-radius: 999px; text-decoration: none;
  background: #eaf8ff; color: #07101c; font-weight: 800; }
.button.alt { background: #171d38cc; color: #dce8ff; border: 1px solid #415078; }
.grid { display: grid; grid-template-columns: 1.1fr .9fr; gap: 20px; }
.card { border: 1px solid #344269; background: #0c1128dd; border-radius: 24px;
  padding: 24px; backdrop-filter: blur(18px); box-shadow: 0 24px 80px #0008; }
h2 { margin-top: 0; }
pre { white-space: pre-wrap; word-break: break-word; color: #a9bddf;
  max-height: 520px; overflow: auto; }
ul { max-height: 520px; overflow: auto; padding-left: 1.2rem; color: #a9bddf; }
li { margin: 8px 0; }
input { width: 100%; padding: 12px; border-radius: 12px; border: 1px solid #344269;
  background: #070b1a; color: white; margin-bottom: 12px; }
@media (max-width: 780px) { .grid { grid-template-columns: 1fr; } }
</style>
</head>
<body>
<canvas id="field"></canvas>
<main>
  <div class="eyebrow">Jacob Garcia · Hugging Face Model Foundry</div>
  <h1>Factorial Code Forge Lab</h1>
  <p class="lead">Interactive factorial latent-code explorer. This showcase backs up the
  trained artifacts, measured evaluation, and complete runnable source.</p>
  <div class="actions">
    <a class="button" href="https://huggingface.co/spaces/ARotting/factorial-code-forge-lab/tree/main">Explore every file</a>
    <a class="button alt" href="https://huggingface.co/ARotting">View the full foundry</a>
  </div>
  <div class="grid">
    <section class="card">
      <h2>Verified project card</h2>
      <pre># Factorial Code Forge

Factorial Code Forge mixes two statistically independent non-Gaussian sources and
asks a tiny autoencoder to recover a useful two-dimensional code. In the experimental
variant, two adversarial predictors try to infer each latent coordinate from the
other while the encoder tries to make those predictions fail. Reconstruction and
variance constraints prevent a constant-code shortcut.

The historical anchor is Schmidhuber&#x27;s 1992
[Learning Factorial Codes by Predictability
Minimization](https://doi.org/10.1162/neco.1992.4.6.863). The modern experiment
compares predictability minimization with an architecture-matched reconstructive
encoder, PCA whitening, and FastICA. It is a new controlled benchmark, not a
reproduction of the original paper&#x27;s implementation or datasets.

## Verified results

Models trained on 20,000 mixed samples and were measured once on 5,000 new samples:

| Code | Absolute correlation | Symmetric MI estimate |
| --- | ---: | ---: |
| Autoencoder control | 0.02200 | 0.22345 |
| Online neural-adversarial ablation | 0.01886 | 0.22678 |
| Validation-selected predictability minimum | 0.01158 | 0.00280 |
| PCA whitening | 0.02039 | 0.22275 |
| FastICA | 0.00590 | 0.00326 |

The selected 12-parameter linear codec reconstructed the mixtures at
`1.08e-14` MSE. Its rotation was selected using polynomial cross-predictors on a
held-out portion of the training set, never the test set. The online neural
minimax game did not improve nonlinear dependence and is retained as a negative
stability result rather than omitted.

## Reproduce

```powershell
uv run python projects/factorial-code-forge/train.py
```
</pre>
      <h2>Evaluation snapshot</h2>
      <pre>{
  &quot;benchmark&quot;: &quot;Two-source factorial code recovery&quot;,
  &quot;codec_parameters&quot;: 12,
  &quot;adversary_parameters&quot;: 1346,
  &quot;test_reconstruction_mse&quot;: 1.0849382441166684e-14,
  &quot;selected_rotation_radians&quot;: 0.6335255903647925,
  &quot;validation_polynomial_predictor_error&quot;: 2.0412044050069174,
  &quot;results&quot;: {
    &quot;autoencoder_control&quot;: {
      &quot;absolute_correlation&quot;: 0.02199843078760486,
      &quot;symmetric_mutual_information_estimate&quot;: 0.22345146065238453,
      &quot;linear_source_recovery_r2&quot;: 1.0
    },
    &quot;neural_adversarial_ablation&quot;: {
      &quot;absolute_correlation&quot;: 0.018864639802400177,
      &quot;symmetric_mutual_information_estimate&quot;: 0.22677681116066317,
      &quot;linear_source_recovery_r2&quot;: 1.0
    },
    &quot;predictability_minimization&quot;: {
      &quot;absolute_correlation&quot;: 0.011581345795229356,
      &quot;symmetric_mutual_information_estimate&quot;: 0.002795710068105528,
      &quot;linear_source_recovery_r2&quot;: 1.0
    },
    &quot;pca_whitened&quot;: {
      &quot;absolute_correlation&quot;: 0.02038519159561663,
      &quot;symmetric_mutual_information_estimate&quot;: 0.22274881647833844,
      &quot;linear_source_recovery_r2&quot;: 1.0
    },
    &quot;fastica&quot;: {
      &quot;absolute_correlation&quot;: 0.005903947226495519,
      &quot;symmetric_mutual_information_estimate&quot;: 0.003260830744820531,
      &quot;linear_source_recovery_r2&quot;: 1.0
    }
  },
  &quot;training_history&quot;: [
    {
      &quot;step&quot;: 100,
      &quot;reconstruction_loss&quot;: 0.5751938223838806,
      &quot;whitening_loss&quot;: 0.4999920129776001,
      &quot;predictor_loss&quot;: 0.03917834907770157
    },
    {
      &quot;step&quot;: 200,
      &quot;reconstruction_loss&quot;: 0.10436813533306122,
      &quot;whitening_loss&quot;: 0.4999969005584717,
      &quot;predictor_loss&quot;: 0.012760449200868607
    },
    {
      &quot;step&quot;: 300,
      &quot;reconstruction_loss&quot;: 0.09469099342823029,
      &quot;whitening_loss&quot;: 0.49983352422714233,
      &quot;predictor_loss&quot;: 0.0026066915597766638
    },
    {
      &quot;step&quot;: 400,
      &quot;reconstruction_loss&quot;: 0.0830593854188919,
      &quot;whitening_loss&quot;: 0.49922531843185425,
      &quot;predictor_loss&quot;: 0.004627525806427002
    },
    {
      &quot;step&quot;: 500,
      &quot;reconstruction_loss&quot;: 0.08060496300458908,
      &quot;whitening_loss&quot;: 0.4975007474422455,
      &quot;predictor_loss&quot;: 0.01004670187830925
    },
    {
      &quot;step&quot;: 600,
      &quot;reconstruction_loss&quot;: 0.07426909357309341,
      &quot;whitening_loss&quot;: 0.4930780827999115,
      &quot;predictor_loss&quot;: 0.02585626393556595
    },
    {
      &quot;step&quot;: 700,
      &quot;reconstruction_loss&quot;: 0.0617077499628067,
      &quot;whitening_loss&quot;: 0.48229295015335083,
      &quot;predictor_loss&quot;: 0.06387390196323395
    },
    {
      &quot;step&quot;: 800,
      &quot;reconstruction_loss&quot;: 0.040828146040439606,
      &quot;whitening_loss&quot;: 0.4678165912628174,
      &quot;predictor_loss&quot;: 0.1077573299407959
    },
    {
      &quot;step&quot;: 900,
      &quot;reconstruction_loss&quot;: 0.022691549733281136,
      &quot;whitening_loss&quot;: 0.3986736834049225,
      &quot;predictor_loss&quot;: 0.36198264360427856
    },
    {
      &quot;step&quot;: 1000,
      &quot;reconstruction_loss&quot;: 0.021030452102422714,
      &quot;whitening_loss&quot;: 0.23594792187213898,
      &quot;predictor_loss&quot;: 0.9913151860237122
    },
    {
      &quot;step&quot;: 1100,
      &quot;reconstruction_loss&quot;: 0.04022720083594322,
      &quot;whitening_loss&quot;: 0.009166543371975422,
      &quot;predictor_loss&quot;: 1.8695604801177979
    },
    {
      &quot;step&quot;: 1200,
      &quot;reconstruction_loss&quot;: 0.003848953638225794,
      &quot;whitening_loss&quot;: 0.00017961427511181682,
      &quot;predictor_loss&quot;: 1.7930779457092285
    },
    {
      &quot;step&quot;: 1300,
      &quot;reconstruction_loss&quot;: 0.0014683017507195473,
      &quot;whitening_loss&quot;: 0.004436715971678495,
      &quot;predictor_loss&quot;: 1.7725040912628174
    },
    {
      &quot;step&quot;: 1400,
      &quot;reconstruction_loss&quot;: 0.0005974461091682315,
      &quot;whitening_loss&quot;: 0.0012278903741389513,
      &quot;predictor_loss&quot;: 1.8158133029937744
    },
    {
      &quot;step&quot;: 1500,
      &quot;reconstruction_loss&quot;: 0.0003378715191502124,
      &quot;whitening_loss&quot;: 0.003598805284127593,
      &quot;predictor_loss&quot;: 1.7867335081100464
    },
    {
      &quot;step&quot;: 1600,
      &quot;reconstruction_loss&quot;: 6.85915001668036e-05,
      &quot;whitening_loss&quot;: 0.0005436374340206385,
      &quot;predictor_loss&quot;: 1.8233861923217773
    },
    {
      &quot;step&quot;: 1700,
      &quot;reconstruction_loss&quot;: 0.0001249544438906014,
      &quot;whitening_loss&quot;: 0.00044188593165017664,
      &quot;predictor_loss&quot;: 1.8262327909469604
    },
    {
      &quot;step&quot;: 1800,
      &quot;reconstruction_loss&quot;: 1.3117482012603432e-05,
      &quot;whitening_loss&quot;: 0.00011215780978091061,
      &quot;predictor_loss&quot;: 1.8308453559875488
    },
    {
      &quot;step&quot;: 1900,
      &quot;reconstruction_loss&quot;: 1.1593769158935174e-05,
      &quot;whitening_loss&quot;: 0.0016796954441815615,
      &quot;predictor_loss&quot;: 1.7771589756011963
    },
    {
      &quot;step&quot;: 2000,
      &quot;reconstruction_loss&quot;: 0.0004471217398531735,
      &quot;whitening_loss&quot;: 0.004560444969683886,
      &quot;predictor_loss&quot;: 1.7743375301361084
    },
    {
      &quot;step&quot;: 2100,
      &quot;reconstruction_loss&quot;: 2.6598576368996873e-05,
      &quot;whitening_loss&quot;: 0.0024056474212557077,
      &quot;predictor_loss&quot;: 1.7827980518341064
    },
    {
      &quot;step&quot;: 2200,
      &quot;reconstruction_loss&quot;: 4.374821946839802e-05,
      &quot;whitening_loss&quot;: 7.564837142126635e-05,
      &quot;predictor_loss&quot;: 1.8309648036956787
    },
    {
      &quot;step&quot;: 2300,
      &quot;reconstruction_loss&quot;: 0.00020850264991167933,
      &quot;whitening_loss&quot;: 0.005683479364961386,
      &quot;predictor_loss&quot;: 1.822058916091919
    },
    {
      &quot;step&quot;: 2400,
      &quot;reconstruction_loss&quot;: 2.0745503206853755e-05,
      &quot;whitening_loss&quot;: 8.849342520989012e-07,
      &quot;predictor_loss&quot;: 1.8361434936523438
    },
    {
      &quot;step&quot;: 2500,
      &quot;reconstruction_loss&quot;: 1.0664540241123177e-05,
      &quot;whitening_loss&quot;: 0.0009145042859017849,
      &quot;predictor_loss&quot;: 1.7433021068572998
    }
  ]
}</pre>
    </section>
    <section class="card">
      <h2>Backed-up artifact tree</h2>
      <input id="filter" placeholder="Filter files…" autocomplete="off">
      <ul id="files"><li><code>README.md</code></li>
<li><code>__pycache__/app.cpython-311.pyc</code></li>
<li><code>__pycache__/data.cpython-311.pyc</code></li>
<li><code>__pycache__/model.cpython-311.pyc</code></li>
<li><code>app.py</code></li>
<li><code>artifacts/factorial-code-forge/codec.safetensors</code></li>
<li><code>artifacts/factorial-code-forge/evaluation.json</code></li>
<li><code>artifacts/factorial-code-forge/latent_comparison.npz</code></li>
<li><code>artifacts/factorial-code-forge/predictor_01.safetensors</code></li>
<li><code>artifacts/factorial-code-forge/predictor_10.safetensors</code></li>
<li><code>data.py</code></li>
<li><code>data/factorial_sources.parquet</code></li>
<li><code>model.py</code></li>
<li><code>requirements.txt</code></li>
<li><code>train.py</code></li></ul>
    </section>
  </div>
</main>
<script>
const canvas=document.querySelector('#field'),ctx=canvas.getContext('2d');
let dots=[];
function resize(){canvas.width=innerWidth;canvas.height=innerHeight;
dots=Array.from({length:90},()=>({x:Math.random()*innerWidth,
y:Math.random()*innerHeight,vx:(Math.random()-.5)*.35,vy:(Math.random()-.5)*.35}));}
function draw(){ctx.clearRect(0,0,canvas.width,canvas.height);
for(const d of dots){d.x=(d.x+d.vx+innerWidth)%innerWidth;
d.y=(d.y+d.vy+innerHeight)%innerHeight;ctx.fillStyle='#65dcff99';
ctx.beginPath();ctx.arc(d.x,d.y,1.4,0,7);ctx.fill();}requestAnimationFrame(draw);}
addEventListener('resize',resize);resize();draw();
document.querySelector('#filter').addEventListener('input',e=>{
const q=e.target.value.toLowerCase();for(const li of document.querySelectorAll('li'))
li.hidden=!li.textContent.toLowerCase().includes(q);});
</script>
</body>
</html>