File size: 7,045 Bytes
86ceb2b a280831 86ceb2b a280831 86ceb2b a280831 86ceb2b a280831 86ceb2b a280831 86ceb2b a280831 86ceb2b a280831 86ceb2b a280831 86ceb2b a280831 86ceb2b a280831 86ceb2b a280831 86ceb2b a280831 86ceb2b a280831 86ceb2b a280831 86ceb2b a280831 86ceb2b a280831 86ceb2b a280831 86ceb2b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 |
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="description"
content="AI Now Lives in Time: Temporal Dense Networks for Distributed Generalization.">
<meta name="keywords" content="Temporal AI, DenseNet, Neural Networks, PyTorch, XOR Generalization">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Temporal DenseNet: AI Now Lives in Time</title>
<link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro" rel="stylesheet">
<link rel="stylesheet" href="./static/css/bulma.min.css">
<link rel="stylesheet" href="./static/css/bulma-carousel.min.css">
<link rel="stylesheet" href="./static/css/bulma-slider.min.css">
<link rel="stylesheet" href="./static/css/fontawesome.all.min.css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css">
<link rel="stylesheet" href="./static/css/index.css">
<link rel="icon" href="./static/images/favicon.svg">
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
<script defer src="./static/js/fontawesome.all.min.js"></script>
<script src="./static/js/bulma-carousel.min.js"></script>
<script src="./static/js/bulma-slider.min.js"></script>
</head>
<body>
<section class="hero">
<div class="hero-body">
<div class="container is-max-desktop">
<div class="columns is-centered">
<div class="column has-text-centered">
<h1 class="title is-1 publication-title">AI Now Lives in Time: Temporal DenseNet</h1>
<div class="is-size-5 publication-authors">
<span class="author-block">Independent Research</span>
</div>
<div class="column has-text-centered">
<div class="publication-links">
<span class="link-block">
<a href="#code-section" class="button is-normal is-rounded is-dark">
<span class="icon"><i class="fab fa-github"></i></span>
<span>Code</span>
</a>
</span>
</div>
</div>
</div>
</div>
</div>
</div>
</section>
<section class="section">
<div class="container is-max-desktop">
<div class="columns is-centered has-text-centered">
<div class="column is-four-fifths">
<h2 class="title is-3">Abstract</h2>
<div class="content has-text-justified">
<p>
This network is a fully connected “temporal” architecture where each neuron sees not only the neurons in earlier layers of the current computation, but also all neurons from the previous computation step (tick).
</p>
<p>
By spreading information across layers and time, no single neuron can memorize an input-output pair directly. Instead, the network learns patterns in a distributed way, naturally favoring generalization over memorization. This makes it useful for tasks where datasets are small or where overfitting is a risk, because the architecture itself prevents simple lookup-table memorization and encourages learning the underlying rules.
</p>
</div>
</div>
</div>
</div>
</section>
<section class="section">
<div class="container is-max-desktop">
<div class="columns is-centered">
<div class="column is-full-width">
<h2 class="title is-3">Temporal Accumulation Results</h2>
<div class="columns is-vcentered">
<div class="column">
<p>
The model demonstrates extreme precision on the XOR dataset by utilizing <strong>time ticks</strong> to accumulate state.
</p>
<ul style="list-style-type: square; margin-left: 20px;">
<li><strong>Final Loss:</strong> 0.000002</li>
<li><strong>Ticks:</strong> 3 iterative steps</li>
<li><strong>Structure:</strong> 8x8x8 Hidden layers</li>
</ul>
</div>
<div class="column">
<table class="table is-narrow is-fullwidth is-bordered">
<thead>
<tr><th>Epoch</th><th>MSE Loss</th></tr>
</thead>
<tbody>
<tr><td>200</td><td>0.000091</td></tr>
<tr><td>1000</td><td>0.000008</td></tr>
<tr><td>2000</td><td>0.000002</td></tr>
</tbody>
</table>
</div>
</div>
<h3 class="title is-4">Final Prediction Accuracy</h3>
<p>The network achieves near-perfect separation for non-linear logic:</p>
<pre style="background: #232323; color: #00ff00; padding: 15px; border-radius: 8px;">
Raw Predictions:
[[7.0460670e-04] [9.9793684e-01] [9.9922490e-01] [2.1156450e-03]]
Rounded: [0, 1, 1, 0]</pre>
</div>
</div>
</div>
</section>
<section class="section" id="code-section">
<div class="container is-max-desktop">
<h2 class="title is-3">PyTorch Implementation</h2>
<div class="content">
<p>The architecture uses <code>nn.ModuleList</code> to manage current tick layers ($U$) and previous tick recurrence ($W$).</p>
<pre style="background-color: #f5f5f5; padding: 20px; border-radius: 10px; border: 1px solid #ddd; font-size: 0.9em;"><code>
import torch
import torch.nn as nn
class TemporalDenseNet(nn.Module):
def __init__(self, input_size, hidden_sizes, output_size):
super().__init__()
self.num_layers = len(hidden_sizes)
self.hidden_sizes = hidden_sizes
self.prev_concat_size = sum(hidden_sizes)
# Current-tick linear layers U[i]
self.U = nn.ModuleList()
for i in range(self.num_layers):
in_size = input_size if i == 0 else sum(hidden_sizes[:i])
self.U.append(nn.Linear(in_size, hidden_sizes[i]))
# Previous-tick linear layers W[i]
self.W = nn.ModuleList([nn.Linear(self.prev_concat_size, hidden_sizes[i])
for i in range(self.num_layers)])
self.out = nn.Linear(self.prev_concat_size, output_size)
self.activation = torch.tanh
def forward(self, x, prev_outputs=None):
layer_outputs = []
prev_cat = torch.cat(prev_outputs, dim=1) if prev_outputs is not None else None
for i in range(self.num_layers):
current_input = x if i == 0 else torch.cat(layer_outputs, dim=1)
out = self.U[i](current_input)
if prev_cat is not None:
out = out + self.W[i](prev_cat)
out = self.activation(out)
layer_outputs.append(out)
final_cat = torch.cat(layer_outputs, dim=1)
return layer_outputs, torch.sigmoid(self.out(final_cat))
</code></pre>
</div>
</div>
</section>
<footer class="footer">
<div class="container">
<div class="content has-text-centered">
<p>Research integrated from <code>timeBasedAIDense.py</code>.</p>
</div>
</div>
</footer>
</body>
</html>
|