Spaces:
Running
Running
| # Claim 5 source audit | |
| Source HTML SHA-256: | |
| `6173e746b37f95c44a391974f88c622e8ae77a3d1ca792bdfffb09f5c85a2aa1`. | |
| Anchor: `Thmdefinition2`. | |
| Definition 2 specifies a ReLU feature encoder `phi_theta: U -> Delta^(m-1)`, | |
| the context aggregation `k^{-1} sum_i phi_theta(x_i,y_i)`, and a clipped ReLU | |
| decoder that also receives the query. The preceding paragraph identifies this | |
| as uniform attention with `Q=K=0`. | |
| For any context length `k>=1`, scaled dot-product attention has all-zero scores. | |
| Softmax therefore assigns exact weight `1/k` to every value vector. Setting | |
| `V_i=phi_theta(x_i,y_i)` gives the displayed Definition 2 expression exactly. | |