Update README.md
Browse files
README.md
CHANGED
|
@@ -16,7 +16,6 @@ SpikingLM is a BERT-base style masked-language model with spiking attention bloc
|
|
| 16 |
This checkpoint uses:
|
| 17 |
|
| 18 |
- temporal Spiking BERT with `T=4`
|
| 19 |
-
- learnable Q/K/V scaling parameters initialized from `7`
|
| 20 |
- LIF nodes for projection, Q, K, V, attention output, and MLP blocks
|
| 21 |
- `FP16OptimizedExp2Softmax` through `self.learnmax(attention_scores)` for attention normalization
|
| 22 |
|
|
|
|
| 16 |
This checkpoint uses:
|
| 17 |
|
| 18 |
- temporal Spiking BERT with `T=4`
|
|
|
|
| 19 |
- LIF nodes for projection, Q, K, V, attention output, and MLP blocks
|
| 20 |
- `FP16OptimizedExp2Softmax` through `self.learnmax(attention_scores)` for attention normalization
|
| 21 |
|