Update README.md
Browse files
README.md
CHANGED
|
@@ -1,11 +1,10 @@
|
|
| 1 |
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
base_model: Qwen/Qwen2-0.5B
|
| 4 |
tags:
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
---
|
| 10 |
|
| 11 |
# **I made Qwen2-0.5B run 1000× smaller with no matrix multiplications and almost no RAM usage — adaptive sparse selection at inference time, and it actually works.**
|
|
@@ -657,6 +656,4 @@ if __name__ == "__main__":
|
|
| 657 |
|
| 658 |
- Does the flagged feature subset stay stable across different prompts, or does it change drastically session to session?
|
| 659 |
- Is there a relationship between which features get flagged and attention patterns in the same layer?
|
| 660 |
-
- Right now `BAYES_EVERY_N = 8` and the deviation threshold (`> 1.0` std) are picked without tuning — sweeping these would show whether the flagged fraction is a real signal or just a threshold artifact.
|
| 661 |
-
|
| 662 |
-
|
|
|
|
| 1 |
---
|
|
|
|
| 2 |
base_model: Qwen/Qwen2-0.5B
|
| 3 |
tags:
|
| 4 |
+
- experimental
|
| 5 |
+
- inference
|
| 6 |
+
- feature-selection
|
| 7 |
+
- bayesian
|
| 8 |
---
|
| 9 |
|
| 10 |
# **I made Qwen2-0.5B run 1000× smaller with no matrix multiplications and almost no RAM usage — adaptive sparse selection at inference time, and it actually works.**
|
|
|
|
| 656 |
|
| 657 |
- Does the flagged feature subset stay stable across different prompts, or does it change drastically session to session?
|
| 658 |
- Is there a relationship between which features get flagged and attention patterns in the same layer?
|
| 659 |
+
- Right now `BAYES_EVERY_N = 8` and the deviation threshold (`> 1.0` std) are picked without tuning — sweeping these would show whether the flagged fraction is a real signal or just a threshold artifact.
|
|
|
|
|
|