amewebstudio commited on
Commit
1587be1
·
verified ·
1 Parent(s): 6fd50bf

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +54 -0
README.md ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ tags:
4
+ - sparseflow
5
+ - sparse-attention
6
+ - conversational
7
+ - efficient
8
+ ---
9
+
10
+ # SparseFlow-Chat v5
11
+
12
+ An efficient conversational AI with **sparse attention** - achieving significant compute savings.
13
+
14
+ ## 🚀 Performance
15
+
16
+ | Metric | Value |
17
+ |--------|-------|
18
+ | Parameters | 39,840,002 |
19
+ | Perplexity | 1.00 |
20
+ | Token Sparsity | 87.5% |
21
+ | Attention Saved | 87.5% |
22
+
23
+ ## 🏗️ Architecture
24
+
25
+ - **Sparse Token Router**: O(n×k) instead of O(n²) attention
26
+ - **Persistent Memory Banks**: Store and retrieve knowledge
27
+ - **Channel Sparsity**: Activates only top-k channels
28
+
29
+ ### Complexity Comparison
30
+
31
+ | Operation | Transformer | SparseFlow | Speedup |
32
+ |-----------|-------------|------------|--------|
33
+ | Attention | O(n²) | O(n×k) | 8x |
34
+ | FFN | O(n×d²) | O(n×k×d) | ~4x |
35
+
36
+ ## 💬 Usage
37
+
38
+ ```python
39
+ # Load model
40
+ import torch
41
+ checkpoint = torch.load("model.pt")
42
+ # ... initialize model with config.json
43
+ model.load_state_dict(checkpoint['model'])
44
+
45
+ # Chat
46
+ response = chat("What is the capital of France?")
47
+ # -> "The capital of France is Paris."
48
+ ```
49
+
50
+ ## 📝 Created By
51
+
52
+ **Logo (Mike Amega)** — [Ame Web Studio](https://github.com/AmeWebStudio)
53
+
54
+ February 2025