hiaripc commited on
Commit
36b3422
·
1 Parent(s): 12c1391
Files changed (1) hide show
  1. index.html +101 -73
index.html CHANGED
@@ -1,39 +1,63 @@
1
  <!DOCTYPE html>
 
2
  <html>
3
  <head>
4
  <meta charset="UTF-8">
5
  <title>SweetSpot v2 — Energy-per-Token demo</title>
6
 
7
- <script src="https://cdn.plot.ly/plotly-latest.min.js"></script>
8
-
9
- <style>
10
- body {
11
- font-family: sans-serif;
12
- max-width: 1000px;
13
- margin: auto;
14
- padding: 20px;
15
- }
16
-
17
- .row {
18
- display: flex;
19
- gap: 20px;
20
- flex-wrap: wrap;
21
- }
22
-
23
- label {
24
- display: block;
25
- margin-top: 10px;
26
- }
27
-
28
- input {
29
- width: 120px;
30
- }
31
-
32
- button {
33
- margin-top: 10px;
34
- padding: 8px 15px;
35
- }
36
- </style>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37
  </head>
38
 
39
  <body>
@@ -43,51 +67,62 @@ SweetSpot v2 — Energy-per-Token demo
43
  </h2>
44
 
45
  <p>
46
- This interactive demo visualizes our analytical model for predicting the <b>energy consumption per generated token</b> as a function of the number of output tokens (<i>n</i><sub>out</sub>).
47
- Select one of the validated models used in the paper or load the architecture of any <b>dense decoder-only Transformer</b> (up to approximately 20B parameters) from Hugging Face to explore how the different terms contribute to the prediction.
 
 
 
 
 
48
  </p>
49
 
50
  <p>
51
- The model was fitted using empirical measurements collected on an <b>NVIDIA H100 GPU</b> with <b>NVIDIA TensorRT-LLM</b> under the following experimental configuration:
 
 
 
 
52
  <ul>
53
  <li><code>max_batch_size = 1024</code></li>
54
  <li><code>max_num_tokens = 8192</code></li>
55
  <li><code>num_requests = 1000</code></li>
56
  </ul>
57
 
58
- Future releases will extend the analysis to additional hardware platforms and inference frameworks.
 
 
59
  </p>
60
 
61
  <p>
62
- For the complete methodology, experimental validation, and citation information, please see the paper:
 
63
  <a href="YOUR_PAPER_URL" target="_blank">
64
- SweetSpot v2: An Architecture-Aware Analytical Model for Predicting Energy Efficiency of LLM Inference
 
65
  </a>.
66
  </p>
67
 
 
68
 
69
- <h4>
70
- Validated models:
71
- </h4>
72
-
73
- <select id="preset">
74
- </select>
75
 
76
  <button onclick="loadPreset()">
77
- Load selected model
78
  </button>
79
 
80
  <h4>
81
- Or load another public Hugging Face model (dense, < 20B):
82
  </h4>
83
 
84
- <input
85
  id="repo"
86
  style="width:400px"
87
- placeholder="tiiuae/falcon-rw-7b">
 
 
88
 
89
  <button onclick="loadModel()">
90
- Load model
91
  </button>
92
 
93
  <p id="status"></p>
@@ -96,28 +131,35 @@ Load model
96
 
97
  <div class="row">
98
 
 
99
  <div>
100
- <label>Hidden size (d):</label>
101
- <input id="d" type="number" value="4096">
102
  </div>
103
 
104
  <div>
105
- <label>Layers (L):</label>
106
- <input id="L" type="number" value="36">
107
  </div>
108
 
109
  <div>
110
- <label>Query heads (<i>n</i><sub>q</sub>):</label>
111
- <input id="nq" type="number" value="64">
112
  </div>
113
 
114
  <div>
115
- <label>KV heads (<i>n</i><sub>kv</sub>):</label>
116
- <input id="nkv" type="number" value="64">
 
 
 
117
  </div>
118
 
 
 
119
  <div id="ninWrapper">
120
 
 
121
  <label>
122
  Input Tokens (<i>n</i><sub>in</sub>):
123
  <span id="ninValue">64</span>
@@ -138,23 +180,13 @@ Load model
138
  <span>4096</span>
139
  </div>
140
 
 
141
  <input id="nin" type="hidden" value="64">
 
142
 
143
  </div>
144
 
145
- <style>
146
- #ninSlider {
147
- width: 100%;
148
- display: block;
149
- }
150
-
151
- #ninLabels {
152
- display: flex;
153
- justify-content: space-between;
154
- font-size: 12px;
155
- color: #555;
156
- }
157
- </style>
158
 
159
  <script>
160
  const MAX_TOKENS = 4096;
@@ -170,11 +202,7 @@ Load model
170
  }
171
  </script>
172
 
173
-
174
- <div id="plot"></div>
175
-
176
-
177
  <script src="app.js"></script>
178
 
179
  </body>
180
- </html>
 
1
  <!DOCTYPE html>
2
+
3
  <html>
4
  <head>
5
  <meta charset="UTF-8">
6
  <title>SweetSpot v2 — Energy-per-Token demo</title>
7
 
8
+ ```
9
+ <script src="https://cdn.plot.ly/plotly-latest.min.js"></script>
10
+
11
+ <style>
12
+ body {
13
+ font-family: sans-serif;
14
+ max-width: 1000px;
15
+ margin: auto;
16
+ padding: 20px;
17
+ }
18
+
19
+ .row {
20
+ display: flex;
21
+ gap: 20px;
22
+ flex-wrap: wrap;
23
+ }
24
+
25
+ label {
26
+ display: block;
27
+ margin-top: 10px;
28
+ }
29
+
30
+ input[type="number"] {
31
+ width: 120px;
32
+ }
33
+
34
+ button {
35
+ margin-top: 10px;
36
+ padding: 8px 15px;
37
+ }
38
+
39
+ /* Input-token slider */
40
+ #ninWrapper {
41
+ width: 100%;
42
+ margin-top: 25px;
43
+ }
44
+
45
+ #ninSlider {
46
+ width: 100%;
47
+ display: block;
48
+ margin-top: 8px;
49
+ }
50
+
51
+ #ninLabels {
52
+ display: flex;
53
+ justify-content: space-between;
54
+ font-size: 12px;
55
+ color: #555;
56
+ margin-top: 3px;
57
+ }
58
+ </style>
59
+ ```
60
+
61
  </head>
62
 
63
  <body>
 
67
  </h2>
68
 
69
  <p>
70
+ This interactive demo visualizes our analytical model for predicting the
71
+ <b>energy consumption per generated token</b> as a function of the number of
72
+ output tokens (<i>n</i><sub>out</sub>).
73
+ Select one of the validated models used in the paper or load the architecture
74
+ of any <b>dense decoder-only Transformer</b> (up to approximately 20B
75
+ parameters) from Hugging Face to explore how the different terms contribute
76
+ to the prediction.
77
  </p>
78
 
79
  <p>
80
+ The model was fitted using empirical measurements collected on an
81
+ <b>NVIDIA H100 GPU</b> with <b>NVIDIA TensorRT-LLM</b> under the following
82
+ experimental configuration:
83
+ </p>
84
+
85
  <ul>
86
  <li><code>max_batch_size = 1024</code></li>
87
  <li><code>max_num_tokens = 8192</code></li>
88
  <li><code>num_requests = 1000</code></li>
89
  </ul>
90
 
91
+ <p>
92
+ Future releases will extend the analysis to additional hardware platforms
93
+ and inference frameworks.
94
  </p>
95
 
96
  <p>
97
+ For the complete methodology, experimental validation, and citation
98
+ information, please see the paper:
99
  <a href="YOUR_PAPER_URL" target="_blank">
100
+ SweetSpot v2: An Architecture-Aware Analytical Model for Predicting Energy
101
+ Efficiency of LLM Inference
102
  </a>.
103
  </p>
104
 
105
+ <h4>Validated models:</h4>
106
 
107
+ <select id="preset"></select>
 
 
 
 
 
108
 
109
  <button onclick="loadPreset()">
110
+ Load selected model
111
  </button>
112
 
113
  <h4>
114
+ Or load another public Hugging Face model (dense, &lt; 20B):
115
  </h4>
116
 
117
+ <input
118
  id="repo"
119
  style="width:400px"
120
+ placeholder="tiiuae/falcon-rw-7b"
121
+
122
+ >
123
 
124
  <button onclick="loadModel()">
125
+ Load model
126
  </button>
127
 
128
  <p id="status"></p>
 
131
 
132
  <div class="row">
133
 
134
+ ```
135
  <div>
136
+ <label>Hidden size (d):</label>
137
+ <input id="d" type="number" value="4096">
138
  </div>
139
 
140
  <div>
141
+ <label>Layers (L):</label>
142
+ <input id="L" type="number" value="36">
143
  </div>
144
 
145
  <div>
146
+ <label>Query heads (<i>n</i><sub>q</sub>):</label>
147
+ <input id="nq" type="number" value="64">
148
  </div>
149
 
150
  <div>
151
+ <label>KV heads (<i>n</i><sub>kv</sub>):</label>
152
+ <input id="nkv" type="number" value="64">
153
+ </div>
154
+ ```
155
+
156
  </div>
157
 
158
+ <!-- Slider is OUTSIDE the architecture-parameter row -->
159
+
160
  <div id="ninWrapper">
161
 
162
+ ```
163
  <label>
164
  Input Tokens (<i>n</i><sub>in</sub>):
165
  <span id="ninValue">64</span>
 
180
  <span>4096</span>
181
  </div>
182
 
183
+ <!-- Actual token count used by app.js -->
184
  <input id="nin" type="hidden" value="64">
185
+ ```
186
 
187
  </div>
188
 
189
+ <div id="plot"></div>
 
 
 
 
 
 
 
 
 
 
 
 
190
 
191
  <script>
192
  const MAX_TOKENS = 4096;
 
202
  }
203
  </script>
204
 
 
 
 
 
205
  <script src="app.js"></script>
206
 
207
  </body>
208
+ </html>