Hugging Face's logo

AlexKitipov
/
phi-2

Text Generation
Safetensors
English
phi
nlp
code
Model card Files
xet
 Community
AlexKitipov commited on
Commit
7b97654
·
verified ·
1 Parent(s): c6181f6

Upload phi_2 (1).ipynb

Browse files
Files changed (1) hide show
  1. phi_2 (1).ipynb +1530 -0
phi_2 (1).ipynb ADDED
@@ -0,0 +1,1530 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "nbformat": 4,
3
+ "nbformat_minor": 0,
4
+ "metadata": {
5
+ "colab": {
6
+ "machine_shape": "hm",
7
+ "gpuType": "T4",
8
+ "provenance": []
9
+ },
10
+ "accelerator": "GPU",
11
+ "kaggle": {
12
+ "accelerator": "gpu"
13
+ },
14
+ "language_info": {
15
+ "name": "python"
16
+ },
17
+ "kernelspec": {
18
+ "name": "python3",
19
+ "display_name": "Python 3"
20
+ },
21
+ "widgets": {
22
+ "application/vnd.jupyter.widget-state+json": {
23
+ "53f437ce9ce144bcaa2dcf90027099b5": {
24
+ "model_module": "@jupyter-widgets/controls",
25
+ "model_name": "HBoxModel",
26
+ "model_module_version": "1.5.0",
27
+ "state": {
28
+ "_dom_classes": [],
29
+ "_model_module": "@jupyter-widgets/controls",
30
+ "_model_module_version": "1.5.0",
31
+ "_model_name": "HBoxModel",
32
+ "_view_count": null,
33
+ "_view_module": "@jupyter-widgets/controls",
34
+ "_view_module_version": "1.5.0",
35
+ "_view_name": "HBoxView",
36
+ "box_style": "",
37
+ "children": [
38
+ "IPY_MODEL_d3e4d83800f74ba2b5f1841d50ff03f9",
39
+ "IPY_MODEL_d776ff80fc404113a7072668f9cbfe65",
40
+ "IPY_MODEL_a73630b02c614c8f81c1f06155276df0"
41
+ ],
42
+ "layout": "IPY_MODEL_edfe23b707274d2e927177e3a7b062cf"
43
+ }
44
+ },
45
+ "d3e4d83800f74ba2b5f1841d50ff03f9": {
46
+ "model_module": "@jupyter-widgets/controls",
47
+ "model_name": "HTMLModel",
48
+ "model_module_version": "1.5.0",
49
+ "state": {
50
+ "_dom_classes": [],
51
+ "_model_module": "@jupyter-widgets/controls",
52
+ "_model_module_version": "1.5.0",
53
+ "_model_name": "HTMLModel",
54
+ "_view_count": null,
55
+ "_view_module": "@jupyter-widgets/controls",
56
+ "_view_module_version": "1.5.0",
57
+ "_view_name": "HTMLView",
58
+ "description": "",
59
+ "description_tooltip": null,
60
+ "layout": "IPY_MODEL_d27a2cd0a69d48b3a95bcf1bf5ab32c0",
61
+ "placeholder": "​",
62
+ "style": "IPY_MODEL_8bef3d5dd0704fca92d05cfdd5a207f9",
63
+ "value": "Loading weights: 100%"
64
+ }
65
+ },
66
+ "d776ff80fc404113a7072668f9cbfe65": {
67
+ "model_module": "@jupyter-widgets/controls",
68
+ "model_name": "FloatProgressModel",
69
+ "model_module_version": "1.5.0",
70
+ "state": {
71
+ "_dom_classes": [],
72
+ "_model_module": "@jupyter-widgets/controls",
73
+ "_model_module_version": "1.5.0",
74
+ "_model_name": "FloatProgressModel",
75
+ "_view_count": null,
76
+ "_view_module": "@jupyter-widgets/controls",
77
+ "_view_module_version": "1.5.0",
78
+ "_view_name": "ProgressView",
79
+ "bar_style": "success",
80
+ "description": "",
81
+ "description_tooltip": null,
82
+ "layout": "IPY_MODEL_61d00629597f4e92aaef0f08ce523390",
83
+ "max": 453,
84
+ "min": 0,
85
+ "orientation": "horizontal",
86
+ "style": "IPY_MODEL_99b2738ceb0b437a87af4a845ef29434",
87
+ "value": 453
88
+ }
89
+ },
90
+ "a73630b02c614c8f81c1f06155276df0": {
91
+ "model_module": "@jupyter-widgets/controls",
92
+ "model_name": "HTMLModel",
93
+ "model_module_version": "1.5.0",
94
+ "state": {
95
+ "_dom_classes": [],
96
+ "_model_module": "@jupyter-widgets/controls",
97
+ "_model_module_version": "1.5.0",
98
+ "_model_name": "HTMLModel",
99
+ "_view_count": null,
100
+ "_view_module": "@jupyter-widgets/controls",
101
+ "_view_module_version": "1.5.0",
102
+ "_view_name": "HTMLView",
103
+ "description": "",
104
+ "description_tooltip": null,
105
+ "layout": "IPY_MODEL_92e1572d00244e42819122ac3a9d3521",
106
+ "placeholder": "​",
107
+ "style": "IPY_MODEL_7f7005e525f74c7389847139207d1f10",
108
+ "value": " 453/453 [00:27<00:00, 23.75it/s]"
109
+ }
110
+ },
111
+ "edfe23b707274d2e927177e3a7b062cf": {
112
+ "model_module": "@jupyter-widgets/base",
113
+ "model_name": "LayoutModel",
114
+ "model_module_version": "1.2.0",
115
+ "state": {
116
+ "_model_module": "@jupyter-widgets/base",
117
+ "_model_module_version": "1.2.0",
118
+ "_model_name": "LayoutModel",
119
+ "_view_count": null,
120
+ "_view_module": "@jupyter-widgets/base",
121
+ "_view_module_version": "1.2.0",
122
+ "_view_name": "LayoutView",
123
+ "align_content": null,
124
+ "align_items": null,
125
+ "align_self": null,
126
+ "border": null,
127
+ "bottom": null,
128
+ "display": null,
129
+ "flex": null,
130
+ "flex_flow": null,
131
+ "grid_area": null,
132
+ "grid_auto_columns": null,
133
+ "grid_auto_flow": null,
134
+ "grid_auto_rows": null,
135
+ "grid_column": null,
136
+ "grid_gap": null,
137
+ "grid_row": null,
138
+ "grid_template_areas": null,
139
+ "grid_template_columns": null,
140
+ "grid_template_rows": null,
141
+ "height": null,
142
+ "justify_content": null,
143
+ "justify_items": null,
144
+ "left": null,
145
+ "margin": null,
146
+ "max_height": null,
147
+ "max_width": null,
148
+ "min_height": null,
149
+ "min_width": null,
150
+ "object_fit": null,
151
+ "object_position": null,
152
+ "order": null,
153
+ "overflow": null,
154
+ "overflow_x": null,
155
+ "overflow_y": null,
156
+ "padding": null,
157
+ "right": null,
158
+ "top": null,
159
+ "visibility": null,
160
+ "width": null
161
+ }
162
+ },
163
+ "d27a2cd0a69d48b3a95bcf1bf5ab32c0": {
164
+ "model_module": "@jupyter-widgets/base",
165
+ "model_name": "LayoutModel",
166
+ "model_module_version": "1.2.0",
167
+ "state": {
168
+ "_model_module": "@jupyter-widgets/base",
169
+ "_model_module_version": "1.2.0",
170
+ "_model_name": "LayoutModel",
171
+ "_view_count": null,
172
+ "_view_module": "@jupyter-widgets/base",
173
+ "_view_module_version": "1.2.0",
174
+ "_view_name": "LayoutView",
175
+ "align_content": null,
176
+ "align_items": null,
177
+ "align_self": null,
178
+ "border": null,
179
+ "bottom": null,
180
+ "display": null,
181
+ "flex": null,
182
+ "flex_flow": null,
183
+ "grid_area": null,
184
+ "grid_auto_columns": null,
185
+ "grid_auto_flow": null,
186
+ "grid_auto_rows": null,
187
+ "grid_column": null,
188
+ "grid_gap": null,
189
+ "grid_row": null,
190
+ "grid_template_areas": null,
191
+ "grid_template_columns": null,
192
+ "grid_template_rows": null,
193
+ "height": null,
194
+ "justify_content": null,
195
+ "justify_items": null,
196
+ "left": null,
197
+ "margin": null,
198
+ "max_height": null,
199
+ "max_width": null,
200
+ "min_height": null,
201
+ "min_width": null,
202
+ "object_fit": null,
203
+ "object_position": null,
204
+ "order": null,
205
+ "overflow": null,
206
+ "overflow_x": null,
207
+ "overflow_y": null,
208
+ "padding": null,
209
+ "right": null,
210
+ "top": null,
211
+ "visibility": null,
212
+ "width": null
213
+ }
214
+ },
215
+ "8bef3d5dd0704fca92d05cfdd5a207f9": {
216
+ "model_module": "@jupyter-widgets/controls",
217
+ "model_name": "DescriptionStyleModel",
218
+ "model_module_version": "1.5.0",
219
+ "state": {
220
+ "_model_module": "@jupyter-widgets/controls",
221
+ "_model_module_version": "1.5.0",
222
+ "_model_name": "DescriptionStyleModel",
223
+ "_view_count": null,
224
+ "_view_module": "@jupyter-widgets/base",
225
+ "_view_module_version": "1.2.0",
226
+ "_view_name": "StyleView",
227
+ "description_width": ""
228
+ }
229
+ },
230
+ "61d00629597f4e92aaef0f08ce523390": {
231
+ "model_module": "@jupyter-widgets/base",
232
+ "model_name": "LayoutModel",
233
+ "model_module_version": "1.2.0",
234
+ "state": {
235
+ "_model_module": "@jupyter-widgets/base",
236
+ "_model_module_version": "1.2.0",
237
+ "_model_name": "LayoutModel",
238
+ "_view_count": null,
239
+ "_view_module": "@jupyter-widgets/base",
240
+ "_view_module_version": "1.2.0",
241
+ "_view_name": "LayoutView",
242
+ "align_content": null,
243
+ "align_items": null,
244
+ "align_self": null,
245
+ "border": null,
246
+ "bottom": null,
247
+ "display": null,
248
+ "flex": null,
249
+ "flex_flow": null,
250
+ "grid_area": null,
251
+ "grid_auto_columns": null,
252
+ "grid_auto_flow": null,
253
+ "grid_auto_rows": null,
254
+ "grid_column": null,
255
+ "grid_gap": null,
256
+ "grid_row": null,
257
+ "grid_template_areas": null,
258
+ "grid_template_columns": null,
259
+ "grid_template_rows": null,
260
+ "height": null,
261
+ "justify_content": null,
262
+ "justify_items": null,
263
+ "left": null,
264
+ "margin": null,
265
+ "max_height": null,
266
+ "max_width": null,
267
+ "min_height": null,
268
+ "min_width": null,
269
+ "object_fit": null,
270
+ "object_position": null,
271
+ "order": null,
272
+ "overflow": null,
273
+ "overflow_x": null,
274
+ "overflow_y": null,
275
+ "padding": null,
276
+ "right": null,
277
+ "top": null,
278
+ "visibility": null,
279
+ "width": null
280
+ }
281
+ },
282
+ "99b2738ceb0b437a87af4a845ef29434": {
283
+ "model_module": "@jupyter-widgets/controls",
284
+ "model_name": "ProgressStyleModel",
285
+ "model_module_version": "1.5.0",
286
+ "state": {
287
+ "_model_module": "@jupyter-widgets/controls",
288
+ "_model_module_version": "1.5.0",
289
+ "_model_name": "ProgressStyleModel",
290
+ "_view_count": null,
291
+ "_view_module": "@jupyter-widgets/base",
292
+ "_view_module_version": "1.2.0",
293
+ "_view_name": "StyleView",
294
+ "bar_color": null,
295
+ "description_width": ""
296
+ }
297
+ },
298
+ "92e1572d00244e42819122ac3a9d3521": {
299
+ "model_module": "@jupyter-widgets/base",
300
+ "model_name": "LayoutModel",
301
+ "model_module_version": "1.2.0",
302
+ "state": {
303
+ "_model_module": "@jupyter-widgets/base",
304
+ "_model_module_version": "1.2.0",
305
+ "_model_name": "LayoutModel",
306
+ "_view_count": null,
307
+ "_view_module": "@jupyter-widgets/base",
308
+ "_view_module_version": "1.2.0",
309
+ "_view_name": "LayoutView",
310
+ "align_content": null,
311
+ "align_items": null,
312
+ "align_self": null,
313
+ "border": null,
314
+ "bottom": null,
315
+ "display": null,
316
+ "flex": null,
317
+ "flex_flow": null,
318
+ "grid_area": null,
319
+ "grid_auto_columns": null,
320
+ "grid_auto_flow": null,
321
+ "grid_auto_rows": null,
322
+ "grid_column": null,
323
+ "grid_gap": null,
324
+ "grid_row": null,
325
+ "grid_template_areas": null,
326
+ "grid_template_columns": null,
327
+ "grid_template_rows": null,
328
+ "height": null,
329
+ "justify_content": null,
330
+ "justify_items": null,
331
+ "left": null,
332
+ "margin": null,
333
+ "max_height": null,
334
+ "max_width": null,
335
+ "min_height": null,
336
+ "min_width": null,
337
+ "object_fit": null,
338
+ "object_position": null,
339
+ "order": null,
340
+ "overflow": null,
341
+ "overflow_x": null,
342
+ "overflow_y": null,
343
+ "padding": null,
344
+ "right": null,
345
+ "top": null,
346
+ "visibility": null,
347
+ "width": null
348
+ }
349
+ },
350
+ "7f7005e525f74c7389847139207d1f10": {
351
+ "model_module": "@jupyter-widgets/controls",
352
+ "model_name": "DescriptionStyleModel",
353
+ "model_module_version": "1.5.0",
354
+ "state": {
355
+ "_model_module": "@jupyter-widgets/controls",
356
+ "_model_module_version": "1.5.0",
357
+ "_model_name": "DescriptionStyleModel",
358
+ "_view_count": null,
359
+ "_view_module": "@jupyter-widgets/base",
360
+ "_view_module_version": "1.2.0",
361
+ "_view_name": "StyleView",
362
+ "description_width": ""
363
+ }
364
+ },
365
+ "36a690fc6d4d45289ca24029ece56dd2": {
366
+ "model_module": "@jupyter-widgets/controls",
367
+ "model_name": "HBoxModel",
368
+ "model_module_version": "1.5.0",
369
+ "state": {
370
+ "_dom_classes": [],
371
+ "_model_module": "@jupyter-widgets/controls",
372
+ "_model_module_version": "1.5.0",
373
+ "_model_name": "HBoxModel",
374
+ "_view_count": null,
375
+ "_view_module": "@jupyter-widgets/controls",
376
+ "_view_module_version": "1.5.0",
377
+ "_view_name": "HBoxView",
378
+ "box_style": "",
379
+ "children": [
380
+ "IPY_MODEL_a9e2f204052645f6ac20b99e3348e07b",
381
+ "IPY_MODEL_72f98858f4f74e50a102b3307c8a1a3b",
382
+ "IPY_MODEL_c1dbf5a94d9546cfb0639702dad01ab8"
383
+ ],
384
+ "layout": "IPY_MODEL_15acdb37b2c444fea8f0315e2ca7da0b"
385
+ }
386
+ },
387
+ "a9e2f204052645f6ac20b99e3348e07b": {
388
+ "model_module": "@jupyter-widgets/controls",
389
+ "model_name": "HTMLModel",
390
+ "model_module_version": "1.5.0",
391
+ "state": {
392
+ "_dom_classes": [],
393
+ "_model_module": "@jupyter-widgets/controls",
394
+ "_model_module_version": "1.5.0",
395
+ "_model_name": "HTMLModel",
396
+ "_view_count": null,
397
+ "_view_module": "@jupyter-widgets/controls",
398
+ "_view_module_version": "1.5.0",
399
+ "_view_name": "HTMLView",
400
+ "description": "",
401
+ "description_tooltip": null,
402
+ "layout": "IPY_MODEL_dd688120d7a449699de136c48e084b69",
403
+ "placeholder": "​",
404
+ "style": "IPY_MODEL_e97172be05b84455b57f921401116412",
405
+ "value": "Loading weights: 100%"
406
+ }
407
+ },
408
+ "72f98858f4f74e50a102b3307c8a1a3b": {
409
+ "model_module": "@jupyter-widgets/controls",
410
+ "model_name": "FloatProgressModel",
411
+ "model_module_version": "1.5.0",
412
+ "state": {
413
+ "_dom_classes": [],
414
+ "_model_module": "@jupyter-widgets/controls",
415
+ "_model_module_version": "1.5.0",
416
+ "_model_name": "FloatProgressModel",
417
+ "_view_count": null,
418
+ "_view_module": "@jupyter-widgets/controls",
419
+ "_view_module_version": "1.5.0",
420
+ "_view_name": "ProgressView",
421
+ "bar_style": "success",
422
+ "description": "",
423
+ "description_tooltip": null,
424
+ "layout": "IPY_MODEL_627dca520f104b3e86ac63fe006b400a",
425
+ "max": 453,
426
+ "min": 0,
427
+ "orientation": "horizontal",
428
+ "style": "IPY_MODEL_53edd077eb0b46eea5cfbff69ff199bc",
429
+ "value": 453
430
+ }
431
+ },
432
+ "c1dbf5a94d9546cfb0639702dad01ab8": {
433
+ "model_module": "@jupyter-widgets/controls",
434
+ "model_name": "HTMLModel",
435
+ "model_module_version": "1.5.0",
436
+ "state": {
437
+ "_dom_classes": [],
438
+ "_model_module": "@jupyter-widgets/controls",
439
+ "_model_module_version": "1.5.0",
440
+ "_model_name": "HTMLModel",
441
+ "_view_count": null,
442
+ "_view_module": "@jupyter-widgets/controls",
443
+ "_view_module_version": "1.5.0",
444
+ "_view_name": "HTMLView",
445
+ "description": "",
446
+ "description_tooltip": null,
447
+ "layout": "IPY_MODEL_d8c681bdf67b48899bd1cc483b03a4e9",
448
+ "placeholder": "​",
449
+ "style": "IPY_MODEL_21243c6ca4754573b3cc5b900a8efa0c",
450
+ "value": " 453/453 [00:42<00:00, 14.30it/s]"
451
+ }
452
+ },
453
+ "15acdb37b2c444fea8f0315e2ca7da0b": {
454
+ "model_module": "@jupyter-widgets/base",
455
+ "model_name": "LayoutModel",
456
+ "model_module_version": "1.2.0",
457
+ "state": {
458
+ "_model_module": "@jupyter-widgets/base",
459
+ "_model_module_version": "1.2.0",
460
+ "_model_name": "LayoutModel",
461
+ "_view_count": null,
462
+ "_view_module": "@jupyter-widgets/base",
463
+ "_view_module_version": "1.2.0",
464
+ "_view_name": "LayoutView",
465
+ "align_content": null,
466
+ "align_items": null,
467
+ "align_self": null,
468
+ "border": null,
469
+ "bottom": null,
470
+ "display": null,
471
+ "flex": null,
472
+ "flex_flow": null,
473
+ "grid_area": null,
474
+ "grid_auto_columns": null,
475
+ "grid_auto_flow": null,
476
+ "grid_auto_rows": null,
477
+ "grid_column": null,
478
+ "grid_gap": null,
479
+ "grid_row": null,
480
+ "grid_template_areas": null,
481
+ "grid_template_columns": null,
482
+ "grid_template_rows": null,
483
+ "height": null,
484
+ "justify_content": null,
485
+ "justify_items": null,
486
+ "left": null,
487
+ "margin": null,
488
+ "max_height": null,
489
+ "max_width": null,
490
+ "min_height": null,
491
+ "min_width": null,
492
+ "object_fit": null,
493
+ "object_position": null,
494
+ "order": null,
495
+ "overflow": null,
496
+ "overflow_x": null,
497
+ "overflow_y": null,
498
+ "padding": null,
499
+ "right": null,
500
+ "top": null,
501
+ "visibility": null,
502
+ "width": null
503
+ }
504
+ },
505
+ "dd688120d7a449699de136c48e084b69": {
506
+ "model_module": "@jupyter-widgets/base",
507
+ "model_name": "LayoutModel",
508
+ "model_module_version": "1.2.0",
509
+ "state": {
510
+ "_model_module": "@jupyter-widgets/base",
511
+ "_model_module_version": "1.2.0",
512
+ "_model_name": "LayoutModel",
513
+ "_view_count": null,
514
+ "_view_module": "@jupyter-widgets/base",
515
+ "_view_module_version": "1.2.0",
516
+ "_view_name": "LayoutView",
517
+ "align_content": null,
518
+ "align_items": null,
519
+ "align_self": null,
520
+ "border": null,
521
+ "bottom": null,
522
+ "display": null,
523
+ "flex": null,
524
+ "flex_flow": null,
525
+ "grid_area": null,
526
+ "grid_auto_columns": null,
527
+ "grid_auto_flow": null,
528
+ "grid_auto_rows": null,
529
+ "grid_column": null,
530
+ "grid_gap": null,
531
+ "grid_row": null,
532
+ "grid_template_areas": null,
533
+ "grid_template_columns": null,
534
+ "grid_template_rows": null,
535
+ "height": null,
536
+ "justify_content": null,
537
+ "justify_items": null,
538
+ "left": null,
539
+ "margin": null,
540
+ "max_height": null,
541
+ "max_width": null,
542
+ "min_height": null,
543
+ "min_width": null,
544
+ "object_fit": null,
545
+ "object_position": null,
546
+ "order": null,
547
+ "overflow": null,
548
+ "overflow_x": null,
549
+ "overflow_y": null,
550
+ "padding": null,
551
+ "right": null,
552
+ "top": null,
553
+ "visibility": null,
554
+ "width": null
555
+ }
556
+ },
557
+ "e97172be05b84455b57f921401116412": {
558
+ "model_module": "@jupyter-widgets/controls",
559
+ "model_name": "DescriptionStyleModel",
560
+ "model_module_version": "1.5.0",
561
+ "state": {
562
+ "_model_module": "@jupyter-widgets/controls",
563
+ "_model_module_version": "1.5.0",
564
+ "_model_name": "DescriptionStyleModel",
565
+ "_view_count": null,
566
+ "_view_module": "@jupyter-widgets/base",
567
+ "_view_module_version": "1.2.0",
568
+ "_view_name": "StyleView",
569
+ "description_width": ""
570
+ }
571
+ },
572
+ "627dca520f104b3e86ac63fe006b400a": {
573
+ "model_module": "@jupyter-widgets/base",
574
+ "model_name": "LayoutModel",
575
+ "model_module_version": "1.2.0",
576
+ "state": {
577
+ "_model_module": "@jupyter-widgets/base",
578
+ "_model_module_version": "1.2.0",
579
+ "_model_name": "LayoutModel",
580
+ "_view_count": null,
581
+ "_view_module": "@jupyter-widgets/base",
582
+ "_view_module_version": "1.2.0",
583
+ "_view_name": "LayoutView",
584
+ "align_content": null,
585
+ "align_items": null,
586
+ "align_self": null,
587
+ "border": null,
588
+ "bottom": null,
589
+ "display": null,
590
+ "flex": null,
591
+ "flex_flow": null,
592
+ "grid_area": null,
593
+ "grid_auto_columns": null,
594
+ "grid_auto_flow": null,
595
+ "grid_auto_rows": null,
596
+ "grid_column": null,
597
+ "grid_gap": null,
598
+ "grid_row": null,
599
+ "grid_template_areas": null,
600
+ "grid_template_columns": null,
601
+ "grid_template_rows": null,
602
+ "height": null,
603
+ "justify_content": null,
604
+ "justify_items": null,
605
+ "left": null,
606
+ "margin": null,
607
+ "max_height": null,
608
+ "max_width": null,
609
+ "min_height": null,
610
+ "min_width": null,
611
+ "object_fit": null,
612
+ "object_position": null,
613
+ "order": null,
614
+ "overflow": null,
615
+ "overflow_x": null,
616
+ "overflow_y": null,
617
+ "padding": null,
618
+ "right": null,
619
+ "top": null,
620
+ "visibility": null,
621
+ "width": null
622
+ }
623
+ },
624
+ "53edd077eb0b46eea5cfbff69ff199bc": {
625
+ "model_module": "@jupyter-widgets/controls",
626
+ "model_name": "ProgressStyleModel",
627
+ "model_module_version": "1.5.0",
628
+ "state": {
629
+ "_model_module": "@jupyter-widgets/controls",
630
+ "_model_module_version": "1.5.0",
631
+ "_model_name": "ProgressStyleModel",
632
+ "_view_count": null,
633
+ "_view_module": "@jupyter-widgets/base",
634
+ "_view_module_version": "1.2.0",
635
+ "_view_name": "StyleView",
636
+ "bar_color": null,
637
+ "description_width": ""
638
+ }
639
+ },
640
+ "d8c681bdf67b48899bd1cc483b03a4e9": {
641
+ "model_module": "@jupyter-widgets/base",
642
+ "model_name": "LayoutModel",
643
+ "model_module_version": "1.2.0",
644
+ "state": {
645
+ "_model_module": "@jupyter-widgets/base",
646
+ "_model_module_version": "1.2.0",
647
+ "_model_name": "LayoutModel",
648
+ "_view_count": null,
649
+ "_view_module": "@jupyter-widgets/base",
650
+ "_view_module_version": "1.2.0",
651
+ "_view_name": "LayoutView",
652
+ "align_content": null,
653
+ "align_items": null,
654
+ "align_self": null,
655
+ "border": null,
656
+ "bottom": null,
657
+ "display": null,
658
+ "flex": null,
659
+ "flex_flow": null,
660
+ "grid_area": null,
661
+ "grid_auto_columns": null,
662
+ "grid_auto_flow": null,
663
+ "grid_auto_rows": null,
664
+ "grid_column": null,
665
+ "grid_gap": null,
666
+ "grid_row": null,
667
+ "grid_template_areas": null,
668
+ "grid_template_columns": null,
669
+ "grid_template_rows": null,
670
+ "height": null,
671
+ "justify_content": null,
672
+ "justify_items": null,
673
+ "left": null,
674
+ "margin": null,
675
+ "max_height": null,
676
+ "max_width": null,
677
+ "min_height": null,
678
+ "min_width": null,
679
+ "object_fit": null,
680
+ "object_position": null,
681
+ "order": null,
682
+ "overflow": null,
683
+ "overflow_x": null,
684
+ "overflow_y": null,
685
+ "padding": null,
686
+ "right": null,
687
+ "top": null,
688
+ "visibility": null,
689
+ "width": null
690
+ }
691
+ },
692
+ "21243c6ca4754573b3cc5b900a8efa0c": {
693
+ "model_module": "@jupyter-widgets/controls",
694
+ "model_name": "DescriptionStyleModel",
695
+ "model_module_version": "1.5.0",
696
+ "state": {
697
+ "_model_module": "@jupyter-widgets/controls",
698
+ "_model_module_version": "1.5.0",
699
+ "_model_name": "DescriptionStyleModel",
700
+ "_view_count": null,
701
+ "_view_module": "@jupyter-widgets/base",
702
+ "_view_module_version": "1.2.0",
703
+ "_view_name": "StyleView",
704
+ "description_width": ""
705
+ }
706
+ },
707
+ "5f74f3acaa9840dfb91de8336143e488": {
708
+ "model_module": "@jupyter-widgets/controls",
709
+ "model_name": "HBoxModel",
710
+ "model_module_version": "1.5.0",
711
+ "state": {
712
+ "_dom_classes": [],
713
+ "_model_module": "@jupyter-widgets/controls",
714
+ "_model_module_version": "1.5.0",
715
+ "_model_name": "HBoxModel",
716
+ "_view_count": null,
717
+ "_view_module": "@jupyter-widgets/controls",
718
+ "_view_module_version": "1.5.0",
719
+ "_view_name": "HBoxView",
720
+ "box_style": "",
721
+ "children": [
722
+ "IPY_MODEL_ecf38f30b070475b881d62336e48dba5",
723
+ "IPY_MODEL_bc6a478c3b76438b8e92b71fb9b26b3b",
724
+ "IPY_MODEL_e0dc86e3345141fd860be18318472eb1"
725
+ ],
726
+ "layout": "IPY_MODEL_fa9902cd507b442c9e98f9ee64d599e6"
727
+ }
728
+ },
729
+ "ecf38f30b070475b881d62336e48dba5": {
730
+ "model_module": "@jupyter-widgets/controls",
731
+ "model_name": "HTMLModel",
732
+ "model_module_version": "1.5.0",
733
+ "state": {
734
+ "_dom_classes": [],
735
+ "_model_module": "@jupyter-widgets/controls",
736
+ "_model_module_version": "1.5.0",
737
+ "_model_name": "HTMLModel",
738
+ "_view_count": null,
739
+ "_view_module": "@jupyter-widgets/controls",
740
+ "_view_module_version": "1.5.0",
741
+ "_view_name": "HTMLView",
742
+ "description": "",
743
+ "description_tooltip": null,
744
+ "layout": "IPY_MODEL_2825815747b04327b5a16adc431f9bd5",
745
+ "placeholder": "​",
746
+ "style": "IPY_MODEL_5a66230e1d6b437a925f1b9cf9d011a5",
747
+ "value": "Loading weights:  13%"
748
+ }
749
+ },
750
+ "bc6a478c3b76438b8e92b71fb9b26b3b": {
751
+ "model_module": "@jupyter-widgets/controls",
752
+ "model_name": "FloatProgressModel",
753
+ "model_module_version": "1.5.0",
754
+ "state": {
755
+ "_dom_classes": [],
756
+ "_model_module": "@jupyter-widgets/controls",
757
+ "_model_module_version": "1.5.0",
758
+ "_model_name": "FloatProgressModel",
759
+ "_view_count": null,
760
+ "_view_module": "@jupyter-widgets/controls",
761
+ "_view_module_version": "1.5.0",
762
+ "_view_name": "ProgressView",
763
+ "bar_style": "",
764
+ "description": "",
765
+ "description_tooltip": null,
766
+ "layout": "IPY_MODEL_d5de5f95ec3142e696571ff7fd588ed9",
767
+ "max": 453,
768
+ "min": 0,
769
+ "orientation": "horizontal",
770
+ "style": "IPY_MODEL_cd9939d1217644b482d05814d23d788a",
771
+ "value": 61
772
+ }
773
+ },
774
+ "e0dc86e3345141fd860be18318472eb1": {
775
+ "model_module": "@jupyter-widgets/controls",
776
+ "model_name": "HTMLModel",
777
+ "model_module_version": "1.5.0",
778
+ "state": {
779
+ "_dom_classes": [],
780
+ "_model_module": "@jupyter-widgets/controls",
781
+ "_model_module_version": "1.5.0",
782
+ "_model_name": "HTMLModel",
783
+ "_view_count": null,
784
+ "_view_module": "@jupyter-widgets/controls",
785
+ "_view_module_version": "1.5.0",
786
+ "_view_name": "HTMLView",
787
+ "description": "",
788
+ "description_tooltip": null,
789
+ "layout": "IPY_MODEL_b10968146d5f4c6a864412e71fe2d2f9",
790
+ "placeholder": "​",
791
+ "style": "IPY_MODEL_a2be91b533d344c1bd3fa5deddaa6be5",
792
+ "value": " 61/453 [00:05<00:29, 13.39it/s]"
793
+ }
794
+ },
795
+ "fa9902cd507b442c9e98f9ee64d599e6": {
796
+ "model_module": "@jupyter-widgets/base",
797
+ "model_name": "LayoutModel",
798
+ "model_module_version": "1.2.0",
799
+ "state": {
800
+ "_model_module": "@jupyter-widgets/base",
801
+ "_model_module_version": "1.2.0",
802
+ "_model_name": "LayoutModel",
803
+ "_view_count": null,
804
+ "_view_module": "@jupyter-widgets/base",
805
+ "_view_module_version": "1.2.0",
806
+ "_view_name": "LayoutView",
807
+ "align_content": null,
808
+ "align_items": null,
809
+ "align_self": null,
810
+ "border": null,
811
+ "bottom": null,
812
+ "display": null,
813
+ "flex": null,
814
+ "flex_flow": null,
815
+ "grid_area": null,
816
+ "grid_auto_columns": null,
817
+ "grid_auto_flow": null,
818
+ "grid_auto_rows": null,
819
+ "grid_column": null,
820
+ "grid_gap": null,
821
+ "grid_row": null,
822
+ "grid_template_areas": null,
823
+ "grid_template_columns": null,
824
+ "grid_template_rows": null,
825
+ "height": null,
826
+ "justify_content": null,
827
+ "justify_items": null,
828
+ "left": null,
829
+ "margin": null,
830
+ "max_height": null,
831
+ "max_width": null,
832
+ "min_height": null,
833
+ "min_width": null,
834
+ "object_fit": null,
835
+ "object_position": null,
836
+ "order": null,
837
+ "overflow": null,
838
+ "overflow_x": null,
839
+ "overflow_y": null,
840
+ "padding": null,
841
+ "right": null,
842
+ "top": null,
843
+ "visibility": null,
844
+ "width": null
845
+ }
846
+ },
847
+ "2825815747b04327b5a16adc431f9bd5": {
848
+ "model_module": "@jupyter-widgets/base",
849
+ "model_name": "LayoutModel",
850
+ "model_module_version": "1.2.0",
851
+ "state": {
852
+ "_model_module": "@jupyter-widgets/base",
853
+ "_model_module_version": "1.2.0",
854
+ "_model_name": "LayoutModel",
855
+ "_view_count": null,
856
+ "_view_module": "@jupyter-widgets/base",
857
+ "_view_module_version": "1.2.0",
858
+ "_view_name": "LayoutView",
859
+ "align_content": null,
860
+ "align_items": null,
861
+ "align_self": null,
862
+ "border": null,
863
+ "bottom": null,
864
+ "display": null,
865
+ "flex": null,
866
+ "flex_flow": null,
867
+ "grid_area": null,
868
+ "grid_auto_columns": null,
869
+ "grid_auto_flow": null,
870
+ "grid_auto_rows": null,
871
+ "grid_column": null,
872
+ "grid_gap": null,
873
+ "grid_row": null,
874
+ "grid_template_areas": null,
875
+ "grid_template_columns": null,
876
+ "grid_template_rows": null,
877
+ "height": null,
878
+ "justify_content": null,
879
+ "justify_items": null,
880
+ "left": null,
881
+ "margin": null,
882
+ "max_height": null,
883
+ "max_width": null,
884
+ "min_height": null,
885
+ "min_width": null,
886
+ "object_fit": null,
887
+ "object_position": null,
888
+ "order": null,
889
+ "overflow": null,
890
+ "overflow_x": null,
891
+ "overflow_y": null,
892
+ "padding": null,
893
+ "right": null,
894
+ "top": null,
895
+ "visibility": null,
896
+ "width": null
897
+ }
898
+ },
899
+ "5a66230e1d6b437a925f1b9cf9d011a5": {
900
+ "model_module": "@jupyter-widgets/controls",
901
+ "model_name": "DescriptionStyleModel",
902
+ "model_module_version": "1.5.0",
903
+ "state": {
904
+ "_model_module": "@jupyter-widgets/controls",
905
+ "_model_module_version": "1.5.0",
906
+ "_model_name": "DescriptionStyleModel",
907
+ "_view_count": null,
908
+ "_view_module": "@jupyter-widgets/base",
909
+ "_view_module_version": "1.2.0",
910
+ "_view_name": "StyleView",
911
+ "description_width": ""
912
+ }
913
+ },
914
+ "d5de5f95ec3142e696571ff7fd588ed9": {
915
+ "model_module": "@jupyter-widgets/base",
916
+ "model_name": "LayoutModel",
917
+ "model_module_version": "1.2.0",
918
+ "state": {
919
+ "_model_module": "@jupyter-widgets/base",
920
+ "_model_module_version": "1.2.0",
921
+ "_model_name": "LayoutModel",
922
+ "_view_count": null,
923
+ "_view_module": "@jupyter-widgets/base",
924
+ "_view_module_version": "1.2.0",
925
+ "_view_name": "LayoutView",
926
+ "align_content": null,
927
+ "align_items": null,
928
+ "align_self": null,
929
+ "border": null,
930
+ "bottom": null,
931
+ "display": null,
932
+ "flex": null,
933
+ "flex_flow": null,
934
+ "grid_area": null,
935
+ "grid_auto_columns": null,
936
+ "grid_auto_flow": null,
937
+ "grid_auto_rows": null,
938
+ "grid_column": null,
939
+ "grid_gap": null,
940
+ "grid_row": null,
941
+ "grid_template_areas": null,
942
+ "grid_template_columns": null,
943
+ "grid_template_rows": null,
944
+ "height": null,
945
+ "justify_content": null,
946
+ "justify_items": null,
947
+ "left": null,
948
+ "margin": null,
949
+ "max_height": null,
950
+ "max_width": null,
951
+ "min_height": null,
952
+ "min_width": null,
953
+ "object_fit": null,
954
+ "object_position": null,
955
+ "order": null,
956
+ "overflow": null,
957
+ "overflow_x": null,
958
+ "overflow_y": null,
959
+ "padding": null,
960
+ "right": null,
961
+ "top": null,
962
+ "visibility": null,
963
+ "width": null
964
+ }
965
+ },
966
+ "cd9939d1217644b482d05814d23d788a": {
967
+ "model_module": "@jupyter-widgets/controls",
968
+ "model_name": "ProgressStyleModel",
969
+ "model_module_version": "1.5.0",
970
+ "state": {
971
+ "_model_module": "@jupyter-widgets/controls",
972
+ "_model_module_version": "1.5.0",
973
+ "_model_name": "ProgressStyleModel",
974
+ "_view_count": null,
975
+ "_view_module": "@jupyter-widgets/base",
976
+ "_view_module_version": "1.2.0",
977
+ "_view_name": "StyleView",
978
+ "bar_color": null,
979
+ "description_width": ""
980
+ }
981
+ },
982
+ "b10968146d5f4c6a864412e71fe2d2f9": {
983
+ "model_module": "@jupyter-widgets/base",
984
+ "model_name": "LayoutModel",
985
+ "model_module_version": "1.2.0",
986
+ "state": {
987
+ "_model_module": "@jupyter-widgets/base",
988
+ "_model_module_version": "1.2.0",
989
+ "_model_name": "LayoutModel",
990
+ "_view_count": null,
991
+ "_view_module": "@jupyter-widgets/base",
992
+ "_view_module_version": "1.2.0",
993
+ "_view_name": "LayoutView",
994
+ "align_content": null,
995
+ "align_items": null,
996
+ "align_self": null,
997
+ "border": null,
998
+ "bottom": null,
999
+ "display": null,
1000
+ "flex": null,
1001
+ "flex_flow": null,
1002
+ "grid_area": null,
1003
+ "grid_auto_columns": null,
1004
+ "grid_auto_flow": null,
1005
+ "grid_auto_rows": null,
1006
+ "grid_column": null,
1007
+ "grid_gap": null,
1008
+ "grid_row": null,
1009
+ "grid_template_areas": null,
1010
+ "grid_template_columns": null,
1011
+ "grid_template_rows": null,
1012
+ "height": null,
1013
+ "justify_content": null,
1014
+ "justify_items": null,
1015
+ "left": null,
1016
+ "margin": null,
1017
+ "max_height": null,
1018
+ "max_width": null,
1019
+ "min_height": null,
1020
+ "min_width": null,
1021
+ "object_fit": null,
1022
+ "object_position": null,
1023
+ "order": null,
1024
+ "overflow": null,
1025
+ "overflow_x": null,
1026
+ "overflow_y": null,
1027
+ "padding": null,
1028
+ "right": null,
1029
+ "top": null,
1030
+ "visibility": null,
1031
+ "width": null
1032
+ }
1033
+ },
1034
+ "a2be91b533d344c1bd3fa5deddaa6be5": {
1035
+ "model_module": "@jupyter-widgets/controls",
1036
+ "model_name": "DescriptionStyleModel",
1037
+ "model_module_version": "1.5.0",
1038
+ "state": {
1039
+ "_model_module": "@jupyter-widgets/controls",
1040
+ "_model_module_version": "1.5.0",
1041
+ "_model_name": "DescriptionStyleModel",
1042
+ "_view_count": null,
1043
+ "_view_module": "@jupyter-widgets/base",
1044
+ "_view_module_version": "1.2.0",
1045
+ "_view_name": "StyleView",
1046
+ "description_width": ""
1047
+ }
1048
+ }
1049
+ }
1050
+ }
1051
+ },
1052
+ "cells": [
1053
+ {
1054
+ "cell_type": "code",
1055
+ "source": [
1056
+ "!pip install -U transformers bitsandbytes accelerate -q"
1057
+ ],
1058
+ "metadata": {
1059
+ "colab": {
1060
+ "base_uri": "https://localhost:8080/"
1061
+ },
1062
+ "id": "o0zcQycu-UEa",
1063
+ "outputId": "e9479e8a-b72e-4ec8-b09d-57397de63e24"
1064
+ },
1065
+ "execution_count": 18,
1066
+ "outputs": [
1067
+ {
1068
+ "output_type": "stream",
1069
+ "name": "stdout",
1070
+ "text": [
1071
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m60.7/60.7 MB\u001b[0m \u001b[31m16.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
1072
+ "\u001b[?25h"
1073
+ ]
1074
+ }
1075
+ ]
1076
+ },
1077
+ {
1078
+ "cell_type": "markdown",
1079
+ "source": [
1080
+ "## Local Inference on GPU\n",
1081
+ "Model page: https://huggingface.co/microsoft/phi-2\n",
1082
+ "\n",
1083
+ "⚠️ If the generated code snippets do not work, please open an issue on either the [model repo](https://huggingface.co/microsoft/phi-2)\n",
1084
+ "\t\t\tand/or on [huggingface.js](https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/src/model-libraries-snippets.ts) 🙏"
1085
+ ],
1086
+ "metadata": {
1087
+ "id": "qhYDm6yk-UEj"
1088
+ }
1089
+ },
1090
+ {
1091
+ "cell_type": "code",
1092
+ "source": [
1093
+ "from transformers import pipeline, AutoTokenizer, AutoModelForCausalLM, AutoConfig, BitsAndBytesConfig\n",
1094
+ "import torch\n",
1095
+ "\n",
1096
+ "model_name = \"AlexKitipov/phi-2\"\n",
1097
+ "\n",
1098
+ "# Load tokenizer and set pad_token\n",
1099
+ "tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)\n",
1100
+ "tokenizer.pad_token = tokenizer.eos_token\n",
1101
+ "\n",
1102
+ "# Load model configuration and set pad_token_id\n",
1103
+ "config = AutoConfig.from_pretrained(model_name, trust_remote_code=True)\n",
1104
+ "config.pad_token_id = tokenizer.eos_token_id\n",
1105
+ "\n",
1106
+ "# Configure 8-bit quantization\n",
1107
+ "quantization_config = BitsAndBytesConfig(load_in_8bit=True)\n",
1108
+ "\n",
1109
+ "# Load model with the modified configuration, device_map, and quantization config\n",
1110
+ "model = AutoModelForCausalLM.from_pretrained(\n",
1111
+ " model_name,\n",
1112
+ " config=config,\n",
1113
+ " trust_remote_code=True,\n",
1114
+ " device_map=\"auto\", # Automatically manage device placement (GPU/CPU)\n",
1115
+ " quantization_config=quantization_config # Apply 8-bit quantization\n",
1116
+ ")\n",
1117
+ "\n",
1118
+ "# Create the pipeline with the correctly loaded model and tokenizer\n",
1119
+ "# device_map=\"auto\" already handles device placement, so explicit .to('cuda') is not needed\n",
1120
+ "pipe = pipeline(\"text-generation\", model=model, tokenizer=tokenizer)\n",
1121
+ "print(\"Pipeline initialized successfully.\")"
1122
+ ],
1123
+ "metadata": {
1124
+ "colab": {
1125
+ "base_uri": "https://localhost:8080/",
1126
+ "height": 66,
1127
+ "referenced_widgets": [
1128
+ "53f437ce9ce144bcaa2dcf90027099b5",
1129
+ "d3e4d83800f74ba2b5f1841d50ff03f9",
1130
+ "d776ff80fc404113a7072668f9cbfe65",
1131
+ "a73630b02c614c8f81c1f06155276df0",
1132
+ "edfe23b707274d2e927177e3a7b062cf",
1133
+ "d27a2cd0a69d48b3a95bcf1bf5ab32c0",
1134
+ "8bef3d5dd0704fca92d05cfdd5a207f9",
1135
+ "61d00629597f4e92aaef0f08ce523390",
1136
+ "99b2738ceb0b437a87af4a845ef29434",
1137
+ "92e1572d00244e42819122ac3a9d3521",
1138
+ "7f7005e525f74c7389847139207d1f10"
1139
+ ]
1140
+ },
1141
+ "id": "b9Ly2RUM-UFe",
1142
+ "outputId": "f7f51bff-d702-4440-bb8d-2d1fa37fb303"
1143
+ },
1144
+ "execution_count": 22,
1145
+ "outputs": [
1146
+ {
1147
+ "output_type": "display_data",
1148
+ "data": {
1149
+ "text/plain": [
1150
+ "Loading weights: 0%| | 0/453 [00:00<?, ?it/s]"
1151
+ ],
1152
+ "application/vnd.jupyter.widget-view+json": {
1153
+ "version_major": 2,
1154
+ "version_minor": 0,
1155
+ "model_id": "53f437ce9ce144bcaa2dcf90027099b5"
1156
+ }
1157
+ },
1158
+ "metadata": {}
1159
+ },
1160
+ {
1161
+ "output_type": "stream",
1162
+ "name": "stdout",
1163
+ "text": [
1164
+ "Pipeline initialized successfully.\n"
1165
+ ]
1166
+ }
1167
+ ]
1168
+ },
1169
+ {
1170
+ "cell_type": "code",
1171
+ "source": [
1172
+ "from transformers import AutoTokenizer, AutoModelForCausalLM, AutoConfig, BitsAndBytesConfig\n",
1173
+ "import torch\n",
1174
+ "\n",
1175
+ "# Load tokenizer first\n",
1176
+ "tokenizer = AutoTokenizer.from_pretrained(\"AlexKitipov/phi-2\", trust_remote_code=True)\n",
1177
+ "\n",
1178
+ "# Set pad_token for tokenizer using eos_token\n",
1179
+ "tokenizer.pad_token = tokenizer.eos_token\n",
1180
+ "\n",
1181
+ "# Load model configuration separately to ensure pad_token_id is set before model initialization\n",
1182
+ "config = AutoConfig.from_pretrained(\"AlexKitipov/phi-2\", trust_remote_code=True)\n",
1183
+ "\n",
1184
+ "# Explicitly set pad_token_id in the config\n",
1185
+ "config.pad_token_id = tokenizer.eos_token_id\n",
1186
+ "\n",
1187
+ "# Configure 8-bit quantization with CPU offload for 32-bit modules if needed\n",
1188
+ "quantization_config = BitsAndBytesConfig(load_in_8bit=True, llm_int8_enable_fp32_cpu_offload=True)\n",
1189
+ "\n",
1190
+ "model = AutoModelForCausalLM.from_pretrained(\n",
1191
+ " \"AlexKitipov/phi-2\",\n",
1192
+ " config=config, # Pass the modified config here\n",
1193
+ " trust_remote_code=True,\n",
1194
+ " device_map=\"cpu\", # Force loading entirely on CPU for testing memory constraints\n",
1195
+ " quantization_config=quantization_config # Apply 8-bit quantization with potential CPU offload\n",
1196
+ ")"
1197
+ ],
1198
+ "metadata": {
1199
+ "colab": {
1200
+ "base_uri": "https://localhost:8080/",
1201
+ "height": 49,
1202
+ "referenced_widgets": [
1203
+ "36a690fc6d4d45289ca24029ece56dd2",
1204
+ "a9e2f204052645f6ac20b99e3348e07b",
1205
+ "72f98858f4f74e50a102b3307c8a1a3b",
1206
+ "c1dbf5a94d9546cfb0639702dad01ab8",
1207
+ "15acdb37b2c444fea8f0315e2ca7da0b",
1208
+ "dd688120d7a449699de136c48e084b69",
1209
+ "e97172be05b84455b57f921401116412",
1210
+ "627dca520f104b3e86ac63fe006b400a",
1211
+ "53edd077eb0b46eea5cfbff69ff199bc",
1212
+ "d8c681bdf67b48899bd1cc483b03a4e9",
1213
+ "21243c6ca4754573b3cc5b900a8efa0c"
1214
+ ]
1215
+ },
1216
+ "id": "q5AOq1_Z-UFu",
1217
+ "outputId": "1433d112-640c-45b5-dfee-feef3f6088ee"
1218
+ },
1219
+ "execution_count": 25,
1220
+ "outputs": [
1221
+ {
1222
+ "output_type": "display_data",
1223
+ "data": {
1224
+ "text/plain": [
1225
+ "Loading weights: 0%| | 0/453 [00:00<?, ?it/s]"
1226
+ ],
1227
+ "application/vnd.jupyter.widget-view+json": {
1228
+ "version_major": 2,
1229
+ "version_minor": 0,
1230
+ "model_id": "36a690fc6d4d45289ca24029ece56dd2"
1231
+ }
1232
+ },
1233
+ "metadata": {}
1234
+ }
1235
+ ]
1236
+ },
1237
+ {
1238
+ "cell_type": "markdown",
1239
+ "source": [
1240
+ "## Remote Inference via Inference Providers\n",
1241
+ "Ensure you have a valid **HF_TOKEN** set in your environment. You can get your token from [your settings page](https://huggingface.co/settings/tokens). Note: running this may incur charges above the free tier.\n",
1242
+ "The following Python example shows how to run the model remotely on HF Inference Providers, automatically selecting an available inference provider for you.\n",
1243
+ "For more information on how to use the Inference Providers, please refer to our [documentation and guides](https://huggingface.co/docs/inference-providers/en/index)."
1244
+ ],
1245
+ "metadata": {
1246
+ "id": "IRyu8RyM-UF-"
1247
+ }
1248
+ },
1249
+ {
1250
+ "cell_type": "markdown",
1251
+ "metadata": {
1252
+ "id": "029ed965"
1253
+ },
1254
+ "source": [
1255
+ "## Gradio Interface for Phi-2 Chat"
1256
+ ]
1257
+ },
1258
+ {
1259
+ "cell_type": "code",
1260
+ "metadata": {
1261
+ "id": "b8b69d12"
1262
+ },
1263
+ "source": [
1264
+ "# Install Gradio library\n",
1265
+ "!pip install gradio -q"
1266
+ ],
1267
+ "execution_count": 4,
1268
+ "outputs": []
1269
+ },
1270
+ {
1271
+ "cell_type": "code",
1272
+ "metadata": {
1273
+ "colab": {
1274
+ "base_uri": "https://localhost:8080/",
1275
+ "height": 66,
1276
+ "referenced_widgets": [
1277
+ "5f74f3acaa9840dfb91de8336143e488",
1278
+ "ecf38f30b070475b881d62336e48dba5",
1279
+ "bc6a478c3b76438b8e92b71fb9b26b3b",
1280
+ "e0dc86e3345141fd860be18318472eb1",
1281
+ "fa9902cd507b442c9e98f9ee64d599e6",
1282
+ "2825815747b04327b5a16adc431f9bd5",
1283
+ "5a66230e1d6b437a925f1b9cf9d011a5",
1284
+ "d5de5f95ec3142e696571ff7fd588ed9",
1285
+ "cd9939d1217644b482d05814d23d788a",
1286
+ "b10968146d5f4c6a864412e71fe2d2f9",
1287
+ "a2be91b533d344c1bd3fa5deddaa6be5"
1288
+ ]
1289
+ },
1290
+ "id": "b5a619ac",
1291
+ "outputId": "9013177a-1d7f-4b37-b18b-b3a3bd478c7c"
1292
+ },
1293
+ "source": [
1294
+ "import gradio as gr\n",
1295
+ "from transformers import pipeline, AutoTokenizer, AutoModelForCausalLM, AutoConfig, BitsAndBytesConfig\n",
1296
+ "import torch\n",
1297
+ "\n",
1298
+ "print(\"Loading Phi-2 model and tokenizer...\")\n",
1299
+ "\n",
1300
+ "# Load tokenizer first\n",
1301
+ "tokenizer = AutoTokenizer.from_pretrained(\"AlexKitipov/phi-2\", trust_remote_code=True)\n",
1302
+ "\n",
1303
+ "# Set pad_token for tokenizer using eos_token\n",
1304
+ "tokenizer.pad_token = tokenizer.eos_token\n",
1305
+ "\n",
1306
+ "# Load model configuration separately to ensure pad_token_id is set before model initialization\n",
1307
+ "config = AutoConfig.from_pretrained(\"AlexKitipov/phi-2\", trust_remote_code=True)\n",
1308
+ "\n",
1309
+ "# Explicitly set pad_token_id in the config, as Phi-2's config might not have it by default\n",
1310
+ "# The model's internal structure expects this attribute to be present.\n",
1311
+ "config.pad_token_id = tokenizer.eos_token_id\n",
1312
+ "\n",
1313
+ "# Configure 8-bit quantization with CPU offload for 32-bit modules if needed\n",
1314
+ "quantization_config = BitsAndBytesConfig(load_in_8bit=True, llm_int8_enable_fp32_cpu_offload=True)\n",
1315
+ "\n",
1316
+ "model = AutoModelForCausalLM.from_pretrained(\n",
1317
+ " \"AlexKitipov/phi-2\",\n",
1318
+ " config=config, # Pass the modified config here\n",
1319
+ " trust_remote_code=True,\n",
1320
+ " device_map=\"cpu\", # Force loading entirely on CPU for testing memory constraints\n",
1321
+ " quantization_config=quantization_config # Apply 8-bit quantization with potential CPU offload\n",
1322
+ ")\n",
1323
+ "\n",
1324
+ "# device_map=\"auto\" already handles device placement, so explicit .to('cuda') is not needed\n",
1325
+ "print(\"Model loaded successfully, device placement handled by device_map='cpu'.\")\n",
1326
+ "\n",
1327
+ "# Create a text generation pipeline and rename it to avoid conflict\n",
1328
+ "# No need to specify device here, as model placement is handled by device_map\n",
1329
+ "text_generator_pipeline = pipeline(\"text-generation\", model=model, tokenizer=tokenizer)\n",
1330
+ "\n",
1331
+ "print(\"Phi-2 model loaded successfully.\")"
1332
+ ],
1333
+ "execution_count": null,
1334
+ "outputs": [
1335
+ {
1336
+ "output_type": "stream",
1337
+ "name": "stdout",
1338
+ "text": [
1339
+ "Loading Phi-2 model and tokenizer...\n"
1340
+ ]
1341
+ },
1342
+ {
1343
+ "output_type": "display_data",
1344
+ "data": {
1345
+ "text/plain": [
1346
+ "Loading weights: 0%| | 0/453 [00:00<?, ?it/s]"
1347
+ ],
1348
+ "application/vnd.jupyter.widget-view+json": {
1349
+ "version_major": 2,
1350
+ "version_minor": 0,
1351
+ "model_id": "5f74f3acaa9840dfb91de8336143e488"
1352
+ }
1353
+ },
1354
+ "metadata": {}
1355
+ }
1356
+ ]
1357
+ },
1358
+ {
1359
+ "cell_type": "code",
1360
+ "metadata": {
1361
+ "id": "61f7b058"
1362
+ },
1363
+ "source": [
1364
+ "def predict(message, history):\n",
1365
+ " conversation_history = \"\"\n",
1366
+ " for human, assistant in history:\n",
1367
+ " conversation_history += f\"Human: {human}\\nAssistant: {assistant}\\n\"\n",
1368
+ " conversation_history += f\"Human: {message}\\nAssistant:\"\n",
1369
+ "\n",
1370
+ " outputs = text_generator_pipeline(\n",
1371
+ " conversation_history, # Pass conversation_history directly as a string to the renamed pipeline object\n",
1372
+ " max_new_tokens=200, # Generate up to 200 new tokens\n",
1373
+ " do_sample=True,\n",
1374
+ " temperature=0.7,\n",
1375
+ " top_k=50,\n",
1376
+ " top_p=0.95,\n",
1377
+ " eos_token_id=tokenizer.eos_token_id # Stop generation at end-of-sequence token\n",
1378
+ " )\n",
1379
+ " generated_text = outputs[0]['generated_text']\n",
1380
+ "\n",
1381
+ " # Extract only the assistant's response part\n",
1382
+ " assistant_response = generated_text.split(\"Assistant:\")[-1].strip()\n",
1383
+ " # Remove the last user input from the response if the model repeats it\n",
1384
+ " if assistant_response.startswith(message):\n",
1385
+ " assistant_response = assistant_response[len(message):].strip()\n",
1386
+ "\n",
1387
+ " return assistant_response"
1388
+ ],
1389
+ "execution_count": 1,
1390
+ "outputs": []
1391
+ },
1392
+ {
1393
+ "cell_type": "markdown",
1394
+ "metadata": {
1395
+ "id": "cbdf81f5"
1396
+ },
1397
+ "source": [
1398
+ "Now, let's launch the Gradio chat interface. Click the public URL to interact with the model."
1399
+ ]
1400
+ },
1401
+ {
1402
+ "cell_type": "code",
1403
+ "metadata": {
1404
+ "colab": {
1405
+ "base_uri": "https://localhost:8080/",
1406
+ "height": 228
1407
+ },
1408
+ "id": "63e656fe",
1409
+ "outputId": "9c4a86ef-106d-4c40-fd4d-2df52958007e"
1410
+ },
1411
+ "source": [
1412
+ "gr.ChatInterface(\n",
1413
+ " predict,\n",
1414
+ " chatbot=gr.Chatbot(height=500), # Make the chatbot window larger\n",
1415
+ " textbox=gr.Textbox(placeholder=\"Ask me a question\", container=False, scale=7),\n",
1416
+ " title=\"Chat with Phi-2\",\n",
1417
+ " description=\"Interact with the Microsoft Phi-2 model. Ask questions, have conversations, or experiment with its generative capabilities!\",\n",
1418
+ " theme=\"soft\", # A pleasant theme\n",
1419
+ " examples=[\"Tell me a short story.\", \"Explain quantum physics simply.\", \"What is the capital of France?\"],\n",
1420
+ " cache_examples=False\n",
1421
+ ").launch(debug=True, share=True)"
1422
+ ],
1423
+ "execution_count": 2,
1424
+ "outputs": [
1425
+ {
1426
+ "output_type": "error",
1427
+ "ename": "NameError",
1428
+ "evalue": "name 'gr' is not defined",
1429
+ "traceback": [
1430
+ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
1431
+ "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)",
1432
+ "\u001b[0;32m/tmp/ipykernel_11865/638742562.py\u001b[0m in \u001b[0;36m<cell line: 0>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m gr.ChatInterface(\n\u001b[0m\u001b[1;32m 2\u001b[0m \u001b[0mpredict\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0mchatbot\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mgr\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mChatbot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mheight\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m500\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;31m# Make the chatbot window larger\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0mtextbox\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mgr\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mTextbox\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mplaceholder\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"Ask me a question\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcontainer\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mscale\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m7\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0mtitle\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"Chat with Phi-2\"\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
1433
+ "\u001b[0;31mNameError\u001b[0m: name 'gr' is not defined"
1434
+ ]
1435
+ }
1436
+ ]
1437
+ },
1438
+ {
1439
+ "cell_type": "code",
1440
+ "metadata": {
1441
+ "id": "61f7b008"
1442
+ },
1443
+ "source": [
1444
+ "def predict(message, history):\n",
1445
+ " conversation_history = \"\"\n",
1446
+ " for human, assistant in history:\n",
1447
+ " conversation_history += f\"Human: {human}\\nAssistant: {assistant}\\n\"\n",
1448
+ " conversation_history += f\"Human: {message}\\nAssistant:\"\n",
1449
+ "\n",
1450
+ " outputs = pipe(\n",
1451
+ " conversation_history, # Pass conversation_history directly as a string\n",
1452
+ " max_new_tokens=200, # Generate up to 200 new tokens\n",
1453
+ " do_sample=True,\n",
1454
+ " temperature=0.7,\n",
1455
+ " top_k=50,\n",
1456
+ " top_p=0.95,\n",
1457
+ " eos_token_id=tokenizer.eos_token_id # Stop generation at end-of-sequence token\n",
1458
+ " )\n",
1459
+ " generated_text = outputs[0]['generated_text']\n",
1460
+ "\n",
1461
+ " # Extract only the assistant's response part\n",
1462
+ " assistant_response = generated_text.split(\"Assistant:\")[-1].strip()\n",
1463
+ " # Remove the last user input from the response if the model repeats it\n",
1464
+ " if assistant_response.startswith(message):\n",
1465
+ " assistant_response = assistant_response[len(message):].strip()\n",
1466
+ "\n",
1467
+ " return assistant_response"
1468
+ ],
1469
+ "execution_count": 3,
1470
+ "outputs": []
1471
+ },
1472
+ {
1473
+ "cell_type": "code",
1474
+ "source": [
1475
+ "import os\n",
1476
+ "os.environ['HF_TOKEN'] = 'YOUR_TOKEN_HERE'"
1477
+ ],
1478
+ "metadata": {
1479
+ "id": "leeUFqBD-UF-"
1480
+ },
1481
+ "execution_count": 4,
1482
+ "outputs": []
1483
+ },
1484
+ {
1485
+ "cell_type": "code",
1486
+ "source": [
1487
+ "import os\n",
1488
+ "from huggingface_hub import InferenceClient\n",
1489
+ "\n",
1490
+ "client = InferenceClient(\n",
1491
+ " provider=\"auto\",\n",
1492
+ " api_key=os.environ[\"HF_TOKEN\"],\n",
1493
+ ")\n",
1494
+ "\n",
1495
+ "completion = client.chat.completions.create(\n",
1496
+ " model=\"AlexKitipov/phi-2\",\n",
1497
+ " messages=\"\\\"Can you please let us know more details about your \\\"\",\n",
1498
+ ")\n",
1499
+ "\n",
1500
+ "print(completion.choices[0].message)"
1501
+ ],
1502
+ "metadata": {
1503
+ "colab": {
1504
+ "base_uri": "https://localhost:8080/",
1505
+ "height": 391
1506
+ },
1507
+ "id": "Zmof51oV-UGN",
1508
+ "outputId": "0b2ad44e-84f6-49a4-c4b7-0267d91f0438"
1509
+ },
1510
+ "execution_count": 5,
1511
+ "outputs": [
1512
+ {
1513
+ "output_type": "error",
1514
+ "ename": "ValueError",
1515
+ "evalue": "Cannot select auto-router when using non-Hugging Face API key.",
1516
+ "traceback": [
1517
+ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
1518
+ "\u001b[0;31mValueError\u001b[0m Traceback (most recent call last)",
1519
+ "\u001b[0;32m/tmp/ipykernel_11865/780418629.py\u001b[0m in \u001b[0;36m<cell line: 0>\u001b[0;34m()\u001b[0m\n\u001b[1;32m 7\u001b[0m )\n\u001b[1;32m 8\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 9\u001b[0;31m completion = client.chat.completions.create(\n\u001b[0m\u001b[1;32m 10\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"AlexKitipov/phi-2\"\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 11\u001b[0m \u001b[0mmessages\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"\\\"Can you please let us know more details about your \\\"\"\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
1520
+ "\u001b[0;32m/usr/local/lib/python3.12/dist-packages/huggingface_hub/inference/_client.py\u001b[0m in \u001b[0;36mchat_completion\u001b[0;34m(self, messages, model, stream, frequency_penalty, logit_bias, logprobs, max_tokens, n, presence_penalty, response_format, seed, stop, stream_options, temperature, tool_choice, tool_prompt, tools, top_logprobs, top_p, extra_body)\u001b[0m\n\u001b[1;32m 920\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mextra_body\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 921\u001b[0m }\n\u001b[0;32m--> 922\u001b[0;31m request_parameters = provider_helper.prepare_request(\n\u001b[0m\u001b[1;32m 923\u001b[0m \u001b[0minputs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mmessages\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 924\u001b[0m \u001b[0mparameters\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mparameters\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
1521
+ "\u001b[0;32m/usr/local/lib/python3.12/dist-packages/huggingface_hub/inference/_providers/_common.py\u001b[0m in \u001b[0;36mprepare_request\u001b[0;34m(self, inputs, parameters, headers, model, api_key, extra_payload)\u001b[0m\n\u001b[1;32m 100\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 101\u001b[0m \u001b[0;31m# routed URL if HF token, or direct URL (to customize in '_prepare_route' in subclasses)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 102\u001b[0;31m \u001b[0murl\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_prepare_url\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mapi_key\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mprovider_mapping_info\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprovider_id\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 103\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 104\u001b[0m \u001b[0;31m# prepare payload (to customize in subclasses)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
1522
+ "\u001b[0;32m/usr/local/lib/python3.12/dist-packages/huggingface_hub/inference/_providers/_common.py\u001b[0m in \u001b[0;36m_prepare_url\u001b[0;34m(self, api_key, mapped_model)\u001b[0m\n\u001b[1;32m 214\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 215\u001b[0m Usually not overwritten in subclasses.\"\"\"\n\u001b[0;32m--> 216\u001b[0;31m \u001b[0mbase_url\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_prepare_base_url\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mapi_key\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 217\u001b[0m \u001b[0mroute\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_prepare_route\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmapped_model\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mapi_key\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 218\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0;34mf\"{base_url.rstrip('/')}/{route.lstrip('/')}\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
1523
+ "\u001b[0;32m/usr/local/lib/python3.12/dist-packages/huggingface_hub/inference/_providers/_common.py\u001b[0m in \u001b[0;36m_prepare_base_url\u001b[0;34m(self, api_key)\u001b[0m\n\u001b[1;32m 300\u001b[0m \u001b[0;31m# Route to the proxy if the api_key is a HF TOKEN\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 301\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mapi_key\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstartswith\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"hf_\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 302\u001b[0;31m \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Cannot select auto-router when using non-Hugging Face API key.\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 303\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 304\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbase_url\u001b[0m \u001b[0;31m# No `/auto` suffix in the URL\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
1524
+ "\u001b[0;31mValueError\u001b[0m: Cannot select auto-router when using non-Hugging Face API key."
1525
+ ]
1526
+ }
1527
+ ]
1528
+ }
1529
+ ]
1530
+ }