syedahmedsoftware commited on
Commit
163fcb6
·
verified ·
1 Parent(s): a1b291c

README.md edited

Browse files
Files changed (1) hide show
  1. README.md +343 -6
README.md CHANGED
@@ -1,7 +1,344 @@
1
- ---
2
- library_name: transformers
3
- pipeline_tag: text-generation
4
- base_model:
5
- - meta-llama/Meta-Llama-3.1-8B
6
- ---
 
 
 
 
 
 
 
 
 
 
7
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {\rtf1\ansi\ansicpg1252\cocoartf2639
2
+ \cocoatextscaling0\cocoaplatform0{\fonttbl\f0\froman\fcharset0 Times-Bold;\f1\froman\fcharset0 Times-Roman;\f2\fmodern\fcharset0 Courier;
3
+ \f3\fmodern\fcharset0 Courier-Bold;\f4\fnil\fcharset0 Menlo-Regular;\f5\fnil\fcharset0 AppleColorEmoji;
4
+ }
5
+ {\colortbl;\red255\green255\blue255;\red0\green0\blue0;\red0\green0\blue233;\red109\green109\blue109;
6
+ \red0\green0\blue0;}
7
+ {\*\expandedcolortbl;;\cssrgb\c0\c0\c0;\cssrgb\c0\c0\c93333;\cssrgb\c50196\c50196\c50196;
8
+ \csgray\c0;}
9
+ {\*\listtable{\list\listtemplateid1\listhybrid{\listlevel\levelnfc23\levelnfcn23\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{disc\}}{\leveltext\leveltemplateid1\'01\uc0\u8226 ;}{\levelnumbers;}\fi-360\li720\lin720 }{\listname ;}\listid1}
10
+ {\list\listtemplateid2\listhybrid{\listlevel\levelnfc23\levelnfcn23\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{disc\}}{\leveltext\leveltemplateid101\'01\uc0\u8226 ;}{\levelnumbers;}\fi-360\li720\lin720 }{\listname ;}\listid2}
11
+ {\list\listtemplateid3\listhybrid{\listlevel\levelnfc23\levelnfcn23\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{disc\}}{\leveltext\leveltemplateid201\'01\uc0\u8226 ;}{\levelnumbers;}\fi-360\li720\lin720 }{\listname ;}\listid3}
12
+ {\list\listtemplateid4\listhybrid{\listlevel\levelnfc23\levelnfcn23\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{disc\}}{\leveltext\leveltemplateid301\'01\uc0\u8226 ;}{\levelnumbers;}\fi-360\li720\lin720 }{\listname ;}\listid4}}
13
+ {\*\listoverridetable{\listoverride\listid1\listoverridecount0\ls1}{\listoverride\listid2\listoverridecount0\ls2}{\listoverride\listid3\listoverridecount0\ls3}{\listoverride\listid4\listoverridecount0\ls4}}
14
+ \margl1440\margr1440\vieww24620\viewh8360\viewkind0
15
+ \deftab720
16
+ \pard\pardeftab720\sa321\partightenfactor0
17
 
18
+ \f0\b\fs48 \cf0 \expnd0\expndtw0\kerning0
19
+ \outl0\strokewidth0 \strokec2 Q2 Debugging LLMs (Friendli AI Take-Home)\
20
+ \pard\pardeftab720\sa240\partightenfactor0
21
+
22
+ \fs24 \cf0 Fixed Model Repo:
23
+ \f1\b0 \uc0\u8232 {\field{\*\fldinst{HYPERLINK "https://huggingface.co/syedahmedsoftware/broken-model-fixed"}}{\fldrslt \cf3 \ul \ulc3 \strokec3 https://huggingface.co/syedahmedsoftware/broken-model-fixed}}\
24
+
25
+ \f0\b Original Model:
26
+ \f1\b0 \uc0\u8232 {\field{\*\fldinst{HYPERLINK "https://huggingface.co/yunmorning/broken-model?utm_source=chatgpt.com"}}{\fldrslt \cf3 \ul \ulc3 \strokec3 https://huggingface.co/yunmorning/broken-model}}\
27
+ This repository contains the
28
+ \f0\b minimal, production-safe fixes
29
+ \f1\b0 required to make the original model usable behind an OpenAI-compatible
30
+ \f2\fs26 /chat/completions
31
+ \f1\fs24 API server.\uc0\u8232
32
+ \f0\b No model weights were modified.
33
+ \f1\b0 Only configuration metadata was corrected.\
34
+ \pard\pardeftab720\partightenfactor0
35
+ \cf0 \
36
+ \pard\pardeftab720\sa298\partightenfactor0
37
+
38
+ \f0\b\fs36 \cf0 Problem (a) \'97 Why inference failed\
39
+ \pard\pardeftab720\sa240\partightenfactor0
40
+
41
+ \f1\b0\fs24 \cf0 The original repository could not serve chat requests because
42
+ \f0\b two required metadata components were missing or incorrect
43
+ \f1\b0 .\
44
+ \pard\pardeftab720\sa280\partightenfactor0
45
+
46
+ \f0\b\fs28 \cf0 Root Causes\
47
+ \pard\pardeftab720\sa319\partightenfactor0
48
+
49
+ \fs24 \cf0 1) Missing
50
+ \f3\fs26 chat_template
51
+ \f0\fs24 (tokenizer_config.json)\
52
+ \pard\pardeftab720\sa240\partightenfactor0
53
+
54
+ \f1\b0 \cf0 Modern OpenAI-style runtimes rely on:\
55
+ python\
56
+ \pard\pardeftab720\partightenfactor0
57
+
58
+ \f2\fs26 \cf0 tokenizer.apply_chat_template(messages)\
59
+ \
60
+ \
61
+ \pard\pardeftab720\sa240\partightenfactor0
62
+
63
+ \f1\fs24 \cf0 The original model
64
+ \f0\b did not define a chat template
65
+ \f1\b0 , so:\
66
+ \pard\tx220\tx720\pardeftab720\li720\fi-720\sa240\partightenfactor0
67
+ \ls1\ilvl0\cf0 \kerning1\expnd0\expndtw0 \outl0\strokewidth0 {\listtext \uc0\u8226 }\expnd0\expndtw0\kerning0
68
+ \outl0\strokewidth0 \strokec2 Chat messages could not be rendered into prompts\
69
+ \ls1\ilvl0
70
+ \f2\fs26 \kerning1\expnd0\expndtw0 \outl0\strokewidth0 {\listtext \uc0\u8226 }\expnd0\expndtw0\kerning0
71
+ \outl0\strokewidth0 \strokec2 /chat/completions
72
+ \f1\fs24 servers had no deterministic formatting\
73
+ \ls1\ilvl0\kerning1\expnd0\expndtw0 \outl0\strokewidth0 {\listtext \uc0\u8226 }\expnd0\expndtw0\kerning0
74
+ \outl0\strokewidth0 \strokec2 Inference failed or produced undefined behavior\
75
+ \pard\pardeftab720\partightenfactor0
76
+
77
+ \f2\fs26 \cf0 \strokec2 \
78
+ \pard\pardeftab720\sa319\partightenfactor0
79
+
80
+ \f0\b\fs24 \cf0 2) Missing/incorrect
81
+ \f3\fs26 pad_token_id
82
+ \f0\fs24 (config.json)\
83
+ \pard\pardeftab720\sa240\partightenfactor0
84
+
85
+ \f1\b0 \cf0 Production inference uses:\
86
+ \pard\tx220\tx720\pardeftab720\li720\fi-720\sa240\partightenfactor0
87
+ \ls2\ilvl0\cf0 \kerning1\expnd0\expndtw0 \outl0\strokewidth0 {\listtext \uc0\u8226 }\expnd0\expndtw0\kerning0
88
+ \outl0\strokewidth0 \strokec2 batched decoding\
89
+ \ls2\ilvl0\kerning1\expnd0\expndtw0 \outl0\strokewidth0 {\listtext \uc0\u8226 }\expnd0\expndtw0\kerning0
90
+ \outl0\strokewidth0 \strokec2 attention masking\
91
+ \ls2\ilvl0\kerning1\expnd0\expndtw0 \outl0\strokewidth0 {\listtext \uc0\u8226 }\expnd0\expndtw0\kerning0
92
+ \outl0\strokewidth0 \strokec2 padded sequences\
93
+ \pard\pardeftab720\sa240\partightenfactor0
94
+ \cf0 \strokec2 The original
95
+ \f2\fs26 config.json
96
+ \f1\fs24 did not define a valid
97
+ \f2\fs26 pad_token_id
98
+ \f1\fs24 , making batching unsafe and causing runtime instability.\
99
+ \pard\pardeftab720\partightenfactor0
100
+
101
+ \f2\fs26 \cf0 \
102
+ \
103
+ \pard\pardeftab720\sa298\partightenfactor0
104
+
105
+ \f0\b\fs36 \cf0 Minimal Fixes Applied (No Weight Changes)\
106
+
107
+ \itap1\trowd \taflags0 \trgaph108\trleft-108 \trbrdrt\brdrnil \trbrdrl\brdrnil \trbrdrr\brdrnil
108
+ \clvertalc \clshdrawnil \clwWidth3432\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx2880
109
+ \clvertalc \clshdrawnil \clwWidth4607\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx5760
110
+ \clvertalc \clshdrawnil \clwWidth4118\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx8640
111
+ \pard\intbl\itap1\pardeftab720\qc\partightenfactor0
112
+
113
+ \fs24 \cf0 \strokec2 File\cell
114
+ \pard\intbl\itap1\pardeftab720\qc\partightenfactor0
115
+ \cf0 Change\cell
116
+ \pard\intbl\itap1\pardeftab720\qc\partightenfactor0
117
+ \cf0 Why\cell \row
118
+
119
+ \itap1\trowd \taflags0 \trgaph108\trleft-108 \trbrdrl\brdrnil \trbrdrr\brdrnil
120
+ \clvertalc \clshdrawnil \clwWidth3432\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx2880
121
+ \clvertalc \clshdrawnil \clwWidth4607\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx5760
122
+ \clvertalc \clshdrawnil \clwWidth4118\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx8640
123
+ \pard\intbl\itap1\pardeftab720\partightenfactor0
124
+
125
+ \f2\b0\fs26 \cf0 tokenizer_config.json
126
+ \f1\fs24 \cell
127
+ \pard\intbl\itap1\pardeftab720\partightenfactor0
128
+ \cf0 Added ChatML-style
129
+ \f2\fs26 chat_template
130
+ \f1\fs24 \cell
131
+ \pard\intbl\itap1\pardeftab720\partightenfactor0
132
+ \cf0 Required for OpenAI-style chat formatting\cell \row
133
+
134
+ \itap1\trowd \taflags0 \trgaph108\trleft-108 \trbrdrl\brdrnil \trbrdrr\brdrnil
135
+ \clvertalc \clshdrawnil \clwWidth3432\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx2880
136
+ \clvertalc \clshdrawnil \clwWidth4607\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx5760
137
+ \clvertalc \clshdrawnil \clwWidth4118\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx8640
138
+ \pard\intbl\itap1\pardeftab720\partightenfactor0
139
+
140
+ \f2\fs26 \cf0 config.json
141
+ \f1\fs24 \cell
142
+ \pard\intbl\itap1\pardeftab720\partightenfactor0
143
+ \cf0 Set
144
+ \f2\fs26 pad_token_id
145
+ \f1\fs24 to tokenizer\'92s pad token\cell
146
+ \pard\intbl\itap1\pardeftab720\partightenfactor0
147
+ \cf0 Enables safe batching and attention masks\cell \row
148
+
149
+ \itap1\trowd \taflags0 \trgaph108\trleft-108 \trbrdrl\brdrnil \trbrdrt\brdrnil \trbrdrr\brdrnil
150
+ \clvertalc \clshdrawnil \clwWidth3432\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx2880
151
+ \clvertalc \clshdrawnil \clwWidth4607\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx5760
152
+ \clvertalc \clshdrawnil \clwWidth4118\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx8640
153
+ \pard\intbl\itap1\pardeftab720\partightenfactor0
154
+
155
+ \f2\fs26 \cf0 generation_config.json
156
+ \f1\fs24 \cell
157
+ \pard\intbl\itap1\pardeftab720\partightenfactor0
158
+ \cf0 Normalized pad/eos fields (kept defaults)\cell
159
+ \pard\intbl\itap1\pardeftab720\partightenfactor0
160
+ \cf0 Prevents decoding edge cases\cell \lastrow\row
161
+ \pard\pardeftab720\sa240\partightenfactor0
162
+ \cf0 \strokec2 \
163
+ No architecture, tokenizer vocab, or model weights were changed.\
164
+ \
165
+ \pard\pardeftab720\partightenfactor0
166
+
167
+ \f0\b \cf0 Verification (Remote)\
168
+ \
169
+ \
170
+ \pard\tx560\tx1120\tx1680\tx2240\tx2800\tx3360\tx3920\tx4480\tx5040\tx5600\tx6160\tx6720\pardirnatural\partightenfactor0
171
+
172
+ \f4\b0\fs22 \cf5 \kerning1\expnd0\expndtw0 \CocoaLigature0 \outl0\strokewidth0 python - <<'PY'\
173
+ from transformers import AutoTokenizer\
174
+ \
175
+ repo_id = "syedahmedsoftware/broken-model-fixed"\
176
+ tok = AutoTokenizer.from_pretrained(repo_id, use_fast=True)\
177
+ \
178
+ print("tokenizer:", tok.__class__.__name__)\
179
+ print("has_chat_template:", bool(getattr(tok, "chat_template", None)))\
180
+ print("pad_token_id:", tok.pad_token_id, "eos_token_id:", tok.eos_token_id)\
181
+ \
182
+ messages = [\
183
+ \{"role":"system","content":"You are helpful."\},\
184
+ \{"role":"user","content":"Say hello."\}\
185
+ ]\
186
+ prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)\
187
+ print("
188
+ \f5 \uc0\u9989
189
+ \f4 REMOTE prompt renders. Preview:")\
190
+ print(prompt[:250])\
191
+ PY\
192
+ \
193
+ Expected output:\
194
+ \
195
+ tokenizer: Qwen2Tokenizer\
196
+ has_chat_template: True\
197
+ pad_token_id: 151643 eos_token_id: 151645\
198
+
199
+ \f5 \uc0\u9989
200
+ \f4 REMOTE prompt renders. Preview:\
201
+ <|im_start|>system\
202
+ You are helpful.<|im_end|>\
203
+ <|im_start|>user\
204
+ Say hello.<|im_end|>\
205
+ <|im_start|>assistant\
206
+ \
207
+ \
208
+ \
209
+ \pard\pardeftab720\sa298\partightenfactor0
210
+
211
+ \f0\b\fs36 \cf0 \expnd0\expndtw0\kerning0
212
+ \CocoaLigature1 \outl0\strokewidth0 \strokec2 Problem (b) \'97 Why
213
+ \f3\fs39 \strokec2 reasoning_effort
214
+ \f0\fs36 \strokec2 has no effect\
215
+ \pard\pardeftab720\sa280\partightenfactor0
216
+
217
+ \fs28 \cf0 Root Cause\
218
+ \pard\pardeftab720\sa240\partightenfactor0
219
+
220
+ \f2\b0\fs26 \cf0 reasoning_effort
221
+ \f1\fs24 is
222
+ \f0\b not a native Transformers generation parameter
223
+ \f1\b0 .\uc0\u8232 Unless the
224
+ \f0\b serving runtime
225
+ \f1\b0 explicitly maps it to real compute policies, it is silently ignored.\
226
+ The base model has
227
+ \f0\b no internal mechanism
228
+ \f1\b0 to interpret \'93effort.\'94\
229
+ \pard\pardeftab720\partightenfactor0
230
+ \cf0 \
231
+ \pard\pardeftab720\sa298\partightenfactor0
232
+
233
+ \f0\b\fs36 \cf0 What is required to make
234
+ \f3\fs39 \strokec2 reasoning_effort
235
+ \f0\fs36 \strokec2 meaningful\
236
+ \pard\pardeftab720\sa280\partightenfactor0
237
+
238
+ \fs28 \cf0 1) Runtime orchestration (required)\
239
+ \pard\pardeftab720\sa240\partightenfactor0
240
+
241
+ \f1\b0\fs24 \cf0 The server must map
242
+ \f2\fs26 reasoning_effort
243
+ \f1\fs24 to actual strategies, for example:\
244
+
245
+ \itap1\trowd \taflags0 \trgaph108\trleft-108 \trbrdrt\brdrnil \trbrdrl\brdrnil \trbrdrr\brdrnil
246
+ \clvertalc \clshdrawnil \clwWidth1246\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx4320
247
+ \clvertalc \clshdrawnil \clwWidth3074\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx8640
248
+ \pard\intbl\itap1\pardeftab720\qc\partightenfactor0
249
+
250
+ \f0\b \cf0 \strokec2 Effort Level\cell
251
+ \pard\intbl\itap1\pardeftab720\qc\partightenfactor0
252
+ \cf0 Runtime Policy\cell \row
253
+
254
+ \itap1\trowd \taflags0 \trgaph108\trleft-108 \trbrdrl\brdrnil \trbrdrr\brdrnil
255
+ \clvertalc \clshdrawnil \clwWidth1246\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx4320
256
+ \clvertalc \clshdrawnil \clwWidth3074\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx8640
257
+ \pard\intbl\itap1\pardeftab720\partightenfactor0
258
+
259
+ \f1\b0 \cf0 low\cell
260
+ \pard\intbl\itap1\pardeftab720\partightenfactor0
261
+ \cf0 single pass, greedy\cell \row
262
+
263
+ \itap1\trowd \taflags0 \trgaph108\trleft-108 \trbrdrl\brdrnil \trbrdrr\brdrnil
264
+ \clvertalc \clshdrawnil \clwWidth1246\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx4320
265
+ \clvertalc \clshdrawnil \clwWidth3074\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx8640
266
+ \pard\intbl\itap1\pardeftab720\partightenfactor0
267
+ \cf0 medium\cell
268
+ \pard\intbl\itap1\pardeftab720\partightenfactor0
269
+ \cf0 temperature + nucleus sampling\cell \row
270
+
271
+ \itap1\trowd \taflags0 \trgaph108\trleft-108 \trbrdrl\brdrnil \trbrdrr\brdrnil
272
+ \clvertalc \clshdrawnil \clwWidth1246\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx4320
273
+ \clvertalc \clshdrawnil \clwWidth3074\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx8640
274
+ \pard\intbl\itap1\pardeftab720\partightenfactor0
275
+ \cf0 high\cell
276
+ \pard\intbl\itap1\pardeftab720\partightenfactor0
277
+ \cf0 multi-sample + rerank\cell \row
278
+
279
+ \itap1\trowd \taflags0 \trgaph108\trleft-108 \trbrdrl\brdrnil \trbrdrt\brdrnil \trbrdrr\brdrnil
280
+ \clvertalc \clshdrawnil \clwWidth1246\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx4320
281
+ \clvertalc \clshdrawnil \clwWidth3074\clftsWidth3 \clmart10 \clmarl10 \clmarb10 \clmarr10 \clbrdrt\brdrnil \clbrdrl\brdrnil \clbrdrb\brdrnil \clbrdrr\brdrnil \clpadt20 \clpadl20 \clpadb20 \clpadr20 \gaph\cellx8640
282
+ \pard\intbl\itap1\pardeftab720\partightenfactor0
283
+ \cf0 very_high\cell
284
+ \pard\intbl\itap1\pardeftab720\partightenfactor0
285
+ \cf0 tree-of-thought / verifier loop\cell \lastrow\row
286
+ \pard\pardeftab720\sa240\partightenfactor0
287
+ \cf0 \strokec2 The runtime must log which policy is applied.\
288
+ \pard\pardeftab720\sa280\partightenfactor0
289
+
290
+ \f0\b\fs28 \cf0 2) Architectural support (one of)\
291
+ \pard\tx220\tx720\pardeftab720\li720\fi-720\sa240\partightenfactor0
292
+ \ls3\ilvl0
293
+ \f1\b0\fs24 \cf0 \kerning1\expnd0\expndtw0 \outl0\strokewidth0 {\listtext \uc0\u8226 }\expnd0\expndtw0\kerning0
294
+ \outl0\strokewidth0 \strokec2 Multi-pass verifier loop\
295
+ \ls3\ilvl0\kerning1\expnd0\expndtw0 \outl0\strokewidth0 {\listtext \uc0\u8226 }\expnd0\expndtw0\kerning0
296
+ \outl0\strokewidth0 \strokec2 Tree search / self-consistency\
297
+ \ls3\ilvl0\kerning1\expnd0\expndtw0 \outl0\strokewidth0 {\listtext \uc0\u8226 }\expnd0\expndtw0\kerning0
298
+ \outl0\strokewidth0 \strokec2 Reflection / critique agent\
299
+ \ls3\ilvl0\kerning1\expnd0\expndtw0 \outl0\strokewidth0 {\listtext \uc0\u8226 }\expnd0\expndtw0\kerning0
300
+ \outl0\strokewidth0 \strokec2 Budgeted decoding controller\
301
+ \ls3\ilvl0\kerning1\expnd0\expndtw0 \outl0\strokewidth0 {\listtext \uc0\u8226 }\expnd0\expndtw0\kerning0
302
+ \outl0\strokewidth0 \strokec2 Control-token trained model\
303
+ \pard\pardeftab720\sa240\partightenfactor0
304
+ \cf0 \strokec2 Without these,
305
+ \f2\fs26 reasoning_effort
306
+ \f1\fs24 remains a
307
+ \f0\b no-op
308
+ \f1\b0 .\
309
+ \pard\pardeftab720\partightenfactor0
310
+ \cf0 \
311
+ \pard\pardeftab720\sa298\partightenfactor0
312
+
313
+ \f0\b\fs36 \cf0 Final Notes\
314
+ \pard\pardeftab720\sa240\partightenfactor0
315
+
316
+ \f1\b0\fs24 \cf0 These fixes:\
317
+ \pard\tx220\tx720\pardeftab720\li720\fi-720\sa240\partightenfactor0
318
+ \ls4\ilvl0\cf0 \kerning1\expnd0\expndtw0 \outl0\strokewidth0 {\listtext \uc0\u8226 }\expnd0\expndtw0\kerning0
319
+ \outl0\strokewidth0 \strokec2 Restore deterministic chat formatting\
320
+ \ls4\ilvl0\kerning1\expnd0\expndtw0 \outl0\strokewidth0 {\listtext \uc0\u8226 }\expnd0\expndtw0\kerning0
321
+ \outl0\strokewidth0 \strokec2 Enable production-safe batching\
322
+ \ls4\ilvl0\kerning1\expnd0\expndtw0 \outl0\strokewidth0 {\listtext \uc0\u8226 }\expnd0\expndtw0\kerning0
323
+ \outl0\strokewidth0 \strokec2 Make the model compatible with OpenAI-style
324
+ \f2\fs26 /chat/completions
325
+ \f1\fs24 servers\
326
+ \pard\pardeftab720\sa240\partightenfactor0
327
+ \cf0 \strokec2 The model is now deployable in real inference environments.\
328
+ \pard\tx560\tx1120\tx1680\tx2240\tx2800\tx3360\tx3920\tx4480\tx5040\tx5600\tx6160\tx6720\pardirnatural\partightenfactor0
329
+
330
+ \f4\fs22 \cf5 \kerning1\expnd0\expndtw0 \CocoaLigature0 \outl0\strokewidth0 \
331
+ \
332
+ \pard\pardeftab720\partightenfactor0
333
+
334
+ \f0\b\fs24 \cf0 \expnd0\expndtw0\kerning0
335
+ \CocoaLigature1 \outl0\strokewidth0 \strokec2 \
336
+
337
+ \f1\b0 \
338
+ \pard\pardeftab720\partightenfactor0
339
+
340
+ \f2\fs26 \cf0 \
341
+ \
342
+ \
343
+ \
344
+ }